Sandia Research Corporation Building

Synthetic Task Environments

 

Synthetic tasks are "research tasks constructed by systematic abstraction from a corresponding real-world task" (Martin, Lyon, & Schreiber, 1998), p. 123). Performance on a synthetic task should exercise some of the same behavioral and cognitive skills associated with the real-world task. A STE (Synthetic Task Environment) provides the context for a suite of synthetic tasks. This environment offers a research platform that bridges the gap between controlled studies using artificial laboratory tasks and uncontrolled field studies on real tasks or using high-fidelity simulators. The design process usually follows from a cognitive task analysis to understand the field of practice to the resulting STE and the research agenda surrounding it.

 

STEs vs. Simulations

An STE can be considered a type of simulation, but philosophically differs from traditional simulations in terms of goals and resulting design decisions. Simulations typically recreate the work environment or the equipment or systems within that environment. An STE is "task centric" in that the goal is to recreate aspects of the task to differing degrees of fidelity. Thus, an STE may not have the "look and feel" of the operational environment, but instead requires the same thoughts and behavior of the operational task. Because tasks are often situated in rich environments, STEs often include simulations of systems required to support the task. However, the focus is on abstracting task features consistent with the purpose of the planned research associated with the STE and concomitant design objectives. As a result, several very different STEs can be based on the same real task by virtue of applying distinct filters, each associated with different objectives. Such is the case with the UAV task in which a variety of STEs have been developed that focus on various cognitive skills of individuals (e.g., Gugerty,Hall, & Tirre, 1998; Martin et al., 1998) and others. Our UAS STE focuses on team cognition.

 

In addition, simulations often replicate the environment at the expense of the simulation's flexibility as a research tool. Researchers are limited in the degree to which they can alter or control the simulation and the measures that they can derive from it.STEs, on the other hand, typically incorporate multiple task scenarios, and often the ability to manipulate aspects of task scenarios, as well as flexibility in measurement. This increased flexibility is not so much inherent to the concept of an STE, as demanded by researchers who appreciate the benefit of trading off some aspects of fidelity for research flexibility (e.g. Fowlkes, Dwyer, Oser, & Salas, 1998; Cannon-Bowers, Burns, Sals, & Pruitt, 1998). Researchers have cautioned against the use of simulations unguided by training principles or an understanding of the actual task requirements and have extolled the virtue of low-fidelity simulations that take such factors into account (Miller, Lehman, & Koedinger, 1999; Salas, Bowers, & Rhodenizer, 1998).

 

STEs and Fidelity

STEs, like high-fidelity simulations, can facilitate research in a safe and inexpensive setting and can also be used for task training and system design in support of tasks. They are also touted as providing a viable middle ground between overly-artificial lab research and uncontrollable field research (Brehmer & Dorner, 1993). In many ways, STEs seem like the best of both worlds -- the laboratory and the field. Alternatively, if they fail to meet the combined objectives of experimental control and sufficient representation of the task in question, they may instead capture the worst of both worlds. Perhaps the biggest criticism levied against STEs does not concern lack of experimental control, but rather the issue of lack of fidelity. However, the labeling of traditional simulations as high-fidelity and of STEs as low-fidelity, makes little sense. STEs have been described as low-fidelity simulations, as opposed to traditional equipment-centric simulations. Indeed, in terms of replicating the features of the equipment, STEs may represent a lower fidelity mock-up. The lack of fidelity is tied to more general concerns about low face validity, but perhaps more importantly, low external validity and therefore generalizeability to the situation of interest. The face validity issue is addressed by Salas et al. (1998) who argue that face validity may dictate acceptance by users, but not necessarily success as a training or research tool. In addition, the low external validity concern may seem sound on the surface, but may break down if fidelity is considered more broadly. Fidelity is generally the match between the research environment and the specific environment to which results are assumed to transfer. The match, however, can be based on a number of dimensions including the equipment and the task requirements. Thus, fidelity is not necessarily a single feature that is high-or-low for a particular simulation, but rather a multidimensional feature that can ultimately result in contexts of mixed fidelity. That is, a simulation may be faithful to the equipment, but not to the task requirements.  Therefore, under this multidimensional view of fidelity, the labeling of traditional simulations as high- fidelity and of STEs as low-fidelity, makes little sense. Instead,STEs are typically high-fidelity with respect to the task and low-fidelity with respect to the equipment. Traditional simulations may more often be associated with the opposite pattern.

Sandia Research specializes in the development of Synthetic Task Environment software and hardware to support research in Cognitive Engineering.

TEAMS (Team Environment And Measurement Suite) Software

 

The TEAMS-STE test-bed uses three participant workstations and an experimenter control station. The TEAMS-STE task software is an abstraction of a Predator UAS ground control station (GCS). This system is a three-team member task in which each team member is provided with distinct, though overlapping, training; has unique, yet interdependent roles; and is presented with unique and overlapping information during the mission. The overall goal of each mission is to fly the UAS to designated target areas and to take acceptable photos at these areas. The Air Vehicle Operator (AVO) controls airspeed, heading, and altitude, and monitors UAS systems. The Payload Operator (PLO) adjusts camera settings, takes photos, and monitors the camera equipment. The Data Exploitation, Mission Planning, and Communication Operator (DEMPC) oversees the mission and determines flight paths under various constraints. To successfully complete a mission, the team members need to share information with one another in a coordinated fashion.

 

The original TEAMS task software was developed in conjunction with Dr. Nancy Cooke's CERTT (Cognitive Engineering Research on Team Tasks) Laboratory, in 1997 at New Mexico State University (current Arizona State University) as a research facility for studying team performance and cognition in complex settings and it houses experimenter-friendly equipment to simulate these settings. This test-bed, configured to simulate a UAS (Unmanned Aerial Systems) ground control task, has supported more than 15 years of research on team cognition that has contributed to theory, measurement, and team applications including UAS crew coordination.

 

The task software, currently the fourth generation, is now available to other team researchers. In addition to supporting all-human teams, the software is also capable of supporting human-avatar teams as well.  The last experiment in the CERTT Laboratory used a synthetic teammate, developed by the the US Air Force Research Laboratory, in lieu of a human AVO.

ASU Student Experimenters using TEAMS Software

Software Features - Base Package

 

The base UAS task software package consists of:

 

Software Applications

  • AVO_1 Participant Software
  • AVO_2 Participant Software
  • PLO_1 Participant Software
  • PLO_2 Participant Software
  • DEMPC_1 Participant Software
  • DEMPC_2 Participant Software
  • Experimenter Software

- TCPIP Server

- System Control

- Output Database Writer

- Scoring System

  • Scenario Design Software
  • Data Complier Software (aggregates data from all teams by mission)

 

 

 

Other

  • Input SQL Database (MS Access)

- 1 Training Mission

- 9 Different Missions - each a different scenario)

  • Task Specific Training Materials (PowerPoint)

 

 

Manuals

  • AVO Operations Manual
  • PLO Operations Manual
  • DEMPC Operations Manual
  • Experimenter Operations Manual
  • Scenario Manual
  • Scoring Manual
  • Input Database Manual
  • Data Complier Manual

 

Participant Training

  • Less than 2 hours
  • Hands-On Training Scenario

 

Software Features - Options

 

Options to the base UAS task software package include:

 

Text Messaging Application

  • SQL Database recording
  • Records time sent
  • Records time read
  • Experimenter can temporarily disable specific Chat links (Road Blocks)
  • Word Corpus generated

 

VOIP Intercom Application

  • SQL Database recording - who is talking to who
  • Records time session started
  • Records time session stopped
  • Experimenter can temporarily disable specific Intercom links (Road Blocks)
  • Headsets with Push To Talk Buttons
  • On Screen party selector

 

Participant Monitoring Application

  • Can view two participant screens on one monitor
  • Can switch between participants (manually or timed)

 

 

 

Avatar Interface Control

  • Visual Studio Control
  • Handles socket communication with Experimenter
  • Is set to a single task (AVO, PLO, or DEMPC)
  • Raises an Event when new task-specific data is available
  • Uses a method to change a value (represents a user button press or mouse click)

 

Experimenter/Observer Application

  • Customizable via Input database
  • Records Coordination
  • Records Process
  • Records TEAM Situation Awareness
  • Records Communication Metrics
  • and more.

 

Additional Participant Applications Package

  • Informed Consent
  • Demographics
  • NASA TLX
  • Task Ratings
  • Personality Questionnaire
  • Debrief