Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Characteristics of fluent skills in a complex, dynamic problem-solving task

We examined critical characteristics of fluent cognitive skills, using the Georgia Tech Aegis Simulation Program, a tactical decision-making computer game that simulates tasks of an anti-air-warfare coordinator. To characterize learning, we adopted the unit-task analysis framework, in which a task is decomposed into several unit tasks that are further decomposed into functional-level subtasks. Our results showed that learning at a global level could be decomposed into learning smaller component tasks. Further, most learning was associated with a reduction in cognitive processes, in which people make inferences from the currently available information. Eye-movement data also revealed that the time spent on task-irrelevant regions of the display decreased more than did the time spent on task-relevant regions. In sum, although fluency in dynamic, complex problem solving was achieved by attaining efficiency in perceptual, motor, and cognitive processes, the magnitude of the gains depended on the preexisting fluency of the component skills. These results imply that a training program should decompose a task into its component skills and emphasize those components with which trainees have relatively little prior experience. Actual or potential applications of this research include learning and training of complex tasks as well as evaluation of performance on those tasks.     

Stress appraisals and training performance on a complex laboratory task

OBJECTIVE: We investigated whether training performance on a complex laboratory task differs for trainees whose stress appraisals denote challenge or threat. BACKGROUND: Past research outside a training context found better performance on tasks when stress appraisals denoted challenge (in which perceived situational demands are commensurate with perceived resources) as opposed to threat (in which perceived coping resources fall short of the demands of the stressor). METHOD: College students performed Space Fortress during 80 3-min practice trials, 20 tests, and posttests (retention, transfer, secondary task interference). Stress appraisals were measured with a two-item scale (Experiments 1-3) or an eight-item scale (Experiment 3) with (Study 1) or without (Experiments 2 and 3) brief hands-on experience. RESULTS: In all experiments, training improved performance and challenged trainees outperformed threatened trainees throughout training as well as on some baseline and posttraining tests. CONCLUSIONS: These studies are the first to document that stress appraisals predict training performance. They suggest that little information about a task is needed for appraisals to account for a significant amount of variance (11%) in training performance. Investigating the dynamic interplay of stress appraisals and training will increase the understanding of stress appraisals and of training. APPLICATION: Stress appraisals may improve training if used as a screening tool and/or by implementing interventions aimed at changing appraisals from threat to challenge.     

In-game assessment and training of nonverbal cognitive skills using TagTiles

We present a field study with a game for children called ‘Tap the little hedgehog’, which is played on the TagTiles console, a tangible electronic interface. The game was developed to train and assess cognitive skills and includes tasks which, in isolation, exhibit a high correlation with a number of subtests from the Wechsler Intelligence Scale for Children (WISC-III-NL). The tasks address a range of nonverbal skills by requiring children to perform different operations on abstract patterns such as copying, reproducing sequences from memory and mirroring patterns. In the current study, we tested whether these tasks kept their ability to address these skills if included in a gaming context, whether children are able to play the game independently and whether they are motivated to play the game. The results of the study support the hypothesis that nonverbal IQ-scores, as measured by the Wechsler Intelligence Scale for Children, can improve by training with a game. Hence, games like ‘Tap the little hedgehog’ can be used to train specific skills and serve as a screening tool for these skills. The results also confirm that children can play the game independently and that they enjoy it. We further found that children quickly learn how to play the game and use the interface.     

网络安全态势感知技术与系统

现如今网络规模正在逐渐壮大,网络结构也变得越来越复杂,各种网络病毒以及黑客攻击成为了普遍现象,过去的网络安全管理技术只是依靠防火墙、各种杀毒软件等单一的被动安全保护进行网络安全管理,在当今这个时代里早已无法满足网络安全的基本需求,基于这种原因,更加迫切的需要一种新的网络安全技术来进行预防和监控。而网络安全态势感知技术则很好的满足了这些需求,它可以对某段时间内的网络进行全方位监控和预警。     

Situation awareness and driving performance in a simulated navigation task

The objective of this study was to identify task and vehicle factors that may affect driver situation awareness (SA) and its relationship to performance, particularly in strategic (navigation) tasks. An experiment was conducted to assess the effects of in-vehicle navigation aids and reliability on driver SA and performance in a simulated navigation task. A total of 20 participants drove a virtual car and navigated a large virtual suburb. They were required to follow traffic signs and navigation directions from either a human aid via a mobile phone or an automated aid presented on a laptop. The navigation aids operated under three different levels of information reliability (100%, 80% and 60%). A control condition was used in which each aid presented a telemarketing survey and participants navigated using a map. Results revealed perfect navigation information generally improved driver SA and performance compared to unreliable navigation information and the control condition (task-irrelevant information). In-vehicle automation appears to mediate the relationship of driver SA to performance in terms of operational and strategic (navigation) behaviours. The findings of this work support consideration of driver SA in the design of future vehicle automation for navigation tasks.     

Interactive skills and individual differences in a word production task

In attempting to solve a wide variety of tasks, people naturally seek to modify their external environment such that the physical space in which they work is more amenable or ‘congenial’ to achieving a desired outcome. Attempts to determine the effectiveness of certain artifacts or spatial reorganizations in aiding reasoners solve problems must be relativised to the difficulty of the task and the cognitive abilities of the reasoners. These factors were examined using a simple word production task with letter tiles. Two sets of tiles that differed in terms of word production difficulty were selected. Participants were asked to produce as many words as they could within a finite time period for each letter set. In one group, participants were encouraged to rearrange or touch the tiles when producing words, and in the other group, participants could not interact with or point to the tiles. Participants were further split in a low and high verbal fluency group as a function of their score on the Thurstone word fluency test taken at the end of the experiment. In the high fluency group, letter rearrangement did not improve the participants’ ability to generate words. In contrast, in the low verbal fluency group, letter rearrangement significantly enhanced the ability to produce new words from both the hard and easy letter set. For these participants, the task was more taxing and the opportunity to restructure the letter set substantially elevated their performance. The advantages of a distributed cognition analysis of this and other reasoning tasks are discussed.     

A dyadic protocol for training complex skills: a replication using female participants

The effectiveness and efficiency of the active interlocked modeling (AIM) dyadic protocol in training complex skills has been extensively demonstrated. However, past evaluation studies have all used male participants exclusively. Consequently, the present study investigated the generalizability of the effectiveness and efficiency gains to women. We randomly assigned 108 female participants to either the AIM-dyad condition or a standard individual control training condition. The results supported the robustness and viability of the AIM protocol. Although their overall performance was lower than that obtained for men in previous studies, women trained in the AIM-dyad condition performed as well as those trained in the individual condition. Thus, the efficiency gains associated with the AIM-dyad protocol, which result from the ability to train two people simultaneously to reach the same performance level as a single person with no increase in training time or machine cost, are generalizable to female participants. The applied and basic research implications of the present study are discussed within the context of well-documented male/female differences in the performance of complex psychomotor tasks. For instance, given the number of women entering the workforce and the significant proportion of women in professions previously deemed to be male-dominated (e.g., air navigation), it is reassuring to know that sex differences in task performance do not necessarily imply sex differences in the effectiveness of training protocols.     

Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands

Cognitive task analysis (CTA) is a set of methods for identifying cognitive skills, or mental demands, needed to perform a task proficiently. The product of the task analysis can be used to inform the design of interfaces and training systems. However, CTA is resource intensive and has previously been of limited use to design practitioners. A streamlined method of CTA, Applied Cognitive Task Analysis (ACTA), is presented in this paper. ACTA consists of three interview methods that help the practitioner to extract information about the cognitive demands and skills required for a task. ACTA also allows the practitioner to represent this information in a format that will translate more directly into applied products, such as improved training scenarios or interface recommendations. This paper will describe the three methods, an evaluation study conducted to assess the usability and usefulness of the methods, and some directions for future research for making cognitive task analysis accessible to practitioners. ACTA techniques were found to be easy to use, flexible, and to provide clear output. The information and training materials developed based on ACTA interviews were found to be accurate and important for training purposes.     

Control of real-world complex robots using a biologically inspired algorithm

Elementary living beings, like bacteria, are able to reach food sources using only limited and very noisy sensory information. In this paper, we describe a very simple algorithm inspired from bacteria chemotaxis. We present a Markov chain model for studying the effect of noise on the behavior of an agent that moves according to this algorithm, and we show that, counterintuitively, the application of noise can increase the expected average performance over a fixed available time. After this theoretical analysis, experiments on real-world application of this algorithm are introduced. In particular, we show that the algorithm is able to control a complex robot arm, actuated by 17 McKibben pneumatic artificial muscles, without the need of any model of the robot or of its environment.     

Performance in a complex task and breathing under odor exposure

We investigated the influences of odor exposure on performance and on breathing measures. The task was composed of tracking, short-term memory, and peripheral reaction parts. During rest or while performing the task, 12 participants were exposed to 4 different odors in 2 intensities. The higher intensity of the malodors induced a short-term decrement in mean inspiration flow (Vi/Ti) after stimulus onset and impaired performance in the short-term memory task, as compared with control trials; no effect was found for the positively judged odors. The study suggests that a distractor as simple as a bad smell may pull a person off task, however briefly, and may result in a detriment to performance. Actual or potential applications of this research involve designing or securing tasks in such a way that a brief withdrawal of attention does not have fatal consequences.     

Elicitation by critiquing as a cognitive task analysis methodology

This paper describes elicitation by critiquing (EBC) as a cognitive task analysis (CTA) methodology. EBC takes advantage of the ability to analyze another’s task performance, a necessary skill for all domains. This technique can be used to help address some barriers to CTA methods such as domain access restrictions, frequency and predictability of observable and self-reported events, and difficulties in recruiting domain experts to participate. The technique enables controlled presentation of problem stimuli in order to obtain repeated measuring of the same task from multiple participants. To investigate this method, our team performed a CTA of inferential analysis using the EBC technique. Specifically, we observed six expert intelligence analysts critiquing a trainee analyzing the Ariane 501 launch failure. A second trainee was critiqued for reference. The method can be combined with other CTA methods to build knowledge about a domain, can be considered as an addition to participatory design methods, and can be varied depending on the domain being investigated.     

Cognitive simulation as a tool for cognitive task analysis

Cognitive simulations are runnable computer programs that represent models of human cognitive activities. We show how one cognitive simulation built as a model of some of the cognitive processes involved in dynamic fault management can be used in conjunction with small-scale empirical data on human performance to uncover the cognitive demands of a task, to identify where intention errors are likely to occur, and to point to improvements in the person-machine system. The simulation, called Cognitive Environment Simulation or CES, has been exercised on several nuclear power plant accident scenarios. Here we report one case to illustrate how a cognitive simulation tool such as CES can be used to clarify the cognitive demands of a problem-solving situation as part of a cognitive task analysis.     

Hierarchical task network and operator-based planning: two complementary approaches to real-world planning

Work on generative planning systems has focused on two diverse approaches to plan construction. Hierarchical task network (HTN) planners build plans by successively refining high-level goals into lower-level activities. Operator-based planners employ means-end analysis to directly formulate plans consisting of low-level activities. While many have argued the universal dominance of a single approach, this paper presents an alternative view: that in different situations either may be most appropriate. To support this view, a number of advantages and disadvantages of these approaches are described in light of experiences in developing two real-world, fielded planning systems.     

Cognitive diversity and team performance in a complex multiple task environment

This article examines the multiple effects of cognitive diversity in teams operating complex human-machine-systems. The study employed a PC-based multiple-task environment, called the Cabin Air Management System, which models a process control task in the operational context of a spacecraft's life support system. Two types of cognitive diversity were examined: system understanding and team specialization. System understanding referred to the depth of understanding team members were given during training (low-level procedure-oriented vs. high level knowledge-oriented training). Team specialization referred to the degree to which knowledge about system fault scenarios was distributed between team members (specialized vs. non-specialized). A total of 72 participants took part in the study. After having received 4.5 h of training on an individual basis, participants completed a 1-h experimental session, in which they worked in two-person teams on a series of fault scenarios of varying difficulty. Measures were taken of primary and secondary task performance, system intervention and information sampling strategies, system knowledge, subjective operator state, communication patterns and conflict. The results provided evidence for the benefits of cognitive diversity with regard to system understanding. This manifested itself in better primary task performance and more efficient manual system control. No advantages were found for cognitive diversity with regard to specialization. There was no effect of cognitive diversity on intra-team conflict, with conflict levels generally being very low. The article concludes with a discussion of the implications of the findings for the engineering of cognitive diversity in teams operating complex human-machine-systems.     

Cognitive diversity and team performance in a complex multiple task environment

This article examines the multiple effects of cognitive diversity in teams operating complex human-machine-systems. The study employed a PC-based multiple-task environment, called the Cabin Air Management System, which models a process control task in the operational context of a spacecraft's life support system. Two types of cognitive diversity were examined: system understanding and team specialization. System understanding referred to the depth of understanding team members were given during training (low-level procedure-oriented vs. high level knowledge-oriented training). Team specialization referred to the degree to which knowledge about system fault scenarios was distributed between team members (specialized vs. non-specialized). A total of 72 participants took part in the study. After having received 4.5 h of training on an individual basis, participants completed a 1-h experimental session, in which they worked in two-person teams on a series of fault scenarios of varying difficulty. Measures were taken of primary and secondary task performance, system intervention and information sampling strategies, system knowledge, subjective operator state, communication patterns and conflict. The results provided evidence for the benefits of cognitive diversity with regard to system understanding. This manifested itself in better primary task performance and more efficient manual system control. No advantages were found for cognitive diversity with regard to specialization. There was no effect of cognitive diversity on intra-team conflict, with conflict levels generally being very low. The article concludes with a discussion of the implications of the findings for the engineering of cognitive diversity in teams operating complex human-machine-systems.