Effects of working memory,attention, and expertise on pilots’ situation awareness

The current study investigates individual differences that predict situation awareness (SA) in professional pilots. The aim of the study is twofold: to examine the roles of divided attention, inhibition, working memory, and expertise in predicting SA, and to demonstrate the relative contributions of these individual differences to online (Situation Awareness Present Method, SPAM) and offline (Situation Awareness Global Assessment Technique, SAGAT) SA measures. Thirty-six professional pilots completed a challenging flight scenario in a full-flight simulator. Divided attention, inhibition, working memory span, and expertise were measured using choice reaction time with dichotic listening, Stroop, and Automated Operation Span tasks, and flight hours in a full-flight simulator, respectively. Results indicated that offline and online SA measure were not correlated, supporting their concurrent use to obtain a comprehensive measure of SA. Offline SA scores were best predicted by working memory and level of expertise, while online SA scores were predicted by expertise, divided attention and inhibition. Results are discussed focusing on both theoretical contributions for defining and measuring SA and applications. Findings have implications for operators of critical domains and their interactions with automated systems, in which SA is crucial for performance and safety.     

The role of working memory on graphical information processing

This research extends previous graphics research by examining how individual differences in working memory (WM) capacity and changes in graphic design influence graphical information processing. An experiment compared decision accuracy of two graphic decision aids and an unaided group for a task at two levels of complexity. There were no accuracy differences for the low complexity task. At high levels of task complexity, accuracy depended upon WM capacity and how the graphic aid influenced perception. Eye movement data show information processing differences also are contingent upon graphic design features and WM capacity. We postulate that graphs reduce cognitive overhead by shifting some of the cognitive burden to our visual perception system. More efficient graphical perceptual will improve decision performance only if our cognitive resources are capacity constrained and those cognitive resources are used elsewhere in the problem solving process.     

The role of working memory on graphical information processing

This research extends previous graphics research by examining how individual differences in working memory (WM) capacity and changes in graphic design influence graphical information processing. An experiment compared decision accuracy of two graphic decision aids and an unaided group for a task at two levels of complexity. There were no accuracy differences for the low complexity task. At high levels of task complexity, accuracy depended upon WM capacity and how the graphic aid influenced perception. Eye movement data show information processing differences also are contingent upon graphic design features and WM capacity. We postulate that graphs reduce cognitive overhead by shifting some of the cognitive burden to our visual perception system. More efficient graphical perceptual will improve decision performance only if our cognitive resources are capacity constrained and those cognitive resources are used elsewhere in the problem solving process.     

Measuring working memory load effects on electrophysiological markers of attention orienting during a simulated drive

Intersection accidents result in a significant proportion of road fatalities, and attention allocation likely plays a role. Attention allocation may depend on (limited) working memory (WM) capacity. Driving is often combined with tasks increasing WM load, consequently impairing attention orienting. This study (n = 22) investigated WM load effects on event-related potentials (ERPs) related to attention orienting. A simulated driving environment allowed continuous lane-keeping measurement. Participants were asked to orient attention covertly towards the side indicated by an arrow, and to respond only to moving cars appearing on the attended side by pressing a button. WM load was manipulated using a concurrent memory task. ERPs showed typical attentional modulation (cue: contralateral negativity, LDAP; car: N1, P1, SN and P3) under low and high load conditions. With increased WM load, lane-keeping performance improved, while dual task performance degraded (memory task: increased error rate; orienting task: increased false alarms, smaller P3).

Practitioner Summary: Intersection driver-support systems aim to improve traffic safety and flow. However, in-vehicle systems induce WM load, increasing the tendency to yield. Traffic flow reduces if drivers stop at inappropriate times, reducing the effectiveness of systems. Consequently, driver-support systems could include WM load measurement during driving in the development phase.     

Effects of backpack weight on the performance of basic short-term/working memory tasks during flat-surface standing

This study empirically investigated the effects of backpack weight on the performance of three basic short-term/working memory (STM/WM) tasks during flat-surface standing. Four levels of backpack weight were considered: 0, 15, 25 and 40% of the body weight. The three STM/WM tasks were the Corsi block, digit span and 3-back tasks, corresponding to the visuo-spatial sketchpad, phonological loop and central executive of WM, respectively. Thirty participants conducted the STM/WM tasks while standing with loaded backpack. Major study findings were that (1) increased backpack weight adversely affected the scores of all three STM/WM tasks; and, (2) the adverse effect of backpack weight was less pronounced for the phonological loop STM task than the other STM/WM tasks. The study findings may help understand and predict the impacts of body-worn equipment weight on the worker’s mental task performance for various work activities requiring simultaneous performance of mental and physical tasks.

Practitioner summary: The current study empirically examined the effects of backpack weight on the performance of three basic STM/WM tasks. The study findings entail that reduces the weight of body-worn equipment can positively impact the worker’s mental task performance in addition to reducing the worker's bodily stresses.

Abbreviations: ACC: anterior cingulate cortex; AP: anterior-posterior; BW: body weight; CoP: centre of pressure; C-S: central executive working memory task and standing; DLPFC: dorsolateral prefrontal cortex; HIP: human information processing; ML: medio-lateral; PMC: premotor cortex; P-S: phonological loop short-term memory task and standing; SMA: supplementary motor area; STM: short-term memory; VLPFC: ventrolateral prefrontal cortex; V-S: visuo-spatial short-term memory task and standing; WM: working memory     

Does situation awareness add to the validity of cognitive tests?

OBJECTIVE: Does adding situation awareness (SA) to a battery of cognitive tests improve prediction? BACKGROUND: Identifying variables that predict skilled performance in a complex task aids in understanding the nature of skill and also aids in the selection of operators to perform that task. SA is thought to be an important predictor of performance. SA is often thought to be based on underlying cognitive mechanisms. METHOD: Three performance measures taken from the Federal Aviation Administration's (FAA's) Air Traffic Scenarios Test, the low-fidelity simulation component of the FAA's controller selection battery, were used as criterion variables in a hierarchical regression. After predicting performance based on a battery of cognitive (e.g., intelligence, working memory, spatial memory) and noncognitive tests (e.g., cognitive style, personality, demographics), we added measures of SA. RESULTS: SA did provide increases in prediction, but only when measured with the Situation Present Assessment Method, an on-line query method. When the same questions were asked off line, SA did not enter the model in two cases and improved prediction by only 2% in the third. CONCLUSION: Thus, some measures of SA do show incremental validity, even against a backdrop of a large number of cognitive variables. APPLICATION: On-line measures of SA can be a worthwhile addition to standard batteries of tests used to predict performance in cognitively oriented industrial tasks.     

Examining the Effects of Conformal Terrain Features in Advanced Head‐Up Displays on Flight Performance and Pilot Situation Awareness

Synthetic vision systems (SVS) render terrain features for pilots through cockpit displays using a GPS database and three‐dimensional graphical models. Enhanced vision systems (EVS) present infrared imagery of terrain using a forward‐looking sensor in the nose of an aircraft. The ultimate goal of SVS and EVS technologies is to support pilots in achieving safety under low‐visibility and night conditions comparable to clear, day conditions. This study assessed pilot performance and situation awareness (SA) effects of SVS and EVS imagery in an advanced head‐up display (HUD) during a simulated landing approach under instrument meteorological conditions. Videos of the landing with various HUD configurations were presented to eight pilots with a superimposed tracking task. The independent variables included four HUD feature configurations (baseline [no terrain imagery], SVS, EVS, and a combination of SVS and EVS), two visibility conditions, and four legs of the flight. Results indicated that SVS increased overall SA but degraded flight path control performance because of visual confusion with other display features. EVS increased flight path control accuracy but decreased system (aircraft) awareness because of visual distractions. The combination of SVS and EVS generated offsetting effects. Display configurations did not affect pilot spatial awareness. Flight performance was not different among phases of the approach, but levels and types of pilot SA did vary from leg to leg. These results are applicable to development of adaptive HUD features to support pilot performance. They support the use of multidimensional measures of SA for insight on pilot information processing with advanced aviation displays. © 2012 Wiley Periodicals, Inc.     

Expository multimedia comprehension in E-learning: Presentation format,verbal ability and working memory capacity

Most of the studies establishing factors affecting digital text and multimedia comprehension have been conducted in controlled conditions. The present study sought to test and extend the modality and seductive details effects, and the role of verbal ability and working memory capacity, to a remote, self-paced, E-learning scenario. Two hundred and thirteen first-year undergraduates read or watched videos about scientific expository content in three formats: digital text (written expository texts, navigated in seven screens), presentation video (audio explanation, with written keywords), and presentation video with dynamic decorative images (audio explanation, written keywords, and dynamic decorative and irrelevant images). In a face-to-face session, they completed working memory and verbal ability tests. Comprehension performance was similar for the three conditions. For the multimedia videos with dynamic decorative irrelevant images, comprehension depended on working memory capacity. Verbal ability was relevant for both expository text and videos.     

How stress influences creativity in game-based situations: Analysis of stress hormones,negative emotions,and working memory

This study aims to integrate neuroscientific techniques into a behavioral experimental design to investigate how stress stimuli may influence stress hormones and negative emotions, subsequently affecting working memory (WM) and creativity in game-based situations. Ninety-six college students participated in this study, in which a game-based experiment lasting 90 min was employed. The main findings were that (1) the employed stress stimuli influence creativity during gaming through two routes: enhancing creativity through cortisol concentration and WM and decreasing creativity by provoking promotion-focused negative emotions (frustration and anger); and (2) the subjective negative emotions and objective cortisol responses do not consistently predict WM and creativity in game-based situations. Accordingly, appropriate challenges or stressors that help increase the cortisol concentration to an attentional level without provoking a strong sense of promotion-focused negative emotions should be considered when designing games aimed at teaching creativity.     

Visuospatial working memory in learning from multimedia systems

Abstract  Multimedia systems involve the association of various types of information: verbal information presented visually or auditorily, static or dynamic pictorial information, and sound information. In a cognitive approach, integrating this information involves complex processes constrained by properties of the learner's cognitive system, and especially by the capacity of working memory. This paper, reports on an experiment focused on the integration of verbal and pictorial information when students learn a series of physics concepts. The involvement of the visuo-spatial working memory was investigated by means of a dual-task paradigm. Results show that pictorial information enhances the learning process. They also suggest that the visual and the spatial components of visuospatial working memory should be considered separately. Finally, they emphasise the need to consider the limitations in cognitive resources available to the learner.     

Investigating pilot performance using mixed-modality simulated data link

OBJECTIVE: General aviation (GA) pilot performance utilizing a mixed-modality simulated data link was objectively evaluated based on the time required in accessing, understanding, and executing data link commands. Additional subjective data were gathered on workload, situation awareness (SA), and preference. BACKGROUND: Research exploring mixed-modality data link integration to the single-pilot GA cockpit is lacking, especially with respect to potential effects on safety. METHODS: Sixteen visual flight rules (VFR)-rated pilots participated in an experiment using a flight simulator equipped with a mixed-modality data link. Data link modalities were text display, synthesized speech, digitized speech, and synthesized speech/text combination. Flight conditions included VFR (unlimited ceiling and visibility) or marginal VFR flight conditions (clouds 2,800 ft above ground level, 3-mile visibility). RESULTS: Statistically significant differences were found in pilot performance, mental workload, and SA across the data link modalities. Textual data link resulted in increased time and workload as compared with the three speech-type data link conditions, which did not differ. SA measures indicated higher performance with textual and digitized speech data link conditions. CONCLUSION: Textual data link can be significantly enhanced for single-pilot GA operations by the addition of a speech component. APPLICATION: Potential applications include operational safety in future GA systems that incorporate data link for use by a single pilot and guidance in the development of flight performance objectives for these systems.     

Computerized working memory training: Can it lead to gains in cognitive skills in students?

Given that working memory is an important cognitive skill that is linked to academic success, there is increasing attention given to exploring ways to support working memory problems in struggling students. One promising approach is computerized training, and the aim of the present study focused on whether computerized working memory training could result in both near and far transfer training effects; and whether such effects would be maintained over time. Students were allocated into one of three groups: Nonactive Control, Active Control, where they trained once a week (WMT-Low frequency); Training group, where they trained four times a week (WMT-High frequency). All three groups were tested on measures of working memory, verbal and nonverbal ability, and academic attainment before training; and re-tested on the same measures after training, as well as 8 months later. The data indicate gains in both verbal and visuo-spatial working memory tasks for the high-frequency Training group. Improvements were also evidenced in tests of verbal and nonverbal ability tests, as well as spelling, in the high-frequency Training group. There were some maintenance effects when students were tested 8 months later. Possible reasons for why the computerized working memory training led to some far transfer effects in the high-frequency Training group are included in the discussion.     

The effects of presentation method and information density on visual search ability and working memory load

This study investigates the effects of successive and simultaneous information presentation methods on learner’s visual search ability and working memory load for different information densities. Since the processing of information in the brain depends on the capacity of visual short-term memory (VSTM), the limited information processing capacity of learners may affect the visual search ability and working memory load differently for successive and simultaneous presentations. A change-detection experiment was conducted in this research to analyze visual search ability and working memory load. Two 4 × 4 dot arrays with three information densities were designed for the two presentation methods to test twenty-two participants. The results of the study indicated significant differences between the visual search abilities and the working memory loads for the two types of presentations at higher levels of information densities. Furthermore, significant differences were identified between visual search abilities for different information densities, due to the limited capacity of VSTM. The correlations of visual search ability and working memory load showed that the attention of the learners with higher visual search ability and lower working memory loads would perform better than the learners with lower visual search ability and higher working memory load.     

The role of working memory, field dependence, visual search, and reaction time in the left turn performance of older female drivers

Older drivers have difficulties at intersections, especially in their performance of left turns. Older females are especially at risk in intersection maneuvers. Information processing abilities have been most promising in explaining the crash involvement of elderly drivers. This study examines the relationship of field dependence, visual search skills, and working memory to the decision to make a left turn at an intersection as well as to gap choice. Participants included thirty-three women ranging in age from 61 to 84 years. The results show that working memory plays a very important role in left-turn performance. Implications for appropriate interventions are discussed.     

The effect of working memory training on situation awareness in a flight simulator

Cognition, Technology & Work - The close relationship between working memory and situation awareness (SA) has been confirmed and further empirical investigations are lacking. The main aim of...     

ADAPT: A Predictive Cognitive Model of User Visual Attention and Action Planning

We present a computational cognitive model of novice and expert aviation pilot action planning called ADAPT that models performance in a dynamically changing simulated flight environment. We perform rigorous tests of ADAPT's predictive validity by comparing the performance of individual human pilots to that of their respective models. Individual pilots were asked to execute a series of flight maneuvers using a flight simulator, and their eye fixations and control movements were recorded in a time-synched database. Computational models of each of the 25 individual pilots were constructed, and the individual models simulated execution of the same flight maneuvers performed by the human pilots. The time-synched eye fixations and control movements of individual pilots and their respective models were compared, and rigorous tests of ADAPT's predictive validity were performed. The model explains and predicts a significant portion of pilot visual attention and control movements during flight as a function of piloting expertise. Implications for adaptive training systems are discussed.     

Structure Processing of Web-Based Menus

Grounded in cognitive load theory, levels of processing theory, modes of learning research, and the facets of working memory (WM) framework, the primary goal of this investigation was to seek empirical evidence for a structure processing mechanism in WM that facilitates the coordination of associations made into coherent structures and the role this mechanism might play in performance and retention. This goal was motivated by a gap in the understanding of mental model production and encoding processes in WM. To carry out this goal, two usability studies were conducted to investigate retention, error, and time performance of Web-based menus created with structured or unstructured design. Relationships between these measures and coordination ability in WM, as well as storage and supervision ability, were also assessed. In Study 1, which tested simple Web-based menus, correlations of .28, ?.56, and ?.54 were detected between coordination (CC) and retention scores, CC and errors, and CC and time; in Study 2, which tested commercial Web-based menus, correlations of .46, ?.45, and ?.25 were detected. For Studies 1 and 2, differences in favor of structured design for retention, error, and time performance were highly significant (p < .001).     

The impact of negative online social network content on expressed sentiment,executive function,and working memory

Given the paucity of research examining emotional states in social networking, the aim of this study was to examine the impact of negatively valanced social media network content on the amount of sentiment that participants included in their free-text responses, as well as on their executive functioning and working memory. Eighty participants aged between 18 and 67 (M = 29.39, SD = 11.21 years) completed baseline mood and cognitive measures (working memory and executive functioning) before exposure to three control posts and one negative emotional post. For each post, participants wrote a free-text response or indicated that they would not respond. Participants then completed the mood and cognitive measures a second time. After exposure to an emotionally negative post, participants' mood was lower and their executive functioning improved (as measured by reaction time and number of incorrectly identified target words). Participants' responses to an emotional post contained higher levels of sentiment compared to their responses to control posts. After controlling for demographic variables, participants' mood and trait empathy predicted the level of sentiment that they included in their responses to the emotional post. Mood, executive function, and trait empathy contributed to individuals’ online social network engagement for emotionally negative posts.     

Working memory modeling using inverse fuzzy relational approach

The paper proposes a novel approach to fuzzy modeling of human working memory (WM) using electroencephalographic (EEG) signals, acquired during human face encoding and recall experiments in connection with a face recognition problem. The EEG signals acquired from the short term memory (STM) during memory encoding instances are considered as the input of the proposed working memory model. On the other hand, the EEG response of the WM to visual stimuli acquired during WM recall instances are considered as the output of the proposed working memory model. The entire experiment is primarily divided into two phases. In the first phase, the WM of a human subject is modeled by a fuzzy implication relation, describing a mapping from the STM response (during encoding) to the WM responses (during recall) to visual stimuli. During STM encoding, the subject is visually presented with the full face stimulus of a person. During WM recall, four partial face stimuli of the same person (made familiar during encoding) are used for the subject to recall the respective full face.The second phase is undertaken to validate the WM model by visually stimulating the subject again with randomly selected partial faces of people, being familiar in the first phase and the WM EEG responses are recorded. The WM responses along with the WM model, developed in the first phase, are used to retrieve the STM information by using an inverse fuzzy (implication) relation. Besides WM modeling, another important contribution of the paper lies in devising a solution to the inverse fuzzy relation computation in the settings of an optimization problem. An error metric is then defined to measure the discrepancy between the model-predicted STM encoding pattern and the actual pattern encoded by the STM (as captured by the EEG signal during encoding in the first phase). Apparently, smaller the error magnitude better is the accuracy of the proposed model to effectively differentiate people with memory failures. Experimentally it is observed that the proposed model yields a very small error, in the order of 10⁻⁴, thus showing a high level of similarity between actual and model predicted STM response for all the healthy subjects. An experiment undertaken using eLORETA software confirms that the orbito-frontal cortex of prefrontal lobe is responsible for STM encoding whereas dorsolateral prefrontal region is responsible for WM recall. An analysis undertaken reveals that the proposed WM model produces the best response in the theta frequency band of EEG spectra, thus assuring the association of the theta frequency range in the face recognition task. Comparative analysis performed also substantiates that the proposed technique of computing max–min inverse fuzzy relation outperforms the existing techniques for inverse fuzzy computation, with a successful retrieval accuracy of 87.92%. The proposed study would find interesting applications to diagnose memory failures for people with Pre-frontal lobe amnesia.     

The influence of training level on manual flight in connection to performance,scan pattern,and task load

This work focuses on the analysis of pilots’ performance during manual flight operations in different stages of training and their influence on gaze strategy. The secure and safe operation of air traffic is highly dependent on the individual performances of the pilots. Before becoming a pilot, he/she has to acquire a broad set of skills by training to pass all the necessary qualification and licensing standards. A basic skill for every pilot is manual control operations, which is a closed-loop control process with several cross-coupled variables. Even with increased automation in the cockpit, the manual control operations are essential for every pilot as a last resort in the event of automation failure. A key element in the analysis of manual flight operations is the development over time in relation to performance and visual perception. An experiment with 28 participants (including 11 certified pilots) was conducted in a Boeing 737 simulator. For defined flight phases, the dynamic time warping method was applied to evaluate the performance for selected criteria, and eye-tracking methodology was utilized to analyze the gaze-pattern development. The manipulation of workload and individual experience influences the performance and the gaze pattern at the same time. Findings suggest that the increase of workload has an increased influence on pilots depending on the flight phase. Gaze patterns from experienced pilots provide insights into the training requirements of both novices and experts. The connection between workload, performance and gaze pattern is complex and needs to be analyzed under as many differing conditions. The results imply the necessity to evaluate manual flight operations with respect to more flight phases and a detailed selection of performance indications.