Semantic congruency of auditory warnings

Abstract

The aim of this study was to explore operator experience and performance for semantically congruent and incongruent auditory icons and abstract alarm sounds. It was expected that performance advantages for congruent sounds would be present initially but would reduce over time for both alarm types. Twenty-four participants (12M/12F) were placed into auditory icon or abstract alarm groupings. For each group both congruent and incongruent alarms were used to represent different driving task scenarios. Once sounded, participants were required to respond to each alarm by selecting a corresponding driving scenario. User performance for all sound types improved over time, however even with experience a decrement in speed of response remained for the incongruent iconic sounds and in accuracy of performance for the abstract warning sounds when compared to the congruent auditory icons. Semantic congruency was found to be of more importance for auditory icons than for abstract sounds.

Practitioner Summary: Alarms are used in many operating systems as emergency, alerting, or continuous monitoring signals for instance. This study found that the type and representativeness of an auditory warning will influence operator performance over time. Semantically congruent iconic sounds produced performance advantages over both incongruent iconic sounds and abstract warnings.     

Mapping candidate within-vehicle auditory displays to their referents

OBJECTIVE: This study investigated speech, environmental sounds (naturally occurring sounds with arbitrary meanings), auditory icons (natural or synthetic sounds with specific meanings), and abstract synthetic warnings as candidates for within-vehicle interfaces. BACKGROUND: Auditory displays and warnings must satisfy certain criteria, such as being appropriately urgent and commanding appropriately fast response times. However, a semiotic analysis suggests that displays, as signals interpreted by users, should also be mapped successfully onto their referents. METHOD: Response times and accuracy were recorded in a computer task of identifying learned mappings of candidate displays to a range of referent driving events (such as "headway closing"); perceived urgency and pleasantness were assessed separately. RESULTS: Speech and auditory icons produced near-ceiling performance in response times and identification accuracy. Abstract sounds produced notably slower response times and less accuracy. Environmental sounds showed an intermediate pattern of performance for accuracy, but the response times were similar to those of the abstract sounds. Speech utterances were similarly and consistently rated as pleasant but also of intermediate perceived urgency. The three other sound types showed a consistent mapping of their perceived urgency to the situational urgency of their referents; for these sounds, perceived urgency and pleasantness were negatively correlated. CONCLUSION: The results point to the importance of considering the role of signal-referent relationships in designing auditory displays. APPLICATION: The results have applicability for auditory displays in the vehicle interface, whereas the theoretical framework is of value in auditory display design in a broader context.     

A new class of auditory warning signals for complex systems: auditory icons

This simulator-based study examined conventional auditory warnings (tonal, nonverbal sounds) and auditory icons (representational, nonverbal sounds), alone and in combination with a dash-mounted visual display, to present information about impending collision situations to commercial motor vehicle operators. Brake response times were measured for impending front-to-rear collision scenarios under 6 display configurations, 2 vehicle speeds, and 2 levels of headway. Accident occurrence was measured for impending side collision scenarios under 2 vehicle speeds, 2 levels of visual workload, 2 auditory displays, absence/presence of mirrors, and absence/presence of a dash-mounted iconic visual display. For both front-to-rear and side collision scenarios, auditory icons elicited significantly improved driver performance over conventional auditory warnings. Driver performance improved when collision warning information was presented through multiple modalities. Brake response times were significantly faster for impending front-to-rear collision scenarios using the longer headway condition. The presence of mirrors significantly reduced the number of accidents for impending side collision scenarios. Subjective preference data indicated that participants preferred multimodal displays over single-modality displays. Actual or potential applications for this research include auditory displays and warnings, information presentation, and the development of alternative user interfaces.     

Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an invehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying ‘ahead’. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application.

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an in-vehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying 'ahead'. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving

OBJECTIVE: This study examined the effectiveness of rear-end collision warnings presented in different sensory modalities as a function of warning timing in a driving simulator. BACKGROUND: The proliferation of in-vehicle information and entertainment systems threatens driver attention and may increase the risk of rear-end collisions. Collision warning systems have been shown to improve inattentive and/or distracted driver response time (RT) in rear-end collision situations. However, most previous rear-end collision warning research has not directly compared auditory, visual, and tactile warnings. METHOD: Sixteen participants in a fixed-base driving simulator experienced four warning conditions: no warning, visual, auditory, and tactile. The warnings activated when the time-to-collision (TTC) reached a critical threshold of 3.0 or 5.0 s. Driver RT was captured from a warning below critical threshold to brake initiation. RESULTS: Drivers with a tactile warning had the shortest mean RT. Drivers with a tactile warning had significantly shorter RT than drivers without a warning and had a significant advantage over drivers with visual warnings. CONCLUSION: Tactile warnings show promise as effective rear-end collision warnings. APPLICATION: The results of this study can be applied to the future design and evaluation of automotive warnings designed to reduce rear-end collisions.     

A psychophysiological evaluation of the perceived urgency of auditory warning signals

One significant concern that pilots have about cockpit auditory warnings is that the signals presently used lack a sense of priority. The relationship between auditory warning sound parameters and perceived urgency is, therefore, an important topic of enquiry in aviation psychology. The present investigation examined the relationship among subjective assessments of urgency, reaction time, and brainwave activity with three auditory warning signals. Subjects performed a tracking task involving automated and manual conditions, and were presented with auditory warnings having various levels of perceived and situational urgency. Subjective assessments revealed that subjects were able to rank warnings on an urgency scale, but rankings were altered after warnings were mapped to a situational urgency scale. Reaction times differed between automated and manual tracking task conditions, and physiological data showed attentional differences in response to perceived and situational warning urgency levels. This study shows that the use of physiological measures sensitive to attention and arousal, in conjunction with behavioural and subjective measures, may lead to the design of auditory warnings that produce a sense of urgency in an operator that matches the urgency of the situation.     

Effects of single versus multiple warnings on driver performance

OBJECTIVE: To explore how a single master alarm system affects drivers' responses when compared with multiple, distinct warnings. BACKGROUND: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event type on driver performance. METHOD: Using a 2 x 2 x 4 mixed factorial design, we investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers' performance across four collision event types (frontal collision warnings, left and right lane departure warnings, and warnings for a fast-approaching following vehicle). RESULTS: Participants' reaction times and accuracy rates were significantly affected by the type of collision event and alarm reliability. The use of individual alarms, rather than a single master alarm, did not significantly affect driving performance in terms of reaction time or response accuracy. CONCLUSION: Even though a master alarm is a relatively uninformative warning, it produced statistically no different reaction times or accuracy results when compared with information-rich auditory icons, some of which were spatially located. In addition, unreliable alarms negatively impacted driver performance, regardless of event type or alarm scheme. APPLICATION: These results have important implications for the development and implementation of multiple driver warning systems.     

Impact of spatial auditory feedback on the efficiency of iconic human–computer interfaces under conditions of visual impairment

This paper investigates the addition of spatial auditory feedback as a tool to assist people with visual impairments in the use of computers, specifically in tasks involving iconic visual search. In this augmented interface, unique sounds were mapped to visual icons on the screen. As the screen cursor traversed the screen, the user heard sounds of nearby icons, spatially, according to the relative position of each icon with respect to the screen cursor. A software prototype of the design was developed to evaluate the performance of users in the search of icons within the proposed interface. Experiments were conducted with simulated visual impairments on volunteer participants to evaluate if the addition of spatial auditory feedback makes the interface more accessible to users with impaired vision. Results demonstrated that spatialization of icon sounds provides additional remote navigational information to users, enabling new strategies for task completion. Directions for future research are discussed and prioritized.     

Multisensory in-car warning signals for collision avoidance

OBJECTIVE: A driving simulator study was conducted in order to assess the relative utility of unimodal auditory, unimodal vibrotactile, and combined audiotactile (i.e., multisensory) in-car warning signals to alert and inform drivers of likely front-to-rear-end collision events in a situation modeled on real-world driving. BACKGROUND: The implementation of nonvisual in-car warning signals may have important safety implications in lessening any visual overload during driving. Multisensory integration can provide synergistic facilitation effects. METHOD: The participants drove along a rural road in a car-following scenario in either the presence or absence of a radio program in the background. The brake light signals of the lead vehicle were also unpredictably either enabled or disabled on a trial-by-trial basis. RESULTS: The results showed that the participants initiated their braking responses significantly more rapidly following the presentation of audiotactile warning signals than following the presentation of either unimodal auditory or unimodal vibrotactile warning signals. CONCLUSION: Multisensory warning signals offer a particularly effective means of capturing driver attention in demanding situations such as driving. APPLICATION: The potential value of such multisensory in-car warning signals is explained with reference to recent cognitive neuroscience research.     

Which is more prominent for fighter pilots under different flight task difficulties: Visual alert or verbal alert?

To explore the behavioural and cognition patterns of fighter pilots under visual and verbal alert conditions, 20 right-handed subjects participated in the experiment after 200 h of flight task training, and ERP technology was used to conduct EEG experiments under four conditions: high time pressure visual alert, low time pressure visual alert, fast speech speed verbal alert, and slow speech speed verbal alert. The results indicated the following: 1) the highest accuracy rate and shortest reaction time could be achieved under the slow speech speed verbal alert condition, indicating that the slow speech speed alert performance was optimal. 2) Subjects in the verbal alert trial showed a “right hemisphere advantage”, and the brain function area was located in the right temporal lobes. The speech speed correlated negatively with the latency and amplitude of P80.3) Subjects in the visual alert trial showed a “left hemisphere advantage”, and the brain activity was located in the left parietal lobes. 4) The latency of P80 evoked by verbal alert was earlier than that of N140 evoked by visual alert, indicating that a verbal alert causes an earlier attention effect in pilots. Based on the eye movement experiment, the performance of visual and verbal dual-channel alert was significantly higher than that of visual alert.     

Alerts for in-vehicle information systems: annoyance, urgency, and appropriateness

OBJECTIVE: This study assesses the influence of the auditory characteristics of alerts on perceived urgency and annoyance and whether these perceptions depend on the context in which the alert is received. BACKGROUND: Alert parameters systematically affect perceived urgency, and mapping the urgency of a situation to the perceived urgency of an alert is a useful design consideration. Annoyance associated with environmental noise has been thoroughly studied, but little research has addressed whether alert parameters differentially affect annoyance and urgency. METHOD: Three 2(3) x 3 mixed within/between factorial experiments, with a total of 72 participants, investigated nine alert parameters in three driving contexts. These parameters were formant (similar to harmonic series), pulse duration, interpulse interval, alert onset and offset, burst duty cycle, alert duty cycle, interburst period, and sound type. Imagined collision warning, navigation alert, and E-mail notification scenarios defined the driving context. RESULTS: All parameters influenced both perceived urgency and annoyance (p < .05), with pulse duration, interpulse interval, alert duty cycle, and sound type influencing urgency substantially more than annoyance. There was strong relationship between perceived urgency and rated appropriateness for high-urgency driving scenarios and a strong relationship between annoyance and rated appropriateness for low-urgency driving scenarios. CONCLUSION: Sound parameters differentially affect annoyance and urgency. Also, urgency and annoyance differentially affect perceived appropriateness of warnings. APPLICATION: Annoyance may merit as much attention as urgency in the design of auditory warnings, particularly in systems that alert drivers to relatively low-urgency situations.     

Cross-modal warnings for orienting attention in older drivers with and without attention impairments

Older adults are overrepresented in fatal crashes on a per-mile basis. Those with useful field of view (UFOV) reductions show a particularly elevated crash risk that might be mitigated with vehicle-based warnings. To evaluate cross-modal cues that could be used in these warnings, we applied a variation of Posner's orienting of attention paradigm. Twenty-nine older drivers with UFOV impairments and 32 older drivers without impairments participated. Cues were presented in either a single modality or a combination of modalities (visual, auditory, haptic). Drivers experienced three cue types (valid spatial information, invalid spatial information, neutral) and an uncued baseline. Following each cue, drivers discriminated the direction of a target (a Landolt square with a gap facing up or down) in the visual panorama. Drivers with and without UFOV impairments showed comparable response times (RTs) across the different cue modalities and cue types. Both groups benefited most from auditory and auditory/haptic cues. Redundant visual cues, when paired with auditory cues, undermined performance rather than enhanced it. Overall, drivers responded faster to targets with valid spatial information followed by neutral, invalid, and uncued targets. Cues provide the greatest benefit in alerting rather than orienting the driver. The cue expected to be most effective at orienting attention - the extra-vehicular cue - performs most poorly when the spatial information is either invalid or neutral. Even when the spatial information is valid the extra-vehicular cue underperforms the auditory cues. The results suggest that temporal information dominates spatial information in the ability of cues to speed responses to targets. This study represents a first step in assessing whether combining a cognitive science paradigm and a driving simulator environment can quickly assess how different warning signals alert and orient drivers.     

Preliminary study of behavioral and safety effects of driver dependence on a warning system in a driving simulator

Warning systems are being developed to improve traffic safety using visual, auditory, and/or tactile displays by informing drivers of the existence of a threat in the roadway. Behavioral and safety effects of driver dependence on such a warning system, especially when the warning system is unreliable, were investigated in a driving-simulator study. Warning-system accuracy was defined in terms of miss rate (MR) and positive predictive value (PPV) (PPV is the fraction of warnings that were correct detections). First, driver behavior and performance were measured across four warning-system accuracy conditions. Second, the authors estimated the probability of collision in each accuracy condition to measure the overall system effectiveness in terms of safety benefit. Combining these results, a method was proposed to evaluate the degree of driver dependence on a warning system and its effect on safety. One major result of the experiment was that the mean driving speed decreased as the missed detection rate increased, demonstrating a decrease in driver's reliance on warnings when the system was less effective in detecting threats. Second, both the acceleration-pedal and brake-pedal reaction times increased as the PPV of the warning system decreased, demonstrating a decrease in driver compliance with warnings when the system became more prone to false alarms. A key implication of the work is that performance is not necessarily directly correlated to warning-system quality or trends in subjective ratings, highlighting the importance of objective evaluation. Practical applications of the work include design and analysis of in-vehicle warning systems.     

Iconic languages: Towards end-user programming of mobile applications

After tracing the steps that led to the current generation of iconic languages starting from the original idea of S.K. Chang, we describe an iconic language, named MicroApp, for modeling pervasive mobile applications directly on the mobile device. MicroApp exploits generalized icons for composing mobile applications: services are represented by icons and are composed of adopting colors for representing data-flow. We also qualitatively evaluate the visual environment that implements this iconic language.     

Effects of warning validity and proximity on responses to warnings

The effects of a warning's validity and display characteristics on the responses to binary warnings were studied in a categorization task that resembled the control of a simulated production environment. Students performed a visual signal detection task and were aided by a binary warning indicator. Experimental conditions differed in the validity of the warning and its proximity to the judged stimulus. Participants' performance improved over the course of the experiment, and they partly adjusted their responses to the validity of the warnings but continued to respond to nonvalid warnings throughout the experiment. It was particularly difficult to ignore the nonvalid information when it was integrated with the continuous information. There was evidence for nonoptimal use of the information from the warning system, whether it was valid or not valid. The results indicate a possible distinction between two dimensions of users' trust in warning systems: compliance and reliance. Actual or potential implications of this research include improved warning design based on analysis of system and operator characteristics.     

A user-centred approach to the design and evaluation of auditory warning signals: 1. Methodology

This paper presents a method for the design and evaluation of auditory warning signals modelled on an existing internationally standardized method for evaluating public information systems (ISO/DIS 7001: 1979). The procedure is essentially user-centred, capitalizing upon users' associations between sounds and their meanings. The procedure is presented in a step-by-step manner, from the initial identification of referents for which warnings might be required, through the generation of ideas for warning sounds, an appropriateness ranking test, a learning and confusion test, an urgency mapping test, a recognition test and an operational test. Practical issues are discussed with respect to each of the stages, and suggestions are made as to courses of action that might be taken if problems are encountered. Theory behind the relationship between sound and meaning is discussed with reference to the practical issues addressed.     

Using technology to facilitate the design and delivery of warnings

This paper describes several ways in which new technologies can assist in the design and delivery of warnings. There are four discussion points: (1) current product information can be delivered via the Internet; (2) computer software and hardware are available to assist in the design, construction, and production of visual and auditory warnings; (3) various detection devices can be used to recognize instances in which warnings might be delivered; and (4) a warning presentation can be modified to fit conditions and persons. Implications, example applications and future prospects of these points are described.     

Effects of in-vehicle warning information displays with or without spatial compatibility on driving behaviors and response performance

A driving simulator study was conducted to evaluate the effects of five in-vehicle warning information displays upon drivers' emergent response and decision performance. These displays include visual display, auditory displays with and without spatial compatibility, hybrid displays in both visual and auditory format with and without spatial compatibility. Thirty volunteer drivers were recruited to perform various tasks that involved driving, stimulus-response, divided attention and stress rating. Results show that for displays of single-modality, drivers benefited more when coping with visual display of warning information than auditory display with or without spatial compatibility. However, auditory display with spatial compatibility significantly improved drivers' performance in reacting to the divided attention task and making accurate S-R task decision. Drivers' best performance results were obtained for hybrid display with spatial compatibility. Hybrid displays enabled drivers to respond the fastest and achieve the best accuracy in both S-R and divided attention tasks.     

Auditory signals in military aircraft: ergonomics principles versus practice

The complete ensembles of auditory signals in selected USAF aircraft (the F-4D, F-15, two models of the F-16, the C-5, and the C-141) are described and evaluated. Human factors research related to the design of speech and non-speech auditory signals is reviewed. Major findings are: that auditory signals are not well standardised among the aircraft, even between those with similar combat roles; that a relatively large number of non-speech auditory signals are used, which may make it difficult for the aircrew to recall the meanings of all the signals; that some non-speech signals are sufficiently similar that they may be confused, particularly in high workload and stressful conditions; and that the criticality of the warnings is not reliably indicated by any characteristic of the signals. Four problem areas requiring further research are discussed: reduction of signal loudness, enhancement of the distinctiveness and masking resistance of non-speech signals, effects of concurrent warning signals on aircrew performance, and additional uses of auditory information.