Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an invehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying ‘ahead’. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

A new class of auditory warning signals for complex systems: auditory icons

This simulator-based study examined conventional auditory warnings (tonal, nonverbal sounds) and auditory icons (representational, nonverbal sounds), alone and in combination with a dash-mounted visual display, to present information about impending collision situations to commercial motor vehicle operators. Brake response times were measured for impending front-to-rear collision scenarios under 6 display configurations, 2 vehicle speeds, and 2 levels of headway. Accident occurrence was measured for impending side collision scenarios under 2 vehicle speeds, 2 levels of visual workload, 2 auditory displays, absence/presence of mirrors, and absence/presence of a dash-mounted iconic visual display. For both front-to-rear and side collision scenarios, auditory icons elicited significantly improved driver performance over conventional auditory warnings. Driver performance improved when collision warning information was presented through multiple modalities. Brake response times were significantly faster for impending front-to-rear collision scenarios using the longer headway condition. The presence of mirrors significantly reduced the number of accidents for impending side collision scenarios. Subjective preference data indicated that participants preferred multimodal displays over single-modality displays. Actual or potential applications for this research include auditory displays and warnings, information presentation, and the development of alternative user interfaces.     

Semantic congruency of auditory warnings

Abstract

The aim of this study was to explore operator experience and performance for semantically congruent and incongruent auditory icons and abstract alarm sounds. It was expected that performance advantages for congruent sounds would be present initially but would reduce over time for both alarm types. Twenty-four participants (12M/12F) were placed into auditory icon or abstract alarm groupings. For each group both congruent and incongruent alarms were used to represent different driving task scenarios. Once sounded, participants were required to respond to each alarm by selecting a corresponding driving scenario. User performance for all sound types improved over time, however even with experience a decrement in speed of response remained for the incongruent iconic sounds and in accuracy of performance for the abstract warning sounds when compared to the congruent auditory icons. Semantic congruency was found to be of more importance for auditory icons than for abstract sounds.

Practitioner Summary: Alarms are used in many operating systems as emergency, alerting, or continuous monitoring signals for instance. This study found that the type and representativeness of an auditory warning will influence operator performance over time. Semantically congruent iconic sounds produced performance advantages over both incongruent iconic sounds and abstract warnings.     

Mapping candidate within-vehicle auditory displays to their referents

OBJECTIVE: This study investigated speech, environmental sounds (naturally occurring sounds with arbitrary meanings), auditory icons (natural or synthetic sounds with specific meanings), and abstract synthetic warnings as candidates for within-vehicle interfaces. BACKGROUND: Auditory displays and warnings must satisfy certain criteria, such as being appropriately urgent and commanding appropriately fast response times. However, a semiotic analysis suggests that displays, as signals interpreted by users, should also be mapped successfully onto their referents. METHOD: Response times and accuracy were recorded in a computer task of identifying learned mappings of candidate displays to a range of referent driving events (such as "headway closing"); perceived urgency and pleasantness were assessed separately. RESULTS: Speech and auditory icons produced near-ceiling performance in response times and identification accuracy. Abstract sounds produced notably slower response times and less accuracy. Environmental sounds showed an intermediate pattern of performance for accuracy, but the response times were similar to those of the abstract sounds. Speech utterances were similarly and consistently rated as pleasant but also of intermediate perceived urgency. The three other sound types showed a consistent mapping of their perceived urgency to the situational urgency of their referents; for these sounds, perceived urgency and pleasantness were negatively correlated. CONCLUSION: The results point to the importance of considering the role of signal-referent relationships in designing auditory displays. APPLICATION: The results have applicability for auditory displays in the vehicle interface, whereas the theoretical framework is of value in auditory display design in a broader context.     

Cough once for danger: icons versus abstract warnings as informative alerts in civil aviation

OBJECTIVE: An experiment investigated the efficacy of auditory icons as warning signals in an aviation context. BACKGROUND: Iconic signals, such as a cough to signal dangerous levels of carbon monoxide, convey information about the nature of an incident and alert the operator that there is a problem, whereas signals that are arbitrarily associated with a critical incident provide relatively less information. Warning recognition speed and accuracy are likely to be influenced by modality of presentation (visual, auditory, auditory + visual) and task demand (low, high). METHODS: The 172 participants completed a computer-based training session and test task that involved responding to abstract or iconic auditory (1 s), visual, or auditory + visual warnings associated with seven critical incidents while performing low- and high-demand concurrent tasks. RESULTS: Significantly fewer training trials were required to learn iconic warnings than abstract warnings. An advantage for iconic warnings persisted into the test phase, evident most consistently as greater warning recognition accuracy. The effect was observed in both high- and low-demand conditions. Auditory abstract warnings, in particular, elicited slow reaction times and poor accuracy. CONCLUSION: Associations between a small number of meaningful environmental sounds and critical incidents can be learned with ease relative to more abstract associations, although training is required and response times are relatively slow. APPLICATION: Sets of distinctive auditory iconic warnings can be designed to alert and inform pilots about non-time-pressured events. Potential applications of language-neutral icons as informative warnings include civil, commercial, and defense aircraft.     

A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving

OBJECTIVE: This study examined the effectiveness of rear-end collision warnings presented in different sensory modalities as a function of warning timing in a driving simulator. BACKGROUND: The proliferation of in-vehicle information and entertainment systems threatens driver attention and may increase the risk of rear-end collisions. Collision warning systems have been shown to improve inattentive and/or distracted driver response time (RT) in rear-end collision situations. However, most previous rear-end collision warning research has not directly compared auditory, visual, and tactile warnings. METHOD: Sixteen participants in a fixed-base driving simulator experienced four warning conditions: no warning, visual, auditory, and tactile. The warnings activated when the time-to-collision (TTC) reached a critical threshold of 3.0 or 5.0 s. Driver RT was captured from a warning below critical threshold to brake initiation. RESULTS: Drivers with a tactile warning had the shortest mean RT. Drivers with a tactile warning had significantly shorter RT than drivers without a warning and had a significant advantage over drivers with visual warnings. CONCLUSION: Tactile warnings show promise as effective rear-end collision warnings. APPLICATION: The results of this study can be applied to the future design and evaluation of automotive warnings designed to reduce rear-end collisions.     

Effects of single versus multiple warnings on driver performance

OBJECTIVE: To explore how a single master alarm system affects drivers' responses when compared with multiple, distinct warnings. BACKGROUND: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event type on driver performance. METHOD: Using a 2 x 2 x 4 mixed factorial design, we investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers' performance across four collision event types (frontal collision warnings, left and right lane departure warnings, and warnings for a fast-approaching following vehicle). RESULTS: Participants' reaction times and accuracy rates were significantly affected by the type of collision event and alarm reliability. The use of individual alarms, rather than a single master alarm, did not significantly affect driving performance in terms of reaction time or response accuracy. CONCLUSION: Even though a master alarm is a relatively uninformative warning, it produced statistically no different reaction times or accuracy results when compared with information-rich auditory icons, some of which were spatially located. In addition, unreliable alarms negatively impacted driver performance, regardless of event type or alarm scheme. APPLICATION: These results have important implications for the development and implementation of multiple driver warning systems.     

Auditory alerts for in-vehicle information systems: The effects of temporal conflict and sound parameters on driver attitudes and performance

In-vehicle information systems will soon confront drivers with an increasing number of warnings and alerts for situations ranging from imminent collisions to the arrival of e-mail messages. Coordinating these alerts can ensure that they enhance rather than degrade driving safety. Two experiments examined how temporal conflict and sound parameters affect driver performance and acceptance. The temporal conflict of an e-mail alert occurring 300?ms before a collision warning interfered with the response to the collision warning, but an e-mail alert occurring 1000?ms before the collision warning had the opposite effect and enhanced the response to the collision warning. These results emphasize the need to consider how in-vehicle devices influence drivers' strategic anticipation of high-demand situations. Regarding sound parameters, results showed that highly urgent sounds tended to speed drivers' accelerator release, but the annoyance associated with highly urgent sounds increased workload. In fact, there was a strong positive association between ratings of annoyance and subjective workload. Consistent with the urgency mapping principle, there was a slight negative association between the differences in the rated urgency of collision warnings and e-mail alerts and subjective workload. The results suggest that warning and alert design should consider an annoyance trade-off in addition to urgency mapping.     

Auditory alerts for in-vehicle information systems: the effects of temporal conflict and sound parameters on driver attitudes and performance

In-vehicle information systems will soon confront drivers with an increasing number of warnings and alerts for situations ranging from imminent collisions to the arrival of e-mail messages. Coordinating these alerts can ensure that they enhance rather than degrade driving safety. Two experiments examined how temporal conflict and sound parameters affect driver performance and acceptance. The temporal conflict of an e-mail alert occurring 300 ms before a collision warning interfered with the response to the collision warning, but an e-mail alert occurring 1000 ms before the collision warning had the opposite effect and enhanced the response to the collision warning. These results emphasize the need to consider how in-vehicle devices influence drivers' strategic anticipation of high-demand situations. Regarding sound parameters, results showed that highly urgent sounds tended to speed drivers' accelerator release, but the annoyance associated with highly urgent sounds increased workload. In fact, there was a strong positive association between ratings of annoyance and subjective workload. Consistent with the urgency mapping principle, there was a slight negative association between the differences in the rated urgency of collision warnings and e-mail alerts and subjective workload. The results suggest that warning and alert design should consider an annoyance trade-off in addition to urgency mapping.     

Learning and retention of associations between auditory icons and denotative referents: implications for the design of auditory warnings

OBJECTIVE: This study examined the way in which the type and preexisting strength of association between an auditory icon and a warning event affects the ease with which the icon/event pairing can be learned and retained. BACKGROUND: To be effective, an auditory warning must be audible, identifiable, interpretable, and heeded. Warnings consisting of familiar environmental sounds, or auditory icons, have potential to facilitate identification and interpretation. The ease with which pairings between auditory icons and warning events can be learned and retained is likely to depend on the type and strength of the preexisting icon/event association. METHOD: Sixty-three participants each learned eight auditory-icon/denotative-referent pairings and attempted to recall them 4 weeks later. Three icon/denotative-referent association types (direct, related, and unrelated) were employed. Participants rated the strength of the association for each pairing on a 7-point scale. RESULTS: The number of errors made while learning pairings was greater for unrelated than for either related or direct associations, whereas the number of errors made while attempting to recall pairings 4 weeks later was greater for unrelated than for related associations and for related than for direct associations. Irrespective of association type, both learning and retention performance remained at very high levels, provided the strength of the association was rated greater than 5. CONCLUSION: This suggests that strong preexisting associations are used to facilitate learning and retention of icon/denotative-referent pairings. APPLICATION: The practical implication of this study is that auditory icons having either direct or strong, indirect associations with warning events should be preferred.     

Learning auditory warnings: The effects of sound type, verbal labelling and imagery on the identification of alarm sounds

This study looks at the effect of sound type, training cue and cue source on people's ability to learn and retain a set of twelve warning-type sounds. Each participant was required to learn and retain each of the twelve sounds using one of four learning methods. One group was required to learn the sounds according to verbal labels given to them by the experimenter; a second group generated their own verbal labels; a third group was required to learn the sounds using graphic images (the waveform of the sound) given to them by the experimenter; and the fourth group was allowed to generate their own graphic images. Three classes of sound were tested: real, environmental sounds, semi-abstract monitoring-type sounds, and abstract sounds already used as hospital warnings. The results show that the verbal labels worked better than graphic images, but only when these were given by the experimenter. When participants generated their own cues, performance was equally good in both graphic and verbal conditions. Across the sounds, it was found that real sounds were easier to learn than the other two groups, although this effect again disappeared when participants were allowed to generate their own cues. The implications for auditory warning design and training are discussed within a broader theoretical and practical context.

Relevance to industry

This paper shows that the difficulty with which alarm-type sounds can be learned depends not only on the type of sounds to be learned, but the way in which they are learnt. Apparent advantages to using environmental sounds and verbal labels disappears as soon as participants are allowed to generate their own learning aids.     

Impact of spatial auditory feedback on the efficiency of iconic human–computer interfaces under conditions of visual impairment

This paper investigates the addition of spatial auditory feedback as a tool to assist people with visual impairments in the use of computers, specifically in tasks involving iconic visual search. In this augmented interface, unique sounds were mapped to visual icons on the screen. As the screen cursor traversed the screen, the user heard sounds of nearby icons, spatially, according to the relative position of each icon with respect to the screen cursor. A software prototype of the design was developed to evaluate the performance of users in the search of icons within the proposed interface. Experiments were conducted with simulated visual impairments on volunteer participants to evaluate if the addition of spatial auditory feedback makes the interface more accessible to users with impaired vision. Results demonstrated that spatialization of icon sounds provides additional remote navigational information to users, enabling new strategies for task completion. Directions for future research are discussed and prioritized.     

Intelligibility of speech in a virtual 3-D environment

In a simulated air traffic control task, improvement in the detection of auditory warnings when using virtual 3-D audio depended on the spatial configuration of the sounds. Performance improved substantially when two of four sources were placed to the left and the remaining two were placed to the right of the participant. Surprisingly, little or no benefits were observed for configurations involving the elevation or transverse (front/back) dimensions of virtual space, suggesting that position on the interaural (left/right) axis is the crucial factor to consider in auditory display design. The relative importance of interaural spacing effects was corroborated in a second, free-field (real space) experiment. Two additional experiments showed that (a) positioning signals to the side of the listener is superior to placing them in front even when two sounds are presented in the same location, and (b) the optimal distance on the interaural axis varies with the amplitude of the sounds. These results are well predicted by the behavior of an ideal observer under the different display conditions. This suggests that guidelines for auditory display design that allow for effective perception of speech information can be developed from an analysis of the physical sound patterns.     

A user-centred approach to the design and evaluation of auditory warning signals: 1. Methodology

This paper presents a method for the design and evaluation of auditory warning signals modelled on an existing internationally standardized method for evaluating public information systems (ISO/DIS 7001: 1979). The procedure is essentially user-centred, capitalizing upon users' associations between sounds and their meanings. The procedure is presented in a step-by-step manner, from the initial identification of referents for which warnings might be required, through the generation of ideas for warning sounds, an appropriateness ranking test, a learning and confusion test, an urgency mapping test, a recognition test and an operational test. Practical issues are discussed with respect to each of the stages, and suggestions are made as to courses of action that might be taken if problems are encountered. Theory behind the relationship between sound and meaning is discussed with reference to the practical issues addressed.     

Auditory Icons: Using Sound in Computer Interfaces

There is growing interest in the use of sound to convey information in computer interfaces. The strategies employed thus far have been based on an understanding of sound that leads to either an arbitrary or metaphorical relation between the sounds used and the data to be represented. In this article, an alternative approach to the use of sound in computer interfaces is outlined, one that emphasizes the role of sound in conveying information about the world to the listener. According to this approach, auditory icons, caricatures of naturally occurring sounds, could be used to provide information about sources of data. Auditory icons provide a natural way to represent dimensional data as well as conceptual objects in a computer system. They allow categorization of data into distinct families, using a single sound. Perhaps the most important advantage of this strategy is that it is based on the way people listen to the world in their everyday lives.     

Collision warning timing,driver distraction,and driver response to imminent rear-end collisions in a high-fidelity driving simulator

Rear-end collisions account for almost 30% of automotive crashes. Rear-end collision avoidance systems (RECASs) may offer a promising approach to help drivers avoid these crashes. Two experiments performed using a high-fidelity motion-based driving simulator examined driver responses to evaluate the efficacy of a RECAS. The first experiment showed that early warnings helped distracted drivers react more quickly--and thereby avoid more collisions--than did late warnings or no warnings. Compared with the no-warning condition, an early RECAS warning reduced the number of collisions by 80.7%. Assuming collision severity is proportional to kinetic energy, the early warning reduced collision severity by 96.5%. In contrast, the late warning reduced collisions by 50.0% and the corresponding severity by 87.5%. The second experiment showed that RECAS benefits even undistracted drivers. Analysis of the braking process showed that warnings provide a potential safety benefit by reducing the time required for drivers to release the accelerator. Warnings do not, however, speed application of the brake, increase maximum deceleration, or affect mean deceleration. These results provide the basis for a computational model of driver performance that was used to extrapolate the findings and identify the most promising parameter settings. Potential applications of these results include methods for evaluating collision warning systems, algorithm design guidance, and driver performance model input.     

A framework for the design of ambulance sirens

Ambulance sirens are essential for assisting the safe and rapid arrival of an ambulance at the scene of an emergency. In this study, the parameters upon which sirens may be designed were examined and a framework for emergency vehicle siren design was proposed. Validity for the framework was supported through acoustic measurements and the evaluation of ambulance transit times over 240 emergency runs using two different siren systems. Modifying existing siren sounds to add high frequency content would improve vehicle penetration, detectability and sound localization cues, and mounting the siren behind the radiator grill, rather than on the light bar or under the wheel arch, would provide less unwanted noise while maintaining or improving the effective distance in front of the vehicle. Ultimately, these considerations will benefit any new attempt to design auditory warnings for the emergency services.     

A psychophysiological evaluation of the perceived urgency of auditory warning signals

One significant concern that pilots have about cockpit auditory warnings is that the signals presently used lack a sense of priority. The relationship between auditory warning sound parameters and perceived urgency is, therefore, an important topic of enquiry in aviation psychology. The present investigation examined the relationship among subjective assessments of urgency, reaction time, and brainwave activity with three auditory warning signals. Subjects performed a tracking task involving automated and manual conditions, and were presented with auditory warnings having various levels of perceived and situational urgency. Subjective assessments revealed that subjects were able to rank warnings on an urgency scale, but rankings were altered after warnings were mapped to a situational urgency scale. Reaction times differed between automated and manual tracking task conditions, and physiological data showed attentional differences in response to perceived and situational warning urgency levels. This study shows that the use of physiological measures sensitive to attention and arousal, in conjunction with behavioural and subjective measures, may lead to the design of auditory warnings that produce a sense of urgency in an operator that matches the urgency of the situation.     

Alerts for in-vehicle information systems: annoyance, urgency, and appropriateness

OBJECTIVE: This study assesses the influence of the auditory characteristics of alerts on perceived urgency and annoyance and whether these perceptions depend on the context in which the alert is received. BACKGROUND: Alert parameters systematically affect perceived urgency, and mapping the urgency of a situation to the perceived urgency of an alert is a useful design consideration. Annoyance associated with environmental noise has been thoroughly studied, but little research has addressed whether alert parameters differentially affect annoyance and urgency. METHOD: Three 2(3) x 3 mixed within/between factorial experiments, with a total of 72 participants, investigated nine alert parameters in three driving contexts. These parameters were formant (similar to harmonic series), pulse duration, interpulse interval, alert onset and offset, burst duty cycle, alert duty cycle, interburst period, and sound type. Imagined collision warning, navigation alert, and E-mail notification scenarios defined the driving context. RESULTS: All parameters influenced both perceived urgency and annoyance (p < .05), with pulse duration, interpulse interval, alert duty cycle, and sound type influencing urgency substantially more than annoyance. There was strong relationship between perceived urgency and rated appropriateness for high-urgency driving scenarios and a strong relationship between annoyance and rated appropriateness for low-urgency driving scenarios. CONCLUSION: Sound parameters differentially affect annoyance and urgency. Also, urgency and annoyance differentially affect perceived appropriateness of warnings. APPLICATION: Annoyance may merit as much attention as urgency in the design of auditory warnings, particularly in systems that alert drivers to relatively low-urgency situations.     

Multiple spatial sounds in hierarchical menu navigation for visually impaired computer users

This paper describes a user study on the benefits and drawbacks of simultaneous spatial sounds in auditory interfaces for visually impaired and blind computer users. Two different auditory interfaces in spatial and non-spatial condition were proposed to represent the hierarchical menu structure of a simple word processing application. In the horizontal interface, the sound sources or the menu items were located in the horizontal plane on a virtual ring surrounding the user’s head, while the sound sources in the vertical interface were aligned one above the other in front of the user. In the vertical interface, the central pitch of the sound sources at different elevations was changed in order to improve the otherwise relatively low localization performance in the vertical dimension. The interaction with the interfaces was based on a standard computer keyboard for input and a pair of studio headphones for output. Twelve blind or visually impaired test subjects were asked to perform ten different word processing tasks within four experiment conditions. Task completion times, navigation performance, overall satisfaction and cognitive workload were evaluated. The initial hypothesis, i.e. that the spatial auditory interfaces with multiple simultaneous sounds should prove to be faster and more efficient than non-spatial ones, was not confirmed. On the contrary—spatial auditory interfaces proved to be significantly slower due to the high cognitive workload and temporal demand. The majority of users did in fact finish tasks with less navigation and key pressing; however, they required much more time. They reported the spatial auditory interfaces to be hard to use for a longer period of time due to the high temporal and mental demand, especially with regards to the comprehension of multiple simultaneous sounds. The comparison between the horizontal and vertical interface showed no significant differences between the two. It is important to point out that all participants were novice users of the system; therefore it is possible that the overall performance could change with a more extensive use of the interfaces and an increased number of trials or experiments sets. Our interviews with visually impaired and blind computer users showed that they are used to sharing their auditory channel in order to perform multiple simultaneous tasks such as listening to the radio, talking to somebody, using the computer, etc. As the perception of multiple simultaneous sounds requires the entire capacity of the auditory channel and total concentration of the listener, it does therefore not enable such multitasking.