Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an invehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying ‘ahead’. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application.

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an in-vehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying 'ahead'. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

Cough once for danger: icons versus abstract warnings as informative alerts in civil aviation

OBJECTIVE: An experiment investigated the efficacy of auditory icons as warning signals in an aviation context. BACKGROUND: Iconic signals, such as a cough to signal dangerous levels of carbon monoxide, convey information about the nature of an incident and alert the operator that there is a problem, whereas signals that are arbitrarily associated with a critical incident provide relatively less information. Warning recognition speed and accuracy are likely to be influenced by modality of presentation (visual, auditory, auditory + visual) and task demand (low, high). METHODS: The 172 participants completed a computer-based training session and test task that involved responding to abstract or iconic auditory (1 s), visual, or auditory + visual warnings associated with seven critical incidents while performing low- and high-demand concurrent tasks. RESULTS: Significantly fewer training trials were required to learn iconic warnings than abstract warnings. An advantage for iconic warnings persisted into the test phase, evident most consistently as greater warning recognition accuracy. The effect was observed in both high- and low-demand conditions. Auditory abstract warnings, in particular, elicited slow reaction times and poor accuracy. CONCLUSION: Associations between a small number of meaningful environmental sounds and critical incidents can be learned with ease relative to more abstract associations, although training is required and response times are relatively slow. APPLICATION: Sets of distinctive auditory iconic warnings can be designed to alert and inform pilots about non-time-pressured events. Potential applications of language-neutral icons as informative warnings include civil, commercial, and defense aircraft.     

Semantic congruency of auditory warnings

Abstract

The aim of this study was to explore operator experience and performance for semantically congruent and incongruent auditory icons and abstract alarm sounds. It was expected that performance advantages for congruent sounds would be present initially but would reduce over time for both alarm types. Twenty-four participants (12M/12F) were placed into auditory icon or abstract alarm groupings. For each group both congruent and incongruent alarms were used to represent different driving task scenarios. Once sounded, participants were required to respond to each alarm by selecting a corresponding driving scenario. User performance for all sound types improved over time, however even with experience a decrement in speed of response remained for the incongruent iconic sounds and in accuracy of performance for the abstract warning sounds when compared to the congruent auditory icons. Semantic congruency was found to be of more importance for auditory icons than for abstract sounds.

Practitioner Summary: Alarms are used in many operating systems as emergency, alerting, or continuous monitoring signals for instance. This study found that the type and representativeness of an auditory warning will influence operator performance over time. Semantically congruent iconic sounds produced performance advantages over both incongruent iconic sounds and abstract warnings.     

A new class of auditory warning signals for complex systems: auditory icons

This simulator-based study examined conventional auditory warnings (tonal, nonverbal sounds) and auditory icons (representational, nonverbal sounds), alone and in combination with a dash-mounted visual display, to present information about impending collision situations to commercial motor vehicle operators. Brake response times were measured for impending front-to-rear collision scenarios under 6 display configurations, 2 vehicle speeds, and 2 levels of headway. Accident occurrence was measured for impending side collision scenarios under 2 vehicle speeds, 2 levels of visual workload, 2 auditory displays, absence/presence of mirrors, and absence/presence of a dash-mounted iconic visual display. For both front-to-rear and side collision scenarios, auditory icons elicited significantly improved driver performance over conventional auditory warnings. Driver performance improved when collision warning information was presented through multiple modalities. Brake response times were significantly faster for impending front-to-rear collision scenarios using the longer headway condition. The presence of mirrors significantly reduced the number of accidents for impending side collision scenarios. Subjective preference data indicated that participants preferred multimodal displays over single-modality displays. Actual or potential applications for this research include auditory displays and warnings, information presentation, and the development of alternative user interfaces.     

A psychophysiological evaluation of the perceived urgency of auditory warning signals

One significant concern that pilots have about cockpit auditory warnings is that the signals presently used lack a sense of priority. The relationship between auditory warning sound parameters and perceived urgency is, therefore, an important topic of enquiry in aviation psychology. The present investigation examined the relationship among subjective assessments of urgency, reaction time, and brainwave activity with three auditory warning signals. Subjects performed a tracking task involving automated and manual conditions, and were presented with auditory warnings having various levels of perceived and situational urgency. Subjective assessments revealed that subjects were able to rank warnings on an urgency scale, but rankings were altered after warnings were mapped to a situational urgency scale. Reaction times differed between automated and manual tracking task conditions, and physiological data showed attentional differences in response to perceived and situational warning urgency levels. This study shows that the use of physiological measures sensitive to attention and arousal, in conjunction with behavioural and subjective measures, may lead to the design of auditory warnings that produce a sense of urgency in an operator that matches the urgency of the situation.     

A user-centred approach to the design and evaluation of auditory warning signals: 1. Methodology

This paper presents a method for the design and evaluation of auditory warning signals modelled on an existing internationally standardized method for evaluating public information systems (ISO/DIS 7001: 1979). The procedure is essentially user-centred, capitalizing upon users' associations between sounds and their meanings. The procedure is presented in a step-by-step manner, from the initial identification of referents for which warnings might be required, through the generation of ideas for warning sounds, an appropriateness ranking test, a learning and confusion test, an urgency mapping test, a recognition test and an operational test. Practical issues are discussed with respect to each of the stages, and suggestions are made as to courses of action that might be taken if problems are encountered. Theory behind the relationship between sound and meaning is discussed with reference to the practical issues addressed.     

Effects of single versus multiple warnings on driver performance

OBJECTIVE: To explore how a single master alarm system affects drivers' responses when compared with multiple, distinct warnings. BACKGROUND: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event type on driver performance. METHOD: Using a 2 x 2 x 4 mixed factorial design, we investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers' performance across four collision event types (frontal collision warnings, left and right lane departure warnings, and warnings for a fast-approaching following vehicle). RESULTS: Participants' reaction times and accuracy rates were significantly affected by the type of collision event and alarm reliability. The use of individual alarms, rather than a single master alarm, did not significantly affect driving performance in terms of reaction time or response accuracy. CONCLUSION: Even though a master alarm is a relatively uninformative warning, it produced statistically no different reaction times or accuracy results when compared with information-rich auditory icons, some of which were spatially located. In addition, unreliable alarms negatively impacted driver performance, regardless of event type or alarm scheme. APPLICATION: These results have important implications for the development and implementation of multiple driver warning systems.     

Verbal collision avoidance messages during simulated driving: perceived urgency, alerting effectiveness and annoyance

Matching the perceived urgency of an alert with the relative hazard level of the situation is critical for effective alarm response. Two experiments describe the impact of acoustic and semantic parameters on ratings of perceived urgency, annoyance and alerting effectiveness and on alarm response speed. Within a simulated driving context, participants rated and responded to collision avoidance system (CAS) messages spoken by a female or male voice (experiments 1 and 2, respectively). Results indicated greater perceived urgency and faster alarm response times as intensity increased from -2 dB signal to noise (S/N) ratio to +10 dB S/N, although annoyance ratings increased as well. CAS semantic content interacted with alarm intensity, indicating that at lower intensity levels participants paid more attention to the semantic content. Results indicate that both acoustic and semantic parameters independently and interactively impact CAS alert perceptions in divided attention conditions and this work can inform auditory alarm design for effective hazard matching. Matching the perceived urgency of an alert with the relative hazard level of the situation is critical for effective alarm response. Here, both acoustic and semantic parameters independently and interactively impacted CAS alert perceptions in divided attention conditions. This work can inform auditory alarm design for effective hazard matching. STATEMENT OF RELEVANCE: Results indicate that both acoustic parameters and semantic content can be used to design collision warnings with a range of urgency levels. Further, these results indicate that verbal warnings tailored to a specific hazard situation may improve hazard-matching capabilities without substantial trade-offs in perceived annoyance.     

Alerts for in-vehicle information systems: annoyance, urgency, and appropriateness

OBJECTIVE: This study assesses the influence of the auditory characteristics of alerts on perceived urgency and annoyance and whether these perceptions depend on the context in which the alert is received. BACKGROUND: Alert parameters systematically affect perceived urgency, and mapping the urgency of a situation to the perceived urgency of an alert is a useful design consideration. Annoyance associated with environmental noise has been thoroughly studied, but little research has addressed whether alert parameters differentially affect annoyance and urgency. METHOD: Three 2(3) x 3 mixed within/between factorial experiments, with a total of 72 participants, investigated nine alert parameters in three driving contexts. These parameters were formant (similar to harmonic series), pulse duration, interpulse interval, alert onset and offset, burst duty cycle, alert duty cycle, interburst period, and sound type. Imagined collision warning, navigation alert, and E-mail notification scenarios defined the driving context. RESULTS: All parameters influenced both perceived urgency and annoyance (p < .05), with pulse duration, interpulse interval, alert duty cycle, and sound type influencing urgency substantially more than annoyance. There was strong relationship between perceived urgency and rated appropriateness for high-urgency driving scenarios and a strong relationship between annoyance and rated appropriateness for low-urgency driving scenarios. CONCLUSION: Sound parameters differentially affect annoyance and urgency. Also, urgency and annoyance differentially affect perceived appropriateness of warnings. APPLICATION: Annoyance may merit as much attention as urgency in the design of auditory warnings, particularly in systems that alert drivers to relatively low-urgency situations.     

Localizable auditory warning pulses

Three important forms of information available to the listener may be identified in any auditory warning: what (semantic), where (location) and when (perceived urgency). Each form is addressed in the present design study of auditory warning pulses. Recordings were made via a dummy head, and were presented dichotically to listeners in a left/right localization task. The first experiment identified a suitable notched noise component for providing location information in the pulse stimulus. The second experiment required participants to simultaneously identify a distinct tonal signature and the location of the sound when they were presented with one of three tonal types, or one of three compound (tone plus noise) stimuli. Response accuracy and response latency for this identification and localization task were significantly better with the compound than with tone alone stimuli. Perceived urgency of compound complex tone plus noise stimuli was investigated in the third experiment. While there may be a trade-off between localization acuity and perceived urgency, the addition of noise components to the auditory warning pulse was shown to enhance the location information available to the listener. It is suggested that some auditory warning designs will benefit from the simultaneous provision of what and where forms of information in the sounds.     

Localizable auditory warning pulses

Three important forms of information available to the listener may be identified in any auditory warning: what (semantic), where (location) and when (perceived urgency). Each form is addressed in the present design study of auditory warning pulses. Recordings were made via a dummy head, and were presented dichotically to listeners in a left/right localization task. The first experiment identified a suitable notched noise component for providing location information in the pulse stimulus. The second experiment required participants to simultaneously identify a distinct tonal signature and the location of the sound when they were presented with one of three tonal types, or one of three compound (tone plus noise) stimuli. Response accuracy and response latency for this identification and localization task were significantly better with the compound than with tone alone stimuli. Perceived urgency of compound complex tone plus noise stimuli was investigated in the third experiment. While there may be a trade-off between localization acuity and perceived urgency, the addition of noise components to the auditory warning pulse was shown to enhance the location information available to the listener. It is suggested that some auditory warning designs will benefit from the simultaneous provision of what and where forms of information in the sounds.     

On using psychophysical techniques to achieve urgency mapping in auditory warnings

It is well established that warning implementation should aim to achieve urgency mapping between the perceived urgency of the warning itself and the situational urgency of the condition that it indicates. This paper describes how Stevens Power Law [Psychological Review, 64, 153 181, 1957], which quantifies the relationship between objective parameters (such as the pitch of a warning) and subjective parameters (such as perceived urgency), can be applied to the design of auditory warnings to facilitate such urgency mapping. Studies that have quantified and predicted the effects of different warning parameters on perceived urgency using an application of Stevens Power Law are reported.     

Impact of spatial auditory feedback on the efficiency of iconic human–computer interfaces under conditions of visual impairment

This paper investigates the addition of spatial auditory feedback as a tool to assist people with visual impairments in the use of computers, specifically in tasks involving iconic visual search. In this augmented interface, unique sounds were mapped to visual icons on the screen. As the screen cursor traversed the screen, the user heard sounds of nearby icons, spatially, according to the relative position of each icon with respect to the screen cursor. A software prototype of the design was developed to evaluate the performance of users in the search of icons within the proposed interface. Experiments were conducted with simulated visual impairments on volunteer participants to evaluate if the addition of spatial auditory feedback makes the interface more accessible to users with impaired vision. Results demonstrated that spatialization of icon sounds provides additional remote navigational information to users, enabling new strategies for task completion. Directions for future research are discussed and prioritized.     

Auditory alerts for in-vehicle information systems: The effects of temporal conflict and sound parameters on driver attitudes and performance

In-vehicle information systems will soon confront drivers with an increasing number of warnings and alerts for situations ranging from imminent collisions to the arrival of e-mail messages. Coordinating these alerts can ensure that they enhance rather than degrade driving safety. Two experiments examined how temporal conflict and sound parameters affect driver performance and acceptance. The temporal conflict of an e-mail alert occurring 300?ms before a collision warning interfered with the response to the collision warning, but an e-mail alert occurring 1000?ms before the collision warning had the opposite effect and enhanced the response to the collision warning. These results emphasize the need to consider how in-vehicle devices influence drivers' strategic anticipation of high-demand situations. Regarding sound parameters, results showed that highly urgent sounds tended to speed drivers' accelerator release, but the annoyance associated with highly urgent sounds increased workload. In fact, there was a strong positive association between ratings of annoyance and subjective workload. Consistent with the urgency mapping principle, there was a slight negative association between the differences in the rated urgency of collision warnings and e-mail alerts and subjective workload. The results suggest that warning and alert design should consider an annoyance trade-off in addition to urgency mapping.     

Auditory alerts for in-vehicle information systems: the effects of temporal conflict and sound parameters on driver attitudes and performance

In-vehicle information systems will soon confront drivers with an increasing number of warnings and alerts for situations ranging from imminent collisions to the arrival of e-mail messages. Coordinating these alerts can ensure that they enhance rather than degrade driving safety. Two experiments examined how temporal conflict and sound parameters affect driver performance and acceptance. The temporal conflict of an e-mail alert occurring 300 ms before a collision warning interfered with the response to the collision warning, but an e-mail alert occurring 1000 ms before the collision warning had the opposite effect and enhanced the response to the collision warning. These results emphasize the need to consider how in-vehicle devices influence drivers' strategic anticipation of high-demand situations. Regarding sound parameters, results showed that highly urgent sounds tended to speed drivers' accelerator release, but the annoyance associated with highly urgent sounds increased workload. In fact, there was a strong positive association between ratings of annoyance and subjective workload. Consistent with the urgency mapping principle, there was a slight negative association between the differences in the rated urgency of collision warnings and e-mail alerts and subjective workload. The results suggest that warning and alert design should consider an annoyance trade-off in addition to urgency mapping.     

Perceived urgency of and response time to multi-tone and frequency-modulated warning signals in broadband noise

Several pulse parameters that were believed to affect the perceived urgency and response time to auditory warning signals were investigated in a factorial experiment. The independent variables included pulse format (sequential, simultaneous, and frequency-modulated pulses), pulse level (65 and 79dBC), and inter-pulse interval (0, 150 and 300 ms). The applications environments of interest were those having steady-state broadband noise. A probability monitoring task from the Criterion Task Set was used as an operator loading task to impose additional attentional demands during the signal detection and response task. The psychophysical methods of free-modulus magnitude estimation and paired comparison were used to measure subjective perceived urgency. An objective measure of response time to the signal was also obtained. Multivariate statistical analyses indicated that response time decreased significantly as perceived urgency increased. Perceived urgency of the signal increased and response time decreased as pulse level increased. Sequential signals took longer to detect and were rated as less urgent than the other two signal types. Shorter inter-pulse interval was associated with greater perceived signal urgency.     

Learning auditory warnings: The effects of sound type, verbal labelling and imagery on the identification of alarm sounds

This study looks at the effect of sound type, training cue and cue source on people's ability to learn and retain a set of twelve warning-type sounds. Each participant was required to learn and retain each of the twelve sounds using one of four learning methods. One group was required to learn the sounds according to verbal labels given to them by the experimenter; a second group generated their own verbal labels; a third group was required to learn the sounds using graphic images (the waveform of the sound) given to them by the experimenter; and the fourth group was allowed to generate their own graphic images. Three classes of sound were tested: real, environmental sounds, semi-abstract monitoring-type sounds, and abstract sounds already used as hospital warnings. The results show that the verbal labels worked better than graphic images, but only when these were given by the experimenter. When participants generated their own cues, performance was equally good in both graphic and verbal conditions. Across the sounds, it was found that real sounds were easier to learn than the other two groups, although this effect again disappeared when participants were allowed to generate their own cues. The implications for auditory warning design and training are discussed within a broader theoretical and practical context.

Relevance to industry

This paper shows that the difficulty with which alarm-type sounds can be learned depends not only on the type of sounds to be learned, but the way in which they are learnt. Apparent advantages to using environmental sounds and verbal labels disappears as soon as participants are allowed to generate their own learning aids.     

Preliminary study of behavioral and safety effects of driver dependence on a warning system in a driving simulator

Warning systems are being developed to improve traffic safety using visual, auditory, and/or tactile displays by informing drivers of the existence of a threat in the roadway. Behavioral and safety effects of driver dependence on such a warning system, especially when the warning system is unreliable, were investigated in a driving-simulator study. Warning-system accuracy was defined in terms of miss rate (MR) and positive predictive value (PPV) (PPV is the fraction of warnings that were correct detections). First, driver behavior and performance were measured across four warning-system accuracy conditions. Second, the authors estimated the probability of collision in each accuracy condition to measure the overall system effectiveness in terms of safety benefit. Combining these results, a method was proposed to evaluate the degree of driver dependence on a warning system and its effect on safety. One major result of the experiment was that the mean driving speed decreased as the missed detection rate increased, demonstrating a decrease in driver's reliance on warnings when the system was less effective in detecting threats. Second, both the acceleration-pedal and brake-pedal reaction times increased as the PPV of the warning system decreased, demonstrating a decrease in driver compliance with warnings when the system became more prone to false alarms. A key implication of the work is that performance is not necessarily directly correlated to warning-system quality or trends in subjective ratings, highlighting the importance of objective evaluation. Practical applications of the work include design and analysis of in-vehicle warning systems.     

Evidence for global feature superiority in menu selection by icons

A search-and-select paradigm was adopted to investigate which visual characteristics of icons are relevant for menu selection. Two icon sets (abstract icons, representational icons) were compared to a word command set. For abstract icons, global features were used in order lo maximize their visual distinctiveness. For representational icons local features were used in order to ensure a high degree of representativeness and a small “articulatory distance”. Results revealed that abstract icons were searched and selected much faster than both word commands and representational icons. In addition, response time functions indicated that abstract icons can be searched in parallel (no effect of menu size) whereas word commands and representational icons have to be searched sequentially. Error rates were small in all conditions, and there was no indication of a speed-accuracy tradeoff. Thus, when icons are used in menu selection, visual distinctiveness (due to global features) seems to override respresentativeness (due to local features).