Mapping candidate within-vehicle auditory displays to their referents

OBJECTIVE: This study investigated speech, environmental sounds (naturally occurring sounds with arbitrary meanings), auditory icons (natural or synthetic sounds with specific meanings), and abstract synthetic warnings as candidates for within-vehicle interfaces. BACKGROUND: Auditory displays and warnings must satisfy certain criteria, such as being appropriately urgent and commanding appropriately fast response times. However, a semiotic analysis suggests that displays, as signals interpreted by users, should also be mapped successfully onto their referents. METHOD: Response times and accuracy were recorded in a computer task of identifying learned mappings of candidate displays to a range of referent driving events (such as "headway closing"); perceived urgency and pleasantness were assessed separately. RESULTS: Speech and auditory icons produced near-ceiling performance in response times and identification accuracy. Abstract sounds produced notably slower response times and less accuracy. Environmental sounds showed an intermediate pattern of performance for accuracy, but the response times were similar to those of the abstract sounds. Speech utterances were similarly and consistently rated as pleasant but also of intermediate perceived urgency. The three other sound types showed a consistent mapping of their perceived urgency to the situational urgency of their referents; for these sounds, perceived urgency and pleasantness were negatively correlated. CONCLUSION: The results point to the importance of considering the role of signal-referent relationships in designing auditory displays. APPLICATION: The results have applicability for auditory displays in the vehicle interface, whereas the theoretical framework is of value in auditory display design in a broader context.     

Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an invehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying ‘ahead’. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application.

In the context of emergency warnings, auditory icons, which convey information about system events by analogy with everyday events, have the potential to be understood more quickly and easily than abstract sounds. To test this proposal, an experiment was carried out to evaluate the use of auditory icons for an in-vehicle collision avoidance application. Two icons, the sounds of a car horn and of skidding tyres, were compared with two conventional warnings, a simple tone and a voice saying 'ahead'. Participants sat in an experimental vehicle with a road scene projected ahead, and they were required to brake in response to on-screen collision situations and their accompanying warning sounds. The auditory icons produced significantly faster reaction times than the conventional warnings, but suffered from more inappropriate responses, where drivers reacted with a brake press to a non-collision situation. The findings are explained relative to the perceived urgency and inherent meaning of each sound. It is argued that optimal warnings could be achieved by adjusting certain sound attributes of auditory icons, as part of a structured, user-centred design and evaluation procedure.     

Semantic congruency of auditory warnings

Abstract

The aim of this study was to explore operator experience and performance for semantically congruent and incongruent auditory icons and abstract alarm sounds. It was expected that performance advantages for congruent sounds would be present initially but would reduce over time for both alarm types. Twenty-four participants (12M/12F) were placed into auditory icon or abstract alarm groupings. For each group both congruent and incongruent alarms were used to represent different driving task scenarios. Once sounded, participants were required to respond to each alarm by selecting a corresponding driving scenario. User performance for all sound types improved over time, however even with experience a decrement in speed of response remained for the incongruent iconic sounds and in accuracy of performance for the abstract warning sounds when compared to the congruent auditory icons. Semantic congruency was found to be of more importance for auditory icons than for abstract sounds.

Practitioner Summary: Alarms are used in many operating systems as emergency, alerting, or continuous monitoring signals for instance. This study found that the type and representativeness of an auditory warning will influence operator performance over time. Semantically congruent iconic sounds produced performance advantages over both incongruent iconic sounds and abstract warnings.     

Cough once for danger: icons versus abstract warnings as informative alerts in civil aviation

OBJECTIVE: An experiment investigated the efficacy of auditory icons as warning signals in an aviation context. BACKGROUND: Iconic signals, such as a cough to signal dangerous levels of carbon monoxide, convey information about the nature of an incident and alert the operator that there is a problem, whereas signals that are arbitrarily associated with a critical incident provide relatively less information. Warning recognition speed and accuracy are likely to be influenced by modality of presentation (visual, auditory, auditory + visual) and task demand (low, high). METHODS: The 172 participants completed a computer-based training session and test task that involved responding to abstract or iconic auditory (1 s), visual, or auditory + visual warnings associated with seven critical incidents while performing low- and high-demand concurrent tasks. RESULTS: Significantly fewer training trials were required to learn iconic warnings than abstract warnings. An advantage for iconic warnings persisted into the test phase, evident most consistently as greater warning recognition accuracy. The effect was observed in both high- and low-demand conditions. Auditory abstract warnings, in particular, elicited slow reaction times and poor accuracy. CONCLUSION: Associations between a small number of meaningful environmental sounds and critical incidents can be learned with ease relative to more abstract associations, although training is required and response times are relatively slow. APPLICATION: Sets of distinctive auditory iconic warnings can be designed to alert and inform pilots about non-time-pressured events. Potential applications of language-neutral icons as informative warnings include civil, commercial, and defense aircraft.     

Impact of spatial auditory feedback on the efficiency of iconic human–computer interfaces under conditions of visual impairment

This paper investigates the addition of spatial auditory feedback as a tool to assist people with visual impairments in the use of computers, specifically in tasks involving iconic visual search. In this augmented interface, unique sounds were mapped to visual icons on the screen. As the screen cursor traversed the screen, the user heard sounds of nearby icons, spatially, according to the relative position of each icon with respect to the screen cursor. A software prototype of the design was developed to evaluate the performance of users in the search of icons within the proposed interface. Experiments were conducted with simulated visual impairments on volunteer participants to evaluate if the addition of spatial auditory feedback makes the interface more accessible to users with impaired vision. Results demonstrated that spatialization of icon sounds provides additional remote navigational information to users, enabling new strategies for task completion. Directions for future research are discussed and prioritized.     

A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving

OBJECTIVE: This study examined the effectiveness of rear-end collision warnings presented in different sensory modalities as a function of warning timing in a driving simulator. BACKGROUND: The proliferation of in-vehicle information and entertainment systems threatens driver attention and may increase the risk of rear-end collisions. Collision warning systems have been shown to improve inattentive and/or distracted driver response time (RT) in rear-end collision situations. However, most previous rear-end collision warning research has not directly compared auditory, visual, and tactile warnings. METHOD: Sixteen participants in a fixed-base driving simulator experienced four warning conditions: no warning, visual, auditory, and tactile. The warnings activated when the time-to-collision (TTC) reached a critical threshold of 3.0 or 5.0 s. Driver RT was captured from a warning below critical threshold to brake initiation. RESULTS: Drivers with a tactile warning had the shortest mean RT. Drivers with a tactile warning had significantly shorter RT than drivers without a warning and had a significant advantage over drivers with visual warnings. CONCLUSION: Tactile warnings show promise as effective rear-end collision warnings. APPLICATION: The results of this study can be applied to the future design and evaluation of automotive warnings designed to reduce rear-end collisions.     

Effects of single versus multiple warnings on driver performance

OBJECTIVE: To explore how a single master alarm system affects drivers' responses when compared with multiple, distinct warnings. BACKGROUND: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event type on driver performance. METHOD: Using a 2 x 2 x 4 mixed factorial design, we investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers' performance across four collision event types (frontal collision warnings, left and right lane departure warnings, and warnings for a fast-approaching following vehicle). RESULTS: Participants' reaction times and accuracy rates were significantly affected by the type of collision event and alarm reliability. The use of individual alarms, rather than a single master alarm, did not significantly affect driving performance in terms of reaction time or response accuracy. CONCLUSION: Even though a master alarm is a relatively uninformative warning, it produced statistically no different reaction times or accuracy results when compared with information-rich auditory icons, some of which were spatially located. In addition, unreliable alarms negatively impacted driver performance, regardless of event type or alarm scheme. APPLICATION: These results have important implications for the development and implementation of multiple driver warning systems.     

Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual search

In joint tasks, adjusting to the actions of others is critical for success. For joint visual search tasks, research has shown that when search partners visually receive information about each other’s gaze, they use this information to adjust to each other’s actions, resulting in faster search performance. The present study used a visual, a tactile and an auditory display, respectively, to provide search partners with information about each other’s gaze. Results showed that search partners performed faster when the gaze information was received via a tactile or auditory display in comparison to receiving it via a visual display or receiving no gaze information. Findings demonstrate the effectiveness of tactile and auditory displays for receiving task-relevant information in joint tasks and are applicable to circumstances in which little or no visual information is available or the visual modality is already taxed with a demanding task such as air-traffic control. Practitioner Summary: The present study demonstrates that tactile and auditory displays are effective for receiving information about actions of others in joint tasks. Findings are either applicable to circumstances in which little or no visual information is available or when the visual modality is already taxed with a demanding task.     

Intelligibility of speech in a virtual 3-D environment

In a simulated air traffic control task, improvement in the detection of auditory warnings when using virtual 3-D audio depended on the spatial configuration of the sounds. Performance improved substantially when two of four sources were placed to the left and the remaining two were placed to the right of the participant. Surprisingly, little or no benefits were observed for configurations involving the elevation or transverse (front/back) dimensions of virtual space, suggesting that position on the interaural (left/right) axis is the crucial factor to consider in auditory display design. The relative importance of interaural spacing effects was corroborated in a second, free-field (real space) experiment. Two additional experiments showed that (a) positioning signals to the side of the listener is superior to placing them in front even when two sounds are presented in the same location, and (b) the optimal distance on the interaural axis varies with the amplitude of the sounds. These results are well predicted by the behavior of an ideal observer under the different display conditions. This suggests that guidelines for auditory display design that allow for effective perception of speech information can be developed from an analysis of the physical sound patterns.     

Acoustic User Interface (AUI) for the auditory displays

When designing a multimodal human–machine interface, the human function allocation must be considered. To reduce the load on the visual sense, the other sensory channels must be utilized. Here, the authors put forward a HMI which provides unique concepts called acoustic user interface (AUI).With an AUI, information can be removed from the visual channel and presented through the auditory channel. The head-related transfer function is used to create the impression of sounds arriving from any direction, either inside or outside the visual field of view. As well as reducing the loading on the visual sense, aural ‘images’ can be ‘projected’ coincident with the location of images displayed on the visual displays, in order to co-ordinate and communicate more effectively multimedia information.     

Effects of in-vehicle warning information displays with or without spatial compatibility on driving behaviors and response performance

A driving simulator study was conducted to evaluate the effects of five in-vehicle warning information displays upon drivers' emergent response and decision performance. These displays include visual display, auditory displays with and without spatial compatibility, hybrid displays in both visual and auditory format with and without spatial compatibility. Thirty volunteer drivers were recruited to perform various tasks that involved driving, stimulus-response, divided attention and stress rating. Results show that for displays of single-modality, drivers benefited more when coping with visual display of warning information than auditory display with or without spatial compatibility. However, auditory display with spatial compatibility significantly improved drivers' performance in reacting to the divided attention task and making accurate S-R task decision. Drivers' best performance results were obtained for hybrid display with spatial compatibility. Hybrid displays enabled drivers to respond the fastest and achieve the best accuracy in both S-R and divided attention tasks.     

Comparative study of the effects of auditory, visual and multimodality displays on drivers' performance in advanced traveller information systems

A simulator study was conducted to compare 16 younger (mean age 22 years) and 16 older (mean age 68 years) drivers' ratings of workload (time, visual, psychological stress) and performance of navigation and button-pushing (identification of vehicle or road hazards) tasks under both high- and low-load driving conditions when simple or complex advanced traveller information (ATI) was presented visually only, aurally only or by multimodality (visual and auditory) display. For all participants, both the auditory and multimodality displays produced better performance in terms of response times, total number of correct turns and subjective workload ratings than those of using the visual-only display. Participants using the multimodality display also made the fewest errors related to push-button and navigation tasks, and controlled their vehicles properly. The visual display led to less safe driving, apparently because it imposed higher demands on the drivers' attention. An age effect was found in the present study, with younger drivers performing better and reporting less stress than older drivers. Notably, however, use of the multimodality display significantly improved the older drivers' performance in the button-pushing task.     

EcoSonic: Auditory peripheral monitoring of fuel consumption for fuel-efficient driving

In this paper, we propose to make use of an auditory fuel efficiency display as means to support car drivers in adopting an energy-efficient driving style. We report on the development of the EcoSonic system as a platform for evaluating such displays and present five design approaches to guide their realization. In a study with 30 participants, we evaluated two prototype auditory displays against the baseline of visual-only eco-driving feedback in a within-subject study. Our selected designs are described in full detail. Key findings include a significantly reduced fuel consumption as well as lower engine speeds compared to the visual display. Furthermore, questionnaire analysis results confirm that the auditory conditions are less obtrusive and also seem to allow a more subconscious processing and comprehension of the provided information. Finally, we have found that the affectiveness of the display design seems to have a positive impact on its perceived helpfulness and the ability to absorb its information subconsciously.     

Learning and retention of associations between auditory icons and denotative referents: implications for the design of auditory warnings

OBJECTIVE: This study examined the way in which the type and preexisting strength of association between an auditory icon and a warning event affects the ease with which the icon/event pairing can be learned and retained. BACKGROUND: To be effective, an auditory warning must be audible, identifiable, interpretable, and heeded. Warnings consisting of familiar environmental sounds, or auditory icons, have potential to facilitate identification and interpretation. The ease with which pairings between auditory icons and warning events can be learned and retained is likely to depend on the type and strength of the preexisting icon/event association. METHOD: Sixty-three participants each learned eight auditory-icon/denotative-referent pairings and attempted to recall them 4 weeks later. Three icon/denotative-referent association types (direct, related, and unrelated) were employed. Participants rated the strength of the association for each pairing on a 7-point scale. RESULTS: The number of errors made while learning pairings was greater for unrelated than for either related or direct associations, whereas the number of errors made while attempting to recall pairings 4 weeks later was greater for unrelated than for related associations and for related than for direct associations. Irrespective of association type, both learning and retention performance remained at very high levels, provided the strength of the association was rated greater than 5. CONCLUSION: This suggests that strong preexisting associations are used to facilitate learning and retention of icon/denotative-referent pairings. APPLICATION: The practical implication of this study is that auditory icons having either direct or strong, indirect associations with warning events should be preferred.     

An exploration of decision support for drivers,inside and outside the vehicle

The purpose of this study was to investigate the feasibility and future promise of decision support systems (DSS A driving simulator study compared three DSSs: a dynamic road sign, displays on the side mirrors, and a vibrating seat (haptic).) inside the vehicle relative to a particular, well‐tested dynamic road sign external to the vehicle. Independent variables included display placement and modality (visual vs. haptic). Dependent variables included measurements of safe driving behavior and usability. A follow‐on focus group study was conducted to gain further insights into the results and improvement ideas. The vibrating seat yielded significantly better safety margins than the dynamic road sign. Haptic DSSs inside the vehicle may have more safety benefits than visual DSSs inside or outside the vehicle. Further investigation is needed to determine what properties of the haptic interface – reduced interference with visual driving tasks, message salience, simplicity, or some combination – are responsible for the advantage. © 2012 Wiley Periodicals, Inc.     

Age differences in driver visual behavior and vehicle control when driving with in-vehicle and on-road deliveries of service logo signs

With the advances in vehicle technologies, more information is communicated in real-time to the driver via an in-vehicle interface. In-vehicle messaging may deliver safety-related information such as warnings as well as non-safety-related information such as an upcoming lodging place. While much research has focused on the design of messaging safety-related information, little is known about the best practice in in-vehicle messaging of non-safety-related information. This study investigated the effects of information source and load on driver signage logo identification, glance behavior, and vehicle control among younger, middle-aged and older drivers. The logos were presented on: (1) an on-road sign panel, (2) an in-vehicle display, or (3) a combination of both, with half of the drives showing logo only, and the other half of the drives showing logo plus additional text. The general findings support the use of in-vehicle displays, especially when it is presented simultaneously with on-road signs. In-vehicle displays did not lead to a higher workload or more visual distraction, and simultaneous presentations resulted in slightly better speed control. The findings also showed minimal negative impacts on logo identification from increased information load. Older drivers performed less well in signage identification and vehicle control, and they made longer glances to logo information suggesting design considerations should be made to accommodate specific driver characteristics.     

Auditory Displays in Human–Machine Interfaces of Mobile Robots for Non-Speech Communication with Humans

Auditory displays are developed and investigated for mobile service robots in a human–machine environment. The service robot domain was chosen as an example for future use of auditory displays within multimedia process supervision and control applications in industrial, transportation, and medical systems. The design of directional sounds and of additional sounds for robot states as well as the design of more complicated robot sound tracks are explained. Basic musical elements and robot-movement sounds are combined. Experimental studies on the auditory perception of directional sounds as well as of sound tracks for the predictive display of intended robot trajectories in a simulated supermarket scenario are described.     

A user-centred approach to the design and evaluation of auditory warning signals: 1. Methodology

This paper presents a method for the design and evaluation of auditory warning signals modelled on an existing internationally standardized method for evaluating public information systems (ISO/DIS 7001: 1979). The procedure is essentially user-centred, capitalizing upon users' associations between sounds and their meanings. The procedure is presented in a step-by-step manner, from the initial identification of referents for which warnings might be required, through the generation of ideas for warning sounds, an appropriateness ranking test, a learning and confusion test, an urgency mapping test, a recognition test and an operational test. Practical issues are discussed with respect to each of the stages, and suggestions are made as to courses of action that might be taken if problems are encountered. Theory behind the relationship between sound and meaning is discussed with reference to the practical issues addressed.     

An overview of auditory display to assist comprehension of molecular information

This paper presents an overview of auditory display (the use of non-speech sounds to convey information) applied to the study of molecular properties in human–computer interfaces, particularly in virtual environments. Chemistry researchers and students have difficulty in analysing and comprehending molecular structure and bonding and other biomolecular characteristics. Research reports that non-speech sounds have been useful in identifying trends in gene sequences and molecular characteristics, which when used in virtual environments, can facilitate comprehension of complex relationships that are difficult to perceive through visualisation alone.