Interactive spatial sonification for non-visual exploration of virtual maps |
| |
| Affiliation: | 1. Department of Information Engineering, University of Padova, Via Gradenigo 6/B, 35131 Padova, Italy;2. Department of Robotics, Brain and Cognitive Sciences, Fondazione Istituto Italiano di Tecnologia, Via Morego 30, 16163 Genova, Italy;1. Division of Hand Surgery, Rothman Orthopaedic Institute, Philadelphia, PA;2. Division of Sports Medicine, Rothman Orthopaedic Institute, Philadelphia, PA;1. Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (Next Lab), Università Campus Bio-Medico di Roma, Rome, Italy;2. Centre for Cognitive Neuroscience, Department of Life Sciences, College of Health, Medicine and Life Sciences, Brunel University London, Uxbridge, UK;3. Department of Life Sciences, College of Health, Medicine and Life Sciences, Brunel University London, Uxbridge, UK;4. School of Engineering, College of Engineering and Science, James Watt Building (south) University of Glasgow, Glasgow G12 8QQ, UK;5. Centre for Cognitive Neuroscience, Department of Clinical Sciences, College of Health, Medicine and Life Sciences, Brunel University London, Uxbridge, UK;1. Airbus D&S, Elancourt and LIMSI-CNRS, Orsay, France;2. LMA-CNRS-UPR 7051, Aix-Marseille Univ., Centrale Marseille, France;3. LIMSI-CNRS, Orsay, France;1. Universita degli Studi di Milano, Department of Computer Science, Via Comelico 39, 20135 Milano, Italy;2. EveryWare Technologies, Via Comelico 39, 20135 Milano, Italy;3. Dyson School of Design Engineering, Imperial College London, South Kensington, SW7 2AZ, London, UK;1. Department of Movement and Training Science, Institute for Human Movement Science, University of Hamburg, Turmweg 2, 20148 Hamburg, Germany;2. The School of Mathematical and Computational Sciences, University of Prince Edward Island, 550 University Ave, Charlottetown PE C1A 4P3, Canada;3. BeSB Sound & Engineering GmbH, Undinestr. 43, 12203 Berlin, Germany |
| |
| Abstract: | This paper presents a multimodal interactive system for non-visual (auditory-haptic) exploration of virtual maps. The system is able to display haptically the height profile of a map, through a tactile mouse. Moreover, spatial auditory information is provided in the form of virtual anchor sounds located in specific points of the map, and delivered through headphones using customized Head-Related Transfer Functions (HRTFs). The validity of the proposed approach is investigated through two experiments on non-visual exploration of virtual maps. The first experiment has a preliminary nature and is aimed at assessing the effectiveness and the complementarity of auditory and haptic information in a goal reaching task. The second experiment investigates the potential of the system in providing subjects with spatial knowledge: specifically in helping with the construction of a cognitive map depicting simple geometrical objects. Results from both experiments show that the proposed concept, design, and implementation allow to effectively exploit the complementary natures of the “proximal” haptic modality and the “distal” auditory modality. Implications for orientation & mobility (O&M) protocols for visually impaired subjects are discussed. |
| |
| Keywords: | Multimodal interaction 3D audio Binaural sound Virtual maps Non-visual navigation Haptics Visual impairment Multisensory integration Haptic mouse |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
