The effect of visual and interaction fidelity on spatial cognition in immersive virtual environments

Accuracy of memory performance per se is an imperfect reflection of the cognitive activity (awareness states) that underlies performance in memory tasks. The aim of this research is to investigate the effect of varied visual and interaction fidelity of immersive virtual environments on memory awareness states. A between groups experiment was carried out to explore the effect of rendering quality on location-based recognition memory for objects and associated states of awareness. The experimental space, consisting of two interconnected rooms, was rendered either flat-shaded or using radiosity rendering. The computer graphics simulations were displayed on a stereo head-tracked head mounted display. Participants completed a recognition memory task after exposure to the experimental space and reported one of four states of awareness following object recognition. These reflected the level of visual mental imagery involved during retrieval, the familiarity of the recollection, and also included guesses. Experimental results revealed variations in the distribution of participants' awareness states across conditions while memory performance failed to reveal any. Interestingly, results revealed a higher proportion of recollections associated with mental imagery in the flat-shaded condition. These findings comply with similar effects revealed in two earlier studies summarized here, which demonstrated that the less "naturalistic" interaction interface or interface of low interaction fidelity provoked a higher proportion of recognitions based on visual mental images.     

Brain activity and presence: a preliminary study in different immersive conditions using transcranial Doppler monitoring

Transcranial Doppler (TCD) sonography is a brain activity measurement technique that monitors the hemodynamic characteristics of the major cerebral arteries in normal and pathological conditions. As it is not invasive, it can be easily used in combination with virtual environments (VE). In the present study, TCD has been used to analyze brain activity variations in different presence conditions during the exposure to a VE. Forty-two subjects have taken part in the experience grouped in two different visualization conditions: a CAVE-like and a single screen projection configuration. In each session, two different navigation conditions were used: a free navigation (controlled by the subject) and an automatic navigation (controlled by the system). Results show that these immersion and navigation modifications in the VE generate changes in brain activity that can be detected using TCD techniques. Several factors, one of them being presence, could be having an influence on this behavior.     

Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality system, not only to assess the student's behavior and progress, but also is able to adapt itself to the student's specific needs. Additionally, the immersive reality system is equipped with sensors that can determine certain behaviors of the students. This paper determines the possible inclusion of immersive virtual reality as a support tool and learning strategy in these particular students' intervention. With this objective two task protocols have been defined with which the behavior and interaction situations performed by participant students are recorded. The conclusions from this study talks in favor of the inclusion of these virtual immersive environments as a support tool in the educational intervention of Asperger syndrome students as their social competences and executive functions have improved.     

Poster3D: a system for three-dimensional authoring and presentation in immersive virtual environments

We developed a simple system for three-dimensional (3D) authoring and presentation in immersive virtual reality environments. Our system enables a user to create and edit content both through direct manipulation using hand gestures in an immersive environment, and by using a 2D GUI on a PC desktop. The system also exploits various features of 3D information visualization to produce effective and interactive presentations. In a system trial, users found that the system enhanced their experience. The system helps users create 3D educational materials more easily and learn through the materials with greater interest and motivation. Noritaka Osawa received his B.S., M.S. and Ph.D. degrees in information science from the University of Tokyo in 1983, 1985, and 1988 respectively. After working for a software company and the University of Electro-communications, he has been at the National Institute of Multimedia Education since 1998. He has also been jointly appointed at the Graduate University for Advanced Studies since 2001. He now works for them as a Professor. His current research interests include human–computer interfaces using virtual reality technologies, information visualization, and system software. Sangtae Kim received B.A. degree in Visual Communication from YougIn University, Korea, in 1996, and M.A. and Ph.D. degrees in Design, from the University of Tsukuba, 2001, and 2004, respectively. He is currently an Assistant Professor of the institute of Art and Design, The University of Tsukuba, Ibaraki, Japan. His research interests include visual communication, dynamic information graphics, multimedia systems, 3DCG, and Web3D technology. He planned for the MDD (Multi-Dynamic Documentation) project in 2003. At present, he is involved in the development of Web3D contents (Interactive Anatomy system) that uses MDD technology.     

Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual search

In joint tasks, adjusting to the actions of others is critical for success. For joint visual search tasks, research has shown that when search partners visually receive information about each other’s gaze, they use this information to adjust to each other’s actions, resulting in faster search performance. The present study used a visual, a tactile and an auditory display, respectively, to provide search partners with information about each other’s gaze. Results showed that search partners performed faster when the gaze information was received via a tactile or auditory display in comparison to receiving it via a visual display or receiving no gaze information. Findings demonstrate the effectiveness of tactile and auditory displays for receiving task-relevant information in joint tasks and are applicable to circumstances in which little or no visual information is available or the visual modality is already taxed with a demanding task such as air-traffic control. Practitioner Summary: The present study demonstrates that tactile and auditory displays are effective for receiving information about actions of others in joint tasks. Findings are either applicable to circumstances in which little or no visual information is available or when the visual modality is already taxed with a demanding task.     

Afforded actions as a behavioral assessment of physical presence in virtual environments

A particular affordance was used as a potential candidate for behavioral assessment of physical presence in virtual environments. The subjects’ task was to walk through a virtual aperture of variable widths. In the case of presence, the subjects’ body orientation, while walking, was hypothesized to be adapted to the width of the aperture and to their own shoulder width. Results show that most subjects adapted their behavior to both their body architecture and the virtual width constraints. These subjects exhibited a behavioral transition from frontal walking to body rotation while walking through broad to narrow apertures. The same behavioral transition has already been documented in real environments (Warren and Whang in J Exp Psychol Human Percept Perform 13(3):371–383, 1987). This behavioral adjustment is thus assumed to be an objective indication of presence. Beyond these results, the present study suggests that every afforded action could be a potential tool for sensorimotor assessment of physical presence. Parts of this research have been first presented at the 11th Annual International Workshop of Presence, Padova, Italy, 16-18 October 2008.     

A comparison of tactile,auditory, and visual feedback in a pointing task using a mouse-type device

A mouse was modified to add tactile feedback via a solenoid-driven pin projecting through a hole in the left mouse button. An experiment is described using a target selection task under five different sensory feedback conditions (‘normal’, auditory, colour, tactile, and combined). No differences were found in overall response times, error rates, or bandwidths; however, significant differences were found in the final positioning times (from the cursor entering the target to selecting the target). For the latter, tactile feedback was the quickest, normal feedback was the slowest. An examination of the spatial distributions in responses showed a peaked, narrow distribution for the normal condition, and a flat, wide distribution for the tactile (and combined) conditions. It is argued that tactile feedback allows subjects to use a wider area of the target and to select targets more quickly once the cursor is inside the target. Design considerations for human-computer interfaces are discussed.     

Pairing computational and scaled physical models to determine permeability as a measure of cellular communication in micro- and nano-scale pericellular spaces

Cells, the living components of tissues, bathe in fluid. The pericellular fluid environment is a challenge to study due to the remoteness and complexity of its nanoscale fluid pathways. The degree to which the pericellular fluid environment modulates the transport of mechanical and molecular signals between cells and across tissues is unknown. As a consequence, experimental and computational studies have been limited and/or highly idealized. In this study we apply a fundamental fluid dynamics technique to measure pericellular permeability through scaled-up physical models obtained from high resolution microscopy. We assess permeability of physiologic tissue by tying together data from parallel experimental and computational models that account for specific structures of the flow cavities and cellular structures therein (cell body, cell process, pericellular matrix). A healthy cellular network devoid of cellular structure is shown to exhibit permeability on the order of 2.8 × 10⁻¹⁶ m²; inclusion of cellular structures reduces permeability to the order of 10⁻¹⁷ to 10⁻¹⁸ m². These permeability studies provide not only unprecedented quantitative experimental measures of the pericellular fluid environment but also provide a novel measure of “infrastructural integrity” that likely influences the efficiency of the cellular communication network across the tissue. Funding: National Institutes of Health, The Whitaker Foundation.     

Time windows-based team performance measures: a framework to measure team performance in dynamic environments

In this paper, we present a framework for implementing Team Performance Measures based on a temporal accuracy measure called the relative accuracy index (RAI) to evaluate and compare team performance in a command-and-control human-in-the-loop simulation. The framework allows researchers to collect and analyze team outcomes in an unbiased objective manner based on a temporal performance measure known as a Time Window. Our framework provides experimenters and subject matter experts the necessary tools to evaluate performance in terms of task demands. We also provide a sample analysis of individual and team performance using the RAI.     

Mobile devices,interaction, and distraction: a qualitative exploration of absent presence

Mobile devices have become an ever-present and indispensible part of our lives. Despite this pronounced ubiquity, few studies have addressed the influence of mobile devices on everyday social interaction. The purpose of this article is to examine such social ramifications. Specifically, this study offers an in-depth qualitative exploration of so-called absent presence, the state where a partner is physically present, yet absorbed by a technologically mediated world of elsewhere. Building on interviews about technology use with Danish students, findings suggest that the microsocial dynamics at stake in such impaired social interaction include delayed responses, mechanical intonation, a motionless body, and a lack of eye contact. Appropriating developmental psychologist Daniel Stern’s terminology, it is suggested that this mismatch between the vitality of a person and his or her absently present conversational partner amounts to a kind of unintentional misattunement which disrupts the smooth flow of ordinary interaction and signals indifference to what is being said. On this basis, absent presence is distinguished from related concepts of daydreaming and mind wandering. Theoretical and practical implications are discussed.     

Occlusion as a measure for visual workload: an overview of TNO occlusion research in car driving

A process-oriented approach by systematically studying driver performance, distraction, and workload is the way to go for assessing safety effects of new telematics applications in vehicles. These systems may strive for drivers' attention and possibly lead to distraction from the primary task. Visual occlusion techniques appear to be an effective means of studying drivers' visual information processing performance. Studies to illustrate this include identifying the minimum visual information drivers need for driving (asking for visual information) and the evaluation of driver support systems such as heading control and adaptive cruise control with respect to visual workload. In other applications the occlusion technique is used to exclude 'visual array' information from the optic flow field. The temporary exclusion of part of the visual field of view was applied in evaluating the effects of different types of driver's side rearview mirrors.     

Distraction 'on the buses': a novel framework of ergonomics methods for identifying sources and effects of bus driver distraction

Driver distraction represents a significant problem in the public transport sector. Various methods exist for investigating distraction; however, the majority are difficult to apply within the context of naturalistic bus driving. This article investigates the nature of bus driver distraction at a major Australian public transport company, including the sources of distraction present, and their effects on driver performance, through the application of a novel framework of ergonomics methods. The framework represents a novel approach for assessing distraction in a real world context. The findings suggest that there are a number of sources of distraction that could potentially distract bus drivers while driving, including those that derive from the driving task itself, and those that derive from the additional requirements associated with bus operation, such as passenger and ticketing-related distractions. A taxonomy of the sources of bus driver distraction identified is presented, along with a discussion of proposed countermeasures designed to remove the sources identified or mitigate their effects on driver performance.     

Effectiveness of auditory and tactile crossmodal cues in a dual-task visual and auditory scenario

Abstract

In this study, we examined how spatially informative auditory and tactile cues affected participants’ performance on a visual search task while they simultaneously performed a secondary auditory task. Visual search task performance was assessed via reaction time and accuracy. Tactile and auditory cues provided the approximate location of the visual target within the search display. The inclusion of tactile and auditory cues improved performance in comparison to the no-cue baseline conditions. In comparison to the no-cue conditions, both tactile and auditory cues resulted in faster response times in the visual search only (single task) and visual–auditory (dual-task) conditions. However, the effectiveness of auditory and tactile cueing for visual task accuracy was shown to be dependent on task-type condition. Crossmodal cueing remains a viable strategy for improving task performance without increasing attentional load within a singular sensory modality.

Practitioner Summary: Crossmodal cueing with dual-task performance has not been widely explored, yet has practical applications. We examined the effects of auditory and tactile crossmodal cues on visual search performance, with and without a secondary auditory task. Tactile cues aided visual search accuracy when also engaged in a secondary auditory task, whereas auditory cues did not.     

Improved visual SLAM: a novel approach to mapping and localization using visual landmarks in consecutive frames

Pathfinding is becoming more and more common in autonomous vehicle navigation, robot localization, and other computer vision applications. In this paper, a novel approach to mapping and localization is presented that extracts visual landmarks from a robot dataset acquired by a Kinect sensor. The visual landmarks are detected and recognized using the improved scale-invariant feature transform (I-SIFT) method. The methodology is based on detecting stable and invariant landmarks in consecutive (red-green-blue depth) RGB-D frames of the robot dataset. These landmarks are then used to determine the robot path, and a map is constructed by using the visual landmarks. A number of experiments were performed on various datasets in an indoor environment. The proposed method performs efficient landmark detection in various environments, which includes changes in rotation and illumination. The experimental results show that the proposed method can solve the simultaneous localization and mapping (SLAM) problem using stable visual landmarks, but with less computation time.     

Wavelet-scalogram based study of non-periodicity in speech signals as a complementary measure of chaotic content

In recent studies, the chaotic behavior of a signal is confirmed using scalogram analysis of continuous-wavelet transform. Chaotic component of a speech signal can be verified through scalogram analysis, since, it investigates the periodicity content of a signal. The periodicity analysis helps in proving that a signal is not periodic, which is an essential condition on chaotic activity. In this work, a scale-index based on scalogram-analysis is calculated for a set of recordings of Arabic vowels. Also, Largest-Lyapunov Exponents (LLE) are computed for these recordings. The obtained measures are, then, compared. The comparison proves the efficacy of scale index for confirming chaotic behavior even for highly-periodic waveforms which is the case in speech vowels. Additionally, it is noted that both LLE and scale-index exhibit classification ability for Arabic vowels.     

Physical and social presence in collaborative virtual environments: Exploring age and gender differences with respect to empathy

Collaborative virtual environments (CVEs) hold the immense potential of enhancing social inclusion and social support not only in younger but especially also in older people. However, there is still considerably little knowledge about the experiences of elderly when using CVEs. Additionally, there is reason to assume that men and women regardless of their age might also differ in their perceptions of CVEs, with this difference very likely being mediated by empathic abilities. Consequently, the main objective of the current study was to evaluate gender specific experiences of social and physical presence in a group of older (N = 62) and younger adults (N = 62) with respect to possible mediating influences of empathy. Results indicate no differences in physical and social presence between the two age groups, yet they support past findings that men experience more spatial presence, involvement and a higher sense of being there than women. Also, the empathy scale Fantasy considerably mediated gender differences in spatial presence, thus strengthening the theoretical assumption of a relationship between presence and empathy. Implications and future directions of these findings are discussed in detail.     

Challenges and opportunities to improve the accessibility of YouTube for people with visual impairments as content creators

Universal Access in the Information Society - This position paper proposes technology opportunities for supporting people with visual impairments (PVI) as content creators, rather than just content...     

From the body to the tools and back: A general framework for presence in mediated interactions

Different neuropsychological studies clearly show that the perception of our body and its surrounding space is not a given fact but it is influenced by the outcome of our actions (both direct and mediated by the use of tools). In this view, a possible starting point for a better understanding of Presence in computer-mediated interactions is the study of mediated action and its effects on our spatial experience.Following a cognitive perspective, the presented framework describes Presence as an intuitive feeling which is the outcome of an experience-based metacognitive judgment that controls our action. This process monitors pre-reflexively our activity by using an embodied intuitive simulation of the intended action developed through practice (implicit learning).When actions are implemented using one or more tools, it is possible to distinguish between two different types of mediated action: first-order (I use the body to control a proximal artifact, e.g. a tennis player striking the ball with the racquet) or second-order (I use the body to control a proximal artifact that controls a different distal one, e.g. a cranemen using a lever to move a mechanical boom to lift materials). These two mediated actions, when produced intuitively, have different effects on our experience of body and space: a successfully learned first-order mediated action produces incorporation – the proximal tool extends the peripersonal space of the subject – while a successfully learned second-order mediated action produces also incarnation – a second peripersonal space centered on the distal tool.     

Broadening the understanding of social presence: Implications and contributions to the mediated communication and online education

Social presence is a concept found to facilitate effective learning experiences in online education. However, there is limited understanding of the theoretical notion in the literature. In an attempt to broaden the understanding of social presence, the current investigation examined two issues: a) distinction between social presence and interactivity, and b) multidimensional nature of social presence and its predictive validity for online learning experiences. Data were collected from 210 undergraduate students using an online survey. Findings indicate that social presence and interactivity are related but distinct constructs. Both psychological involvement and co-presence appear to be under the concept of social presence. The data indicate strong predictive validity of social presence for online learning experiences when it is measured with both dimensions of psychological involvement and co-presence.