Two-handed assembly with immersive task planning in virtual reality

Assembly modelling is the process of capturing entities and activity information related to assembling and assembly. Currently, most CAD systems have been developed to ease the design of individual components, but are limited in their support for assembly designs and planning capability, which are crucial for reducing the cost and processing time in complex design, constraint analysis and assembly task planning. This paper presents a framework of a two-handed virtual assembly (VA) planner for assembly tasks, which coordinates two hands jointly for feature-based manipulation, assembly analysis and constraint-based task planning. Feature-based manipulation highlights the important assembling features (e.g. dynamic reference frames, moving arrow, mating features) to guide users for the ease of assembly and in an efficient and fluid manner. The users can freely navigate and move the mating pair along the collision-free path. The free motion of two-handed input in assembly is further restricted to the allowable motion guided by the constraints recognised on-line. The allowable motion in assembly is planned by the logic steps derived from the analysis of constraints and their translation in the progress of assembly. No preprocessing or predefined assembly sequence is necessary since the planning is produced in real-time upon the two-handed interactions. Mating features and constraints in databases are automatically updated after each assembly to simplify the planning process. The two-handed task planner has been developed and experimented for several assembly examples including a drill (12-parts) and a robot (17-parts). The system can be generally applied for the interactive task planning of assembly-type applications.     

ERT-VR: an immersive virtual reality system for emergency rescue training

Virtual reality technology offers a cost effective means to train emergency rescuers, which is an urgent task on account of the increasing terrorist acctivities. An immersive virtual reality system called ERT-VR is introduced. In ERT-VR, the display system based on stereoscopic projectors is used to train the emergency rescue commanders. The members of the operational teams use head-mounted display as the display system. The 3D scenario creator is the most important unit in ERT-VR. Instructors assign a specific training scenario to the trainees by using the scenario creator. Trainees take on the role of the characters in the training scenario and control their actions and ultimately the scenario outcomes. All the actions are recorded into the database system and can be replayed anytime. The potential of each trainee is evaluated by an expert system.     

User experimentation: an evaluation of velocity control techniques in immersive virtual environments

While many of the existing velocity control techniques are well designed, the techniques are often application-specific, making it difficult to compare their effectiveness. In this paper, we evaluate five known velocity control techniques using the same experimental settings. We compare the techniques based on the assumption that a good travel technique should be easy to learn and easy to use, should cause the user to have few collisions with the VE, should allow the user to complete tasks faster, and should promote better recollection of the environment afterwards. In our experiments, we ask twenty users to use each velocity control technique to navigate through virtual corridors while performing information-gathering tasks. In all cases, the users use pointing to indicate the direction of travel. We then measure the users’ ability to recollect the information they see in the VE, as well as how much time they spend in the VE and how often they collide with the virtual walls. After each test, we use questionnaires to evaluate the ease of learning and ease of use of the velocity control technique, and the users’ sense of presence in the environment. Each of the travel techniques is then evaluated based on the users’ performances in the VE and the results of their questionnaires.     

The gain and pain in taking the pilot seat: learning dynamics in a non immersive virtual solar system

This study describes and analyzes the learning interactions of nine high-school students’ free exploration of a virtual solar system (VSS). The VSS is a non-immersive three dimensional virtual environment based on real NASA planetary images. The computer screen serves as a “spacecraft’s window” for the learner to “fly” between objects, to change the system’s frame of reference and its pace. A systematic analysis of participants’ real-time observable interactions together with what they said revealed that each of them created an unique learning pattern within at least five different dimensions: (1) the cognitive dimension, (2) the affective dimension, (3) the navigation dimension, (4) the interface dimension, and (5) the assistance seeking dimension. The construction of meaning emerged as a non-linear process, which includes transitions between and within these dimensions. Three different styles of learning interactions were identified, suggesting that individual differences might be enhanced due to the unique VSS’ features. Overall, the VSS served as an enriching and motivational learning experience. The design of additional navigation tools and content scaffolding might help participants’ in building a sustained deep scientific understanding.     

基于模型法应对企业信息系统工程中不确定性需求

在软件工程或具体的需求工程中,用户需求通常具有不确定性.这成为了企业信息化实践中的主要问题之一;在企业信息系统工程中这尤其是一关键问题.通过扩展模型的概念、分析企业领域中可用模型的情况,提出了一种基于模型来应对用户需求之不确定性的方法.描述了应用基于模型的方法确定企业信息系统需求的基本逻辑与主要活动过程,并给出了一个应用ARIS(集成信息系统结构)参考模型库解决需求问题的实例.研究表明,基于模型的方法可用于有效地应对企业信息系统工程中的不确定性需求.     

Instructional mechanisms in immersive virtual reality serious games: Earthquake emergency training for children

Children are vulnerable in earthquakes, but they are also essential to foster earthquake-resilient communities. It is critical to enhance the preparedness of children against earthquakes through effective education and training. Immersive virtual reality (IVR) and serious games (SGs) are innovative digital technologies that enable realistic and engaging training environments. However, little research has been made on the applications of IVR SGs for earthquake training targeting children. In order to fill this gap, this paper presents an IVR SG training system based on a problem-based gaming framework. Three instructional mechanisms within the training system, namely prior instruction, immediate feedback, and post-game assessment, were investigated to promote learning through effective reflection. A controlled experiment involving 125 secondary school students aged from 11 to 15 years old was undertaken, using leaflets as a traditional training approach for the control group and the IVR SG training system as the main intervention. Results revealed that the IVR SG training system with post-game assessment was the most effective way to train children, with greater knowledge acquisition and self-efficacy improvement observed. Possible improvements, such as increasing the time for reflection and differentiating the stimulation between positive and negative feedback, are suggested for further research.     

Exploring high-order three dimensional virtual elements: Bases and stabilizations

We present numerical tests of the virtual element method (VEM) tailored for the discretization of a three dimensional Poisson problem with high-order “polynomial” degree (up to $p=10$). Besides, we discuss possible reasons for which the method could return suboptimal/wrong error convergence curves. Among these motivations, we highlight ill-conditioning of the stiffness matrix and not particularly “clever” choices of the stabilizations. We propose variants of the definition of face/bulk degrees of freedom, as well as of stabilizations, which lead to methods that are much more robust in terms of numerical performances.     

VirSchool: The effect of background music and immersive display systems on memory for facts learned in an educational virtual environment

Game technology has been widely used for educational applications, however, despite the common use of background music in games, its effect on learning has been largely unexplored. This paper discusses how music played in the background of a computer-animated history lesson affected participants’ memory for facts. A virtual history lesson was presented to participants with different background stimuli (music or no-music) to test the effect of music on memory. To test the role of immersion on memory and its possible relationship to the music, two different display systems (3-monitor display system or immersive Reality Center) were used in the study. Overall, participants remembered a significantly higher number of facts using the 3-monitor display system, particularly if no background music was played in the second half of the history lesson. Conversely, for participants using the Reality Center, significantly higher recall of facts was found when participants listened to music in the second half of the history lesson. Cognitive load/overload and (un-)familiarity with the technology are offered as explanations.     

Robustness analysis framework for computations associated with building performance models and immersive virtual experiments

Building performance models (BPMs) have been used to simulate and analyze building performance during design. While extensive research efforts have made to improve the performance of BPMs, little attention has given to their robustness. Uncertainty is a crucial factor affecting the robustness of BPMs, in which such effect needs to be quantified through a suitable approach. The paper offers a robustness analysis framework for BPMs by using perturbation techniques to simulate uncertainty in input datasets. To investigate the efficacy of the framework, a generative adversarial network (GAN)-based framework was selected as a case study to analyze light switch usages in a single-occupancy office simulated using an immersive virtual environment (IVE). The robustness of the GAN was analyzed by comparing differences between a baseline (i.e., a BPM obtained from the GAN trained on a non-perturbed dataset) and BPMs obtained from the GAN trained on perturbed datasets. Overall, the robustness of the GAN significantly reduced when the training datasets were perturbed by using structured transformation techniques. The GAN remained relatively robust when the training datasets were perturbed by using an additive perturbation. Additionally, the sensitivity of the GAN involves different magnitudes corresponding to different levels of perturbed input datasets. The study suggests that the perturbation analysis is effective for investigating data uncertainty affecting the robustness of BPMs.     

Psychological and physiological responses to stressful situations in immersive virtual reality: Differences between users who practice mindfulness meditation and controls

Several studies in the literature have shown positive psychophysical effects during or immediately after mindfulness meditation. However, the extent to which such positive effects are maintained in real-life, stressful contexts, remains unclear. This paper investigates the effects of an 8-week mindfulness-oriented meditation (MOM) program on the psychological and physiological responses evoked by immersive virtual environments (IVEs) that simulate emergency situations that may occur in life. Before and after the 8-week period, healthy MOM participants and a group of controls not involved in any meditation course were administered self-report measures of mindfulness and anxiety, and acted in the IVEs while a set of physiological parameters were recorded. Responses of MOM participants to the immersive virtual experiences were different from those of controls. MOM participants showed increased mindfulness and decreased anxiety levels. They also showed decreased heart rate and corrugator muscle activity while facing IVEs. We explain these results in terms of the awareness and acceptance components of mindfulness. More generally, the present experimental methods could also open up new lines of research that combine psychological and physiological indices with ecologically valid stimuli provided by IVEs in an effort to increase understanding of the impact of mindfulness meditation on realistic life situations.     

Distributed planning and control systems for the virtual enterprise: organizational requirements and development life-cycle

This paper describes the requirements analysis and system specification of an Order Promise module to be used as part of a broader Decision Support System for production and operations planning of a Virtual Enterprise. This work is part of a broader project with a particular focus on the microelectronics industry which is a good example of Virtual Enterprise, and where a quick response to the customers needs and to unpredictable changes in production conditions is considered a major factor for success. First, the analysis and specification are presented within a development framework that involves the study of organizational issues of semiconductor enterprises. The use of ontological engineering for supporting the communication and shared understanding of the system concepts is explained and a virtual enterprise ontology is outlined. Following the clarification of the concept of virtual enterprise, the generic techno-organizational requirements for the information system are derived. Finally, a specification of the global planning module and a more detailed one regarding the order promise module is presented.     

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.     

System design and user evaluation of Co-Star: An immersive stereoscopic system for cable harness design

The system design and results of a user evaluation of Co-Star an immersive design system for cable harness design is described. The system used a stereoscopic head mounted graphical display, user motion tracking and hand-gesture controlled interface to enable cable harnesses to be designed using direct 3D user interaction with a product model. In order to determine how such a system interface would be used by a designer and to obtain user feedback on its main features a practical user evaluation was undertaken involving ten participants each completing three cable harness design tasks with the system. All user interactions with the system were recorded in a time stamped log file during each of the tasks, which were also followed by questionnaire (5 point scale) and interview sessions with each participant. The recorded interaction data for the third task was analysed using functional decomposition techniques and used to construct a single activity profile for the task based on the mean results obtained from the participant group. The goal was to identify in general terms the relative distribution of user activity between specific purposes during practical system operation, and it was found that in this task Navigation accounted for 41%, Design 27%, System Operation 23% and looking at Task Instructions 9% of all user activity. The scored questionnaire data collected immediately after the completion of each task was used to rank the major features of the system according to user opinion. This was further enhanced by also collecting real interview comments from the user group about these same features. The combination of both quantitative performance analysis and subjective user opinion data obtained during a practical design exercise has enabled an in depth evaluation of the system, leading to a much greater understanding of many of the key user and interface requirements that should be considered during the development of immersive interfaces and systems for practical engineering applications.     

Conversations with a virtual human: Synthetic emotions and human responses

To test whether synthetic emotions expressed by a virtual human elicit positive or negative emotions in a human conversation partner and affect satisfaction towards the conversation, an experiment was conducted where the emotions of a virtual human were manipulated during both the listening and speaking phase of the dialogue. Twenty-four participants were recruited and were asked to have a real conversation with the virtual human on six different topics. For each topic the virtual human’s emotions in the listening and speaking phase were different, including positive, neutral and negative emotions. The results support our hypotheses that (1) negative compared to positive synthetic emotions expressed by a virtual human can elicit a more negative emotional state in a human conversation partner, (2) synthetic emotions expressed in the speaking phase have more impact on a human conversation partner than emotions expressed in the listening phase, (3) humans with less speaking confidence also experience a conversation with a virtual human as less positive, and (4) random positive or negative emotions of a virtual human have a negative effect on the satisfaction with the conversation. These findings have practical implications for the treatment of social anxiety as they allow therapists to control the anxiety evoking stimuli, i.e., the expressed emotion of a virtual human in a virtual reality exposure environment of a simulated conversation. In addition, these findings may be useful to other virtual applications that include conversations with a virtual human.     

Performing ergonomics analyses through virtual interactive design: Validity and reliability assessment

Virtual interactive design methodology was initially proposed as an integrated environment for digital human model–based ergonomics analysis of human–machine interactions, which uses a motion capture system and virtual environment to realize task simulation. The validity and reliability of this analysis methodology has not been systematically studied. In this article, potential errors are first investigated based on the technology used in the system structure and work process; then, three experimental integration levels are proposed to use different resources for creating manikin postures. Validity and reliability to use this integrated environment for static ergonomics analysis are then assessed by comparing the ergonomics analysis outputs achieved based on the three integration levels. The results indicate that the reliability of this methodology is good; however, the validity of the integrated system is affected by the limitation of the virtual environment used in the testing. © 2011 Wiley Periodicals, Inc.     

Context-dependent product line engineering with lightweight formal approaches

This paper proposes a new style of product line engineering methods. It focuses on constructing embedded systems that take into account the contexts such as the external physical environments. In current product line development projects, Feature Analysis is mainly conducted from the viewpoint of system configurations: how hardware and software components are configured to constitute a system. In most cases, contexts are not considered explicitly. As a result, unexpected and unfavorable behavior might emerge in a system if a developer does not recognize any possible conflicting combinations between the system and contexts. To deal with this problem, this paper provides the notion of a context-dependent product line, which is composed of the system and context lines. The former is obtained by analyzing a family of systems. The latter is obtained by analyzing features of contexts associated to the systems. The system and context lines contain reusable core assets. The configuration of selected system components and contexts can be formally checked at the specification level. In this paper, we show a development process that includes the creation of both product line assets as well as context assets.     

Carpeno: interfacing remote collaborative virtual environments with table-top interaction

Creativity is enhanced by communication and collaboration. Thus, the increasing number of distributed creative tasks requires better support from computer-mediated communication and collaborative tools. In this paper we introduce “Carpeno”, a new system for facilitating intuitive face-to-face and remote collaboration on creative tasks. Normally the most popular and efficient way for people to collaborate is face-to-face, sitting around a table. Computer augmented surface environments, in particular interactive table-top environments, are increasingly used to support face-to-face meetings. They help co-located teams to develop new ideas by facilitating the presentation, manipulation, and exchange of shared digital documents displayed on the table-top surface. Users can see each other at the same time as the information they are talking about. In this way the task space and communication space can be brought together in a more natural and intuitive way. The discussion of digital content is redirected from a computer screen, back to a table where people can gather around. In contrast, collaborative virtual environments (CVE) are used to support remote collaboration. They frequently create familiar discussion scenarios for remote interlocutors by utilizing room metaphors. Here, virtual avatars and table metaphors are used, where the participants can get together and communicate with each other in a way that allows behaviour that is as close to face-to-face collaboration as possible. The Carpeno system described here combines table-top interaction with a CVE to support intuitive face-to-face and remote collaboration. This allows for simultaneous co-located and remote collaboration around a common, interactive table.

I became an attractive person in the virtual world: Users’ identification with virtual communities and avatars

A number of studies have examined virtual worlds, which can facilitate knowledge sharing, education, and enjoyment, among others. However, no study has provided an insightful research model for evaluating virtual worlds. This study suggests that users’ identification with virtual communities and avatars plays a critical role in the construction of attractive virtual worlds. The proposed model measures the level of the user’s identification with virtual communities, through which the user builds his or her trust in other community members. In addition, the study suggests that users’ identification with avatars is an important element of their satisfaction with virtual worlds. The results indicate that users’ identification with virtual communities as well as avatars can enhance their efficacy and trust and thus facilitate their sustained use of virtual services. The results have important theoretical and practical implications.     

The changing nature of user attitudes toward virtual world technology: A longitudinal study

Virtual world technologies have been utilized in gaming for a number of years but only recently have they been applied as a serious tool for business. Many business applications have been identified, including the use of virtual worlds for team collaboration, training, and education, but a question remains about whether users will accept the premise that virtual worlds represent useful environments for engaging in business functions. We address this question by examining user reactions to virtual worlds. The first study looks at attitudes of users of the virtual world Second Life during three time periods (i.e., before exposure to the environment, after an information session and discussion of Second Life, and after use of the environment). Two variables, user acceptance of virtual world technologies and user self-efficacy, were examined as the primary dependent measures. Results show that while self-efficacy increases over time, user acceptance decreases in a highly correlated pattern. A second study investigates the underlying causes of the observed pattern of user acceptance using a content analysis of written reflections of user experiences. Both studies paint a detailed picture of user intentions and some of the reasons these intentions developed after use. The paper concludes with a discussion of the implications of these results for business managers and researchers.