VADE: a Virtual Assembly Design Environment

Virtual Assembly Design Environment (VADE) resulted from a research and development project started in 1995, sponsored by the National Institute of Standards and Technology (NIST). The main purpose of this project was to explore the potential and the technical challenges in using VR technologies for design and manufacturing by creating a VE for assembly planning and evaluation. In this article, we describe the overall system, the important features, and examples of using VADE. We also discuss the benefits and limitations of virtual assembly systems. In addition, we compare virtual assembly and automated assembly planning systems     

A desktop VR prototype for industrial training applications

The recent advances in computer graphics has spurred interest from both academics and industries in virtual reality (VR) enabled training applications. This paper presents a desktop VR prototype for industrial training applications. It is designed and implemented as a general shell by providing the data interface to import both the virtual environment models and specific domain knowledge. The geometric models of the virtual environment are constructed using feature-based modelling and assembly function by external CAD tools, and then transferred into the prototype through a conversion module. A hierarchical structure is proposed to partition and organise these imported virtual environment models. Based on this structure, a visibility culling approach is developed for fast rendering and user interaction. The case study has demonstrated the functionality of the proposed prototype system by applying it to a maintenance training application for a refinery bump system, which, in general, has a large number of polygons and a certain depth complexity. Significant speedup in both context rendering and response to user manipulations has been achieved to provide the user with a fast system response within the desktop virtual environment. Compared with the immersive VR system, the proposed system has offered an affordable and portable training media for industrial applications.The work was done in Nanyang Technological University.     

Establishing the Usability of a Virtual Training System for Assembly Operations within the Automotive Industry

Virtual training systems deliver training within a virtual environment (VE) using virtual reality (VR) or augmented reality (AR) technologies. However, to be fully accepted as a valid tool for training within the automotive industry, evidence is required on the ability of these systems to deliver effective and efficient training to the relevant users. This paper aims to investigate the effectiveness and efficiency of the first prototype of the virtual training system (VTS) developed within the VISTRA (Virtual Simulation and Training of Assembly and Service Processes in Digital Factories) project (FP7‐ICT‐285176), using real end users from the OPEL automotive plant in Rüsselsheim, Germany. Two separate and independent studies were employed that used objective and subjective methods of investigation to establish performance and usability measures. The objective results show that virtual training was effective in reducing error during task performance when compared to traditional training. The subjective results concluded that the opinions of the participants were mainly positive concerning the overall use of the VTS for assembly operation training; however, a number of issues were highlighted and reported to the developers for further advancement of the system.     

基于行车的Fusion虚拟装配系统的设计研究

核装置体积大、结构复杂、装配精度高,为降低该类型装置的装配风险,节约装配成本,提出了一个以面向PC平台、支持大型复杂装置装配规划、训练、演示为主要目标的基于行车的虚拟装配系统(FVAS).文中通过对三维场景交互、大模型实时碰撞检测、装配定位、装配过程记录、多视点漫游观察等虚拟装配主要技术环节的研究,利用面向对象的视景仿真开发包Open Inventor,结合VC 和商用CAD软件Unigraphics,实现了FVAS的原型设计.该系统成功仿真了我国即将运行的先进超导托克马克实验装置EAST的装配方案,并在先进的聚变驱动次临界堆FDS-I概念设计研究中得到应用.     

Reflecting on the Design Process for Virtual Reality Applications

A reflective analysis on the experience of virtual environment (VE) design is presented focusing on the human–computer interaction (HCI) challenges presented by virtual reality (VR). HCI design guidelines were applied to development of two VRs, one in marine archaeology and the other in situation awareness simulation experiments. The impact of methods and HCI knowledge on the VR design process is analyzed, leading to proposals for presenting HCI and cognitive knowledge in the context of design trade-offs in the choice of VR design techniques. Problems reconciling VE and standard Graphical User Interface (GUI) design components are investigated. A trade-off framework for design options set against criteria for usability, efficient operation, realism, and presence is proposed. HCI-VR design advice and proposals for further research aimed towards improving human factor-related design in VEs are discussed.     

Fatigue evaluation in maintenance and assembly operations by digital human simulation in virtual environment

Virtual human techniques have been used a lot in industrial design in order to consider human factors and ergonomics as early as possible, and it has been integrated into VR applications to complete ergonomic evaluation tasks. In order to generalize the evaluation task in VE, especially for physical fatigue evaluation, we integrated a new fatigue model into a virtual environment platform. Virtual Human Status is proposed in this paper in order to assess the difficulty of manual handling operations, especially from the physical perspective. The decrease of the physical capacity before and after an operation is used as an index to indicate the difficulty level. The reduction of physical strength is simulated in a theoretical approach on the basis of a fatigue model in which fatigue resistances of different muscle groups were regressed from 24 existing maximum endurance time models. A framework based on digital human modeling technique is established to realize the comparison of physical status. An assembly case in airplane assembly is simulated and analyzed under the framework in VRHIT experiment platform. The endurance time and the decrease of the joint moment strengths are simulated. The experimental result in simulated operations under laboratory conditions confirms the feasibility of the theoretical approach: integration of virtual human simulation into virtual reality for physical fatigue evaluation.     

Manipulating Objects in Virtual Worlds: Categorization and Empirical Evaluation of Interaction Techniques

The acceptance of virtual environment (VE) technology requires scrupulous optimization of the most basic interactions to maximize user performance and provide efficient and enjoyable virtual interfaces. Motivated by insufficient understanding of human factors implications in the design of interaction techniques for object manipulation in virtual worlds, this paper presents results of a formal study that evaluated two basic interaction metaphors for virtual manipulation—virtual pointer and virtual hand—in object selection and positioning tasks. In this work, we survey and categorize current virtual manipulation techniques according to their basic design metaphors, conduct experimental studies of the most basic techniques, and derive guidelines to aid designers in the practical development of VE applications.     

Virtual and interactive environments for work of the future

Virtual reality (VR) systems and the virtual environments (VEs) experienced within them have presented challenges to human computer interaction over many years. The sheer range of different interfaces which might be experienced and of different behaviours which might be exhibited have caused difficulties in general understanding of participants’ performance within VR/VE and in providing coherent guidance for designers. We have recently completed a European Information Society Technologies (IST) project, Virtual and Interactive Environments for Workplaces of the Future (VIEW of the Future), which has made great strides in developing improved VR systems and interaction concepts and devices, based upon good understanding of participation in VEs. Particular emphasis in the VIEW of the Future project has been upon mobility and multiple active collaboration in use of VR/VE. This paper introduces a special issue devoted to this project and overviews the project as a whole. In doing so it also reviews some of the human factors issues defined for VR/VE over the years and the contribution of VIEW of the Future to addressing these.     

Using virtual environments to train firefighters

Using virtual environments for training and mission rehearsal gives US Navy firefighters an edge in fighting real fires. A test run on the ex-USS Shadwell measured the improvement. The results suggest that virtual environments serve effectively for training and mission rehearsal for shipboard firefighting. VE training provides a flexible environment where a firefighter can not only learn an unfamiliar part of the ship, but also practice tactics and procedures for fighting a fire by interacting with simulated smoke and fire without risking lives or property. These tests proved a successful first step in developing a new training technology for shipboard firefighting based on immersive virtual environments. The tests also indicated potential areas for improvement, requiring additional research. User interaction techniques for manipulating objects in VEs need further study, along with usability studies to determine their effectiveness or utility. Other areas that could enhance VE training systems include more natural and intuitive I/O devices such as 3D sound, speech and natural language input, integrated multimedia and hypermedia instruction, and multi user interaction     

Multiple decoupled interaction: An interaction design approach for groupware interaction in co-located virtual environments

Interactive visualizations such as virtual environments and their associated input and interface techniques have traditionally focused on localized single-user interactions and have lacked co-present active collaboration mechanisms where two or more co-located users can share and actively cooperate and interact with the visual simulation. VR facilities such as CAVEs or PowerWalls, among many others, seem to promise such collaboration but due to the special requirements in terms of 3D input and output devices and the physical configuration and layout, they are generally designed to support an active controlling participant—the immersed user—and a passive viewing only audience. In this paper we explore the integration of different technologies, such as small handheld devices and wireless networks with VR/VEs in order to develop a technical and conceptual interaction approach that allows creation of a more ad hoc, interaction rich, multimodal and multi-device environment, where multiple users can access certain interactive capabilities of VE and support co-located collaboration.     

单兵装备效能评估系统中虚拟现实平台的设计

探讨了单兵装备效能评估系统中虚拟现实分系统的设计，重点研究了虚拟现实场景的设计实现，人机交互接口的设计及音效设计等关键问题。该分系统用于营造满足装备效能评估实验需求的虚拟战场环境。     

Provision and maintenance of presence and immersion in hand‐held virtual reality through motion based interaction

Hand‐held devices are also becoming computationally more powerful and being equipped with special sensors and non‐traditional displays for diverse applications aside from just making phone calls. As such, it raises the question of whether realizing virtual reality, providing a minimum level of immersion and presence, might be possible on a hand‐held device capable of only relatively “small” display. In this paper, we propose that motion based interaction can widen the perceived field of view (FOV) more than the actual physical FOV, and in turn, increase the sense of presence and immersion up to a level comparable to that of a desktop or projection display based VR systems. We have implemented a prototype hand‐held VR platform and conducted two experiments to verify our hypothesis. Our experimental study has revealed that when a motion based interaction was used, the FOV perceived by the user for the small hand held device was significantly greater than (around 50%) the actual. Other larger display platforms using the conventional button or mouse/keyboard interface did not exhibit such a phenomenon. In addition, the level of user felt presence in the hand‐held platform was higher than or comparable to those in VR platforms with larger displays. We hypothesize that this phenomenon is related to and analogous to the way the human vision system compensates for differences in acuity resolution in the eye/retina through the saccadic activity. The paper demonstrates the distinct possibility of realizing reasonable virtual reality even with devices with a small visual FOV and limited processing power. Copyright © 2010 John Wiley & Sons, Ltd.     

Multi-modal virtual environments for education with haptic and olfactory feedback

It has been suggested that immersive virtual reality (VR) technology allows knowledge-building experiences and in this way provides an alternative educational process. Important key features of constructivist educational computer-based environments for science teaching and learning, include interaction, size, transduction and reification. Indeed, multi-sensory VR technology suits very well the needs of sciences that require a higher level of visualization and interaction. Haptics that refers to physical interactions with virtual environments (VEs) may be coupled with other sensory modalities such as vision and audition but are hardly ever associated with other feedback channels, such as olfactory feedback. A survey of theory and existing VEs including haptic or olfactory feedback, especially in the field of education is provided. Our multi-modal human-scale VE VIREPSE (virtual reality platform for simulation and experimentation) that provides haptic interaction using a string-based interface called SPIDAR (space interface device for artificial reality), olfactory and auditory feedbacks is described. An application that allows students experiencing the abstract concept of the Bohr atomic model and the quantization of the energy levels has been developed. Different configurations that support interaction, size and reification through the use of immersive and multi-modal (visual, haptic, auditory and olfactory) feedback are proposed for further evaluation. Haptic interaction is achieved using different techniques ranging from desktop pseudo-haptic feedback to human-scale haptic interaction. Olfactory information is provided using different fan-based olfactory displays (ODs). Significance of developing such multi-modal VEs for education is discussed.     

集成虚拟现实环境在工程设计分析中的应用 *

虚拟现实技术在工程设计与分析上有很大的应用潜力。可是 ,虚拟环境的创建时间长、成本高,极大地限制了虚拟现实技术在工程上的应用。为此 ,提出了虚拟现实脚本生成器的概念,将虚拟现实技术与过程的设计、分析集成化,创造性地提出了一种自上而下的设计方法———VR-IEDA,以便高效、快速地创建虚拟原型,提高计算机虚拟复杂系统工程的性价比。在 VR-IEDA中复杂系统的结构和行为被计算机捕捉并自动生成可执行的 VR仿真代码 ,从而减少了创建虚拟环境的时间,便利了基于仿真的过程分析和设计。最后 ,以武器系统的维修过程为     

基于虚拟现实的流程工业过程模拟仿真系统

将虚拟现实技术与系统仿真技术相结合,建立了炼油企业虚拟现实过程模拟仿真平台,分析了仿真平台的结构设计与实现原理。介绍了一种基于虚拟现实的炼油企业过程模拟仿真算法,详细说明了该算法在虚拟现实仿真平台上的应用过程,包括重点装置动态模拟、罐区油品调合仿真、进出厂模拟等。虚拟现实仿真平台通过生产过程与数据的可视化呈现与人机交互以及现场应用,可以为炼油企业流程模拟仿真培训与生产过程三维可视化监控提供有效的支持。     

Virtual orthopedic surgery training

Medicine is one of the most promising areas for emerging computer graphics and virtual reality techniques. VR training simulators let surgeons practice on virtual body tissue and get the same feedback they would experience in performing a real operation. Hybrid VR systems permit medical practitioners to view the patient overlaid with 3D data sets derived from 3D scanners, thus providing doctors and surgeons with pseudo X-ray vision. While currently available immersive VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computers     

Robot-enabled tangible virtual assembly with coordinated midair object placement

The assembly in Virtual Reality (VR) enables users to fit virtual parts into existing 3D models immersively. However, users cannot physically feel the haptic feedback when connecting the parts with the virtual model. This work presents a robot-enabled tangible interface that dynamically moves a physical structure with a robotic arm to provide physical feedback for holding a handheld proxy in VR. This enables the system to provide force feedback during virtual assembly. The cooperation between the physical support and the handheld proxy produces realistic physical force feedback, providing a tangible experience for various virtual parts in virtual assembly scenarios. We developed a prototype system that allowed the operator to place a virtual part onto other models in VR by placing the proxy onto the matched structure attached to a robotic arm. We conducted a user evaluation to explore user performance and system usability in a virtual assembly task. The results indicated that the robot-enabled tangible support increased the task completion time but significantly improved the system usability and sense of presence with a more realistic haptic experience.     

虚拟教学环境中基于草图的自适应用户界面研究

将草图技术、自适应技术与传统的虚拟现实技术相结合应用于虚拟教学过程中,通过增强用户和系统之间个性化需求的交互处理来提高系统的智能性和友好性,进而有效提高虚拟教学环境的应用效果.旨在虚拟环境中构建基于草图的自适应用户界面并应用于虚拟教学来满足特定教学需求,着重结合实例分析了基于自适应草图用户界面的虚拟教学环境中的草图上下文处理机制.在上述研究基础上,设计开发了一个虚拟教学原型系统,实验证明该系统在用户体验上有明显的改善.     

Interaction and visual performance in stereoscopic displays: A review

In this paper, authors systematically selected and reviewed articles related to stereoscopic displays and their advances, with a special focus on perception, interaction, and corresponding challenges. The aim was to understand interaction‐related problems, provide possible explanations, and identify factors that limit their applications. Despite promising advancements, there are still issues that researchers in the field fail to explain precisely. The two major problems in stereoscopic viewing are, compared with the real world, objects are perceived to be smaller than they actually are and there are discomfort and visual syndromes. Furthermore, there is general agreement that humans underestimate their egocentric distance in a virtual environment (VE). Our analysis revealed that in the real world, distance estimation is about 94% accurate, but in VE, it is only about 80% accurate. This problem could reduce the efficacy of different sensory motor‐based applications where interaction is important. Experts from human factors, computing, psychology, and others have studied contributing factors such as types of perception/response method, quality of graphics, associated stereoscopic conditions, experience in virtual reality (VR), and distance signals. This paper discusses the factors requiring further investigation if the VR interaction is to be seamlessly realized. In addition, engineering research directions aiming at improving current interaction performances are recommended.     

基于虚拟现实技术的运动辅助训练系统设计

为了提高教练的指导效率和丰富羽毛球教学与训练的方法,基于虚拟现实技术和KINECT,开发了一套羽毛球辅助训练系统。该系统利用虚拟现实技术,通过3DS MAX工具进行训练场地的虚拟化建模,搭配Cry Engine引擎技术渲染训练场地场景。KINECT被用来识别这些动作以及使用者的位置变化,并将识别结果通过蓝牙模块传输至VR眼镜内,使VR眼镜内的虚拟环境做出图像、声音等相应的改变,以实现虚拟环境下的人机互动。测试表明,本系统能够准确地实现训练场景的虚拟化,对使用者的各个动作的识别率较高,提高了训练者的学习效率。