A desktop VR prototype for industrial training applications

The recent advances in computer graphics has spurred interest from both academics and industries in virtual reality (VR) enabled training applications. This paper presents a desktop VR prototype for industrial training applications. It is designed and implemented as a general shell by providing the data interface to import both the virtual environment models and specific domain knowledge. The geometric models of the virtual environment are constructed using feature-based modelling and assembly function by external CAD tools, and then transferred into the prototype through a conversion module. A hierarchical structure is proposed to partition and organise these imported virtual environment models. Based on this structure, a visibility culling approach is developed for fast rendering and user interaction. The case study has demonstrated the functionality of the proposed prototype system by applying it to a maintenance training application for a refinery bump system, which, in general, has a large number of polygons and a certain depth complexity. Significant speedup in both context rendering and response to user manipulations has been achieved to provide the user with a fast system response within the desktop virtual environment. Compared with the immersive VR system, the proposed system has offered an affordable and portable training media for industrial applications.The work was done in Nanyang Technological University.     

CAD模型转换到VR模型的实时可视性研究

实时3D 可视化是机械产品设计当中的一个主要问题,针对CAD几何学和装配以及机械产品的高可视化提出了MEMPHIS中间件框架,目的是连接实时虚拟现实应用,特别是CAD模型的管理操作设计。同时还保证了当进行模型转换时,在CAD模型上必须作相应的几何改变,系统会同步变化,这样避免了传统的从CAD模型转换到VR模型转换时人工加入虚拟现实特殊信息,如光照设置、材质、纹理、行为这些刻画VR模型可视化特性的信息属性后才能实现这种转换。从而大大提高了机械产品协同设计的效率和可视性性能。     

A virtual reality-based experimentation environment for the verification of human-related factors in assembly processes

This paper investigates the use of virtual reality (VR)-based methods for the verification of performance factors related to manual assembly processes. An immersive and interactive virtual environment has been created to provide functionality for realistic process experimentation. Ergonomic models and functions have been embedded into the VR environment to support verification and constrain experimentation to ergonomically acceptable conditions. A specific assembly test case is presented, for which a semi-empirical time model is developed employing statistical design experimentation in the virtual environment. The virtual experimentation results enable the quantification and prediction of the influence of a number of process parameters and their combination at the process cycle time.     

Research on environmental landscape design based on virtual reality technology and deep learning

Virtual Reality (VR) provides immersive visualization and intuitive interaction. The VR is used to enable any biomedical profession to develop a deep learning (DL) model for image classification. The Deep Neural Network (DNN) models can be used as a powerful tool for data analysis, but they are also challenging to understand and develop. To make deep learning more convenient and fast operation, it have established a landscape of DNN development environment based on virtual reality. In this environment, users can move concrete objects only to build neural networks with their own hands. It automatically transforms these configurations into a trainable model and reports on the real-time test dataset. In addition to realizing the insights users are developing into DNN models, it has also visually enriched the virtual environmental landscape objects with the parts of the model. In this way bridge the gap between professionals in different disciplines, providing a new perspective on the model analysis and data interaction. This system further demonstrates that learning and visualization technologies developed in Shenzhen can benefit from integrating virtual reality.     

基于Web的虚拟展厅的设计与实现

当前的VR标准，可以描述一个被动的VR系统，即系统应用VR技术来实现一些预先设计的3D内容的可视化。然而，现实中的一个挑战在于构造主动VR应用，这种应用允许动态合成虚拟场景。为了能构造主动VR应用，该文结合目前工作描述了网络环境下虚拟展览厅系统，提出虚拟世界的数据库建模方法。文中阐述了该系统的用途和功能，给出了系统的结构模型等。     

Effects of immersion on visual analysis of volume data

Volume visualization has been widely used for decades for analyzing datasets ranging from 3D medical images to seismic data to paleontological data. Many have proposed using immersive virtual reality (VR) systems to view volume visualizations, and there is anecdotal evidence of the benefits of VR for this purpose. However, there has been very little empirical research exploring the effects of higher levels of immersion for volume visualization, and it is not known how various components of immersion influence the effectiveness of visualization in VR. We conducted a controlled experiment in which we studied the independent and combined effects of three components of immersion (head tracking, field of regard, and stereoscopic rendering) on the effectiveness of visualization tasks with two x-ray microscopic computed tomography datasets. We report significant benefits of analyzing volume data in an environment involving those components of immersion. We find that the benefits do not necessarily require all three components simultaneously, and that the components have variable influence on different task categories. The results of our study improve our understanding of the effects of immersion on perceived and actual task performance, and provide guidance on the choice of display systems to designers seeking to maximize the effectiveness of volume visualization applications.     

Immersive Multiplayer Games With Tangible and Physical Interaction

In this paper, we present a new immersive multiplayer game system developed for two different environments, namely, virtual reality (VR) and augmented reality (AR). To evaluate our system, we developed three game applications-a first-person-shooter game (for VR and AR environments, respectively) and a sword game (for the AR environment). Our immersive system provides an intuitive way for users to interact with the VR or AR world by physically moving around the real world and aiming freely with tangible objects. This encourages physical interaction between players as they compete or collaborate with other players. Evaluation of our system consists of users' subjective opinions and their objective performances. Our design principles and evaluation results can be applied to similar immersive game applications based on AR/VR.     

Visualization and analysis using virtual reality

Current virtual reality technologies have not yet crossed the threshold of usability. Not surprisingly, VR has so far shown more promise than practical applications. Yet the promise looks bright for fields such as data visualization and analysis. For such problems, VR offers a natural interface between human and computer that will simplify complicated manipulations of the data. It also provides an opportunity to rely on the interplay of combined senses rather than on a single or even dominant sense. Still, we cannot yet say whether VR is better than other visualization and analysis approaches for certain classes of data and, if so, by how much. The payoff will come not for those applications or tasks for which VR is merely better, even if significantly, but for those applications or tasks for which it offers some unique advantage unavailable otherwise. To answer these questions, we embarked on a multipronged program involving the Graphics, Visualization. and Usability (GVU) Center, the Office of Information Technology Scientific Visualization Lab. and other research groups at Georgia Tech. Integration is mandatory since these questions involve basic considerations: how immersive environments affect user interfaces and human-computer interactions; the ranges and capabilities of sensors; computer graphics and the VR optical system; and applications' needs. We describe some of our results     

VR-Rides: An object-oriented application framework for immersive virtual reality exergames

Exercise can improve health and well-being. With this in mind, immersive virtual reality (VR) games are being developed to promote physical activity, and are generally evaluated through user studies. However, building such applications is time consuming and expensive. This paper introduces VR-Rides, an object-oriented application framework focused on the development of experiment-oriented VR exergames. Following the modular programming pattern, this framework facilitates the integration of different hardware (such as VR devices, sensors, and physical activity devices) within immersive VR experiences that overlay game narratives on Google Street View panoramas. Combining software engineering and interaction patterns, modules of VR-Rides can be easily added and managed in the Unity game engine. We evaluate the code efficiency and development effort across our VR exergames developed using VR-Rides. The reliability, maintainability, and usability of our framework are also demonstrated via code metrics analysis and user studies. The results show that investing in a systematic approach to reusing code and design can be a worthwhile effort for researchers beyond software engineering.     

Frontiers in User-Computer Interaction

In this age of (near-)adequate computing power, the power and usability of the user interface is as key to an application's success as its functionality. Most of the code in modern desktop productivity applications resides in the user interface. But despite its centrality, the user interface field is currently in a rut: the WIMP (Windows, Icons, Menus, Point-and-Click GUI based on keyboard and mouse) has evolved little since it was pioneered by Xerox PARC in the early '70s. Computer and display form factors will change dramatically in the near future and new kinds of interaction devices will soon become available. Desktop environments will be enriched not only with PDAs such as the Newton and Palm Pilot, but also with wearable computers and large-screen displays produced by new projection technology, including office-based immersive virtual reality environments. On the input side, we will finally have speech-recognition and force-feedback devices. Thus we can look forward to user interfaces that are dramatically more powerful and better matched to human sensory capabilities than those dependent solely on keyboard and mouse. 3D interaction widgets controlled by mice or other interaction devices with three or more degrees of freedom are a natural evolution from their two-dimensional WIMP counterparts and can decrease the cognitive distance between widget and task for many tasks that are intrinsically 3D, such as scientific visualization and MCAD. More radical post-WIMP UIs are needed for immersive virtual reality where keyboard and mouse are absent. Immersive VR provides good driving applications for developing post-WIMP UIs based on multimodal interaction that involve more of our senses by combining the use of gesture, speech, and haptics.     

Effects of virtual presence and learning outcome using low-end virtual reality systems

Low-cost technology is essential to integrate Virtual Reality (VR) into educative institutions around the world. However, low-cost technology usually refers to low-end technology, which may compromise the level of immersion of the VR system. This study evaluates whether low-end and high-end VR systems achieve a comparable learning outcome regardless their immersion level. We also analyze the relationship between virtual presence and the learning outcome arising from a VR educational experience. An evaluation with 42 participants was conducted. We measured learning outcome and virtual presence under three different configurations, namely: a desktop computer, a low-end VR system, and a high-end VR system. The impact of simulator sickness was also analyzed. Results revealed a lower learning outcome in the less immersive configuration (i.e. desktop) and a similar learning outcome in both low-end and high-end VR systems. Even though low-end VR systems are less immersive and produce a lower level of virtual presence than high-end VR systems, the results support the use of low-end VR systems for educative applications.     

Using virtual reality and 3D industrial numerical models for immersive interactive checklists

At the different stages of the PLM, companies develop numerous checklist-based procedures involving prototype inspection and testing. Besides, techniques from CAD, 3D imaging, animation and virtual reality now form a mature set of tools for industrial applications. The work presented in this article develops a unique framework for immersive checklist-based project reviews that applies to all steps of the PLM. It combines immersive navigation in the checklist, virtual experiments when needed and multimedia update of the checklist. It provides a generic tool, independent of the considered checklist, relies on the integration of various VR tools and concepts, in a modular way, and uses an original gesture recognition. Feasibility experiments are presented, validating the benefits of the approach.     

Extending the desktop workplace by a portable virtual reality system

Users are increasingly recognizing the potential of virtual reality (VR) technology for applications such as data analysis, design review, product development, production planning, marketing, training, etc. The currently established workflow is to design and construct at a desktop system with CAD or modeling software, and visualize and evaluate the results at one or more VR centers equipped with CAVEs or Powerwalls.Discussions with users of VR installations have shown that there is a demand for smaller and more cost efficient VR installations. We have proposed the concept of a small VR system, PI-casso, based on user requirements, guidelines for office workplaces and some end-user tests which showed important limitations and the ergonomics problems of current VR installations. PI-casso is a compact, fully immersive VR system which complements the classic desktop workplace.In this paper we describe a set of user requirements and the results of the design in forming end-user tests, in addition to the concept and the technical specifications of the newly developed system. The first prototype of PI-casso was demonstrated at HCII 2003, where specialists from the human factors/ergonomics and the VR communities used our system and provided suggestions for improvement. This expert feedback was used to develop the improved versions described in this paper.     

VRMosaic: Web access from within a virtual environment

For the foreseeable future, users of virtual reality systems will likely spend more time in the “real” environment than in a virtual environment. In the “real” environment, users access much of their data using flat screen applications, which include tools such as authoring and analysis programs and even hyperlinked browsers. We believe that users will find VEs more appealing if they can import their flat screen applications. We also believe that the World Wide Web infrastructure, and supporting tools such as NCSA Mosaic, Netscape's Navigator, and Sun's HotJava, have become a de facto standard both for making data available and allowing limited interaction with that data. These beliefs motivated us to develop a technology for porting flat screen applications based on the 3.1 version of the InterViews toolkit into VR. NCSA Mosaic provided us with a real application that would: (1) test our infrastructure effectively; and (2) provide a compelling application example. We used our “2D interface in VR” infrastructure to port Mosaic into RealEyes, our VR system. Dubbed “VRMosaic”, this application lets users familiar with Mosaic access the Web from within an immersive VE. But VRMosaic is not just an embedded version of NCSA Mosaic-it also allows for VR specific features such as navigation within the VE     

VR-CAD integration: Multimodal immersive interaction and advanced haptic paradigms for implicit edition of CAD models

This paper presents an approach for the integration of Virtual Reality (VR) and Computer-Aided Design (CAD). Our general goal is to develop a VR-CAD framework making possible intuitive and direct 3D edition on CAD objects within Virtual Environments (VE). Such a framework can be applied to collaborative part design activities and to immersive project reviews. The cornerstone of our approach is a model that manages implicit editing of CAD objects. This model uses a naming technique of B-Rep components and a set of logical rules to provide straight access to the operators of Construction History Graphs (CHG). Another set of logical rules and the replay capacities of CHG make it possible to modify in real-time the parameters of these operators according to the user’s 3D interactions. A demonstrator of our model has been developed on the OpenCASCADE geometric kernel, but we explain how it can be applied to more standard CAD systems such as CATIA. We combined our VR-CAD framework with multimodal immersive interaction (using 6 DoF tracking, speech and gesture recognition systems) to gain direct and intuitive deformation of the objects’ shapes within a VE, thus avoiding explicit interactions with the CHG within a classical WIMP interface. In addition, we present several haptic paradigms specially conceptualized and evaluated to provide an accurate perception of B-Rep components and to help the user during his/her 3D interactions. Finally, we conclude on some issues for future researches in the field of VR-CAD integration.     

V-Pong: an immersive table tennis simulation

Most applications for immersive virtual environments (VEs) allow slow-or medium-speed user interaction. Examples of this kind of interaction include changing an object's position, triggering an action, or setting a control parameter. To broaden the application range for VR systems, we need to integrate technologies that allow for faster VE-user movements. This raises the question, What response times can we achieve with VR systems built from standard recent hardware components? To answer this question, we created a table tennis simulation as this game involves fast user movements and has moderate space requirements. In this article we report on the realization of our immersive table tennis simulation, V-Pong.     

Virtual and augmented reality for cultural computing and heritage: a case study of virtual exploration of underwater archaeological sites (preprint)

The paper presents different issues dealing with both the preservation of cultural heritage using virtual reality (VR) and augmented reality (AR) technologies in a cultural context. While the VR/AR technologies are mentioned, the attention is paid to the 3D visualization, and 3D interaction modalities illustrated through three different demonstrators: the VR demonstrators (immersive and semi-immersive) and the AR demonstrator including tangible user interfaces. To show the benefits of the VR and AR technologies for studying and preserving cultural heritage, we investigated the visualisation and interaction with reconstructed underwater archaeological sites. The base idea behind using VR and AR techniques is to offer archaeologists and general public new insights on the reconstructed archaeological sites allowing archaeologists to study directly from within the virtual site and allowing the general public to immersively explore a realistic reconstruction of the sites. Both activities are based on the same VR engine, but drastically differ in the way they present information and exploit interaction modalities. The visualisation and interaction techniques developed through these demonstrators are the results of the ongoing dialogue between the archaeological requirements and the technological solutions developed.     

Jabiru: harnessing Java 3D behaviors for device and display portability

The aim of this paper is to use the Java 3D ConfiguredUniverse utilities to create a set of behaviors that we can integrate into existing Java 3D programs, as long as these programs use the ConfiguredUniverse. We call this behavior package Jabiru (Java 3D Application Behavior Immersive Virtual Reality Utilities). The Jabiru set of behaviors, accessible from a master menu, facilitates moving Java 3D applications originally created for desktop environments to immersive VR environments and vice versa. Jabiru also provides a six-degrees-of-freedom (6DOF) device emulator for a conventional mouse, meant to facilitate offsite testing of immersive VR behaviors on a desktop. The main focus of our work with a CAVE is in relation to bioinformatics, which has embraced Java and Java 3D as one of the choice programming environments. By creating this package, we help bring 3D graphics to both the bioinformatics community and the casual Java 3D developer or user. We discuss the design and implementation of Jabiru.     

Bayesian tracking fusion framework with online classifier ensemble for immersive visual applications

During the last decade, the development of the immersive virtual reality (VR) has achieved a great progress in different application areas. For more advanced large-scale immersive VR environments or systems, one of the most challenge is to accurately track the position of the user’s body part such as head when he/she is immersived in the environment to feel the changes among the synthetic stereoscopic image sequences. Unfortunately, accurate tracking is not easy in the virtual reality scenarios due to the variety types of existing intrinsic and extrinsic changes when tracking is on-the-fly. Especially for the single tracker, a long time accurate tracking is usually not possible because of the model adaption problem in different environments. Recent trend of research in tracking is to incorporate multiple trackers into a compositive learning framework and utilize the advantages of different trackers for more effective tracking. Therefore, in this paper, we propose a novel Bayesian tracking fusion framework with online classifier ensemble strategy. The proposed tracking formulates a fusion framework for online learning of multiple trackers by modeling a cumulative loss minimization process. With an optimal pair-wise sampling scheme for the SVM classifier, the proposed fusion framework can achieve more accurate tracking performance when compared with the other state-of-art trackers. In addition, the experiments on the standard benchmark database also verify that the proposed tracking is able to handle the challenges in many immersive VR applications and environments.