A study on the factors that improve the velocity perception of a virtual reality‐based vehicle simulator

The Virtual Reality (VR) system of a real‐time VR‐linked vehicle simulator that was used in this study provides visual information and sound effects to participants. The VR system of a VR‐linked vehicle simulator should provide a perceived velocity similar with the perceived velocity in actual driving. To achieve these goals, modeling and rendering methods that offer an improved performance for complex VR applications, such as the 3D road model, were implemented and evaluated. We also evaluated the influences of graphic and engine sound effects on the driver and analyzed each result according to a driver's viewpoint, the dot densities of road texture provided, the lateral distance between a virtual driver and environmental objects, and the engine sound. Each factor was individually analyzed through an experiment that evaluated the influence of visual images or sound effects in the vehicle simulator. Through the experimental evaluation, the research results could be used for improving the effectiveness of VR‐based vehicle simulators.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

A research on an intelligent multipurpose fuzzy semantic enhanced 3D virtual reality simulator for complex maritime missions

In this article, the idea of multipurpose fuzzy semantic enhanced 3D virtual reality simulator for the evaluation of maritime robot algorithms, and for the analysis of maritime missions is presented. The simulator uses the digital mockup technology in blending with semantic domain knowledge of the system, to analyze the tasks remotely. The 3D virtual reality (VR) is proposed to help the operators provide the detailed information and guidance during real-time tele-operations, and the incorporation of fuzzy semantic knowledge makes the virtual environment intelligent and automatic. The integration of fuzzy scene independent ontology with the virtual environment (VE) engenders a knowledge driven inter operable virtual environment which eases the user in natural language querying, personalization, interpretations and manipulation. The distinctive semantic VR scene builder utility of the proposed system draws the VR environment automatically while getting the high level specification of the system for the client. The proposed simulator can be effectively used for real-time robots trainings and for the evaluation of AI based algorithms designed for intelligent vessels and AUV’s without knowing the complex underlying VR scene building technologies. Furthermore, it provides the benefits to optimize the pre-process physical environment operations to mimic the real world into a virtual environment. The remote operations and feasibility analysis performed on virtual simulator can efficiently save the cost, time and claims to provide the operators a preprocessor information and guide.     

Sense of Immersion in Computer Games Using Single and Stereoscopic Augmented Reality

As a novel interaction and presentation technology, Augmented Reality (AR) has been widely used in education, tourism, entertainment, psychology, and many other applications. In the game industry, this revolutionary technology has introduced a new form of interaction between players and the gaming environment. In this article, we aim to find if exploiting AR in computer games can enhance the sense of immersion, and consequently, satisfying the players. To this end, a mobile game called Ladybug is implemented using three different techniques including the regular virtual reality (VR) camera, single AR camera, and stereoscopic AR camera. The results of a comprehensive user study on these techniques showed that the single AR camera and stereoscopic AR camera can result in more sense of immersion in players in comparison to the VR camera. However, the hypothesis that the stereoscopic AR results in more sense of presence in comparison to the single AR camera is rejected. On the other hand, our findings show that the AR stereoscopic camera can lead to a better emotional involvement and enjoyment in comparison to the single AR camera and VR camera.     

一种基于视沉的增强现实三维注册算法

为了提高增强现实系统三维注册的精度和效率，提出了一种基于计算机视觉的增强现实三维注射算法，并已将其用于采用透射式头盔显示器的增强现实系统中，该算法有如下特点：（1）构架简单，实用性强，一般情况下只需4个平面标志物就可实现三维注册；（2）工作范围大，甚至可以应用到室外的增强现实系统中；（30数值求解过程最线性过程，误差小，可以满足增强现实系统高精度三维注册的要求，另外，应用这种算法所需要作的图象处理计算量很小，其典型系统包括一个彩色CCD和几个不同颜色的标志点，由于很容易获得标志点的对应像面位置，而且不需要计算图象对，因此实时性好，是一般图形工作站和PC机上的增强现实系统进行实时注册的算法基础。     

Augmented reality-assisted robot programming system for industrial applications

Robots are important in high-mix low-volume manufacturing because of their versatility and repeatability in performing manufacturing tasks. However, robots have not been widely used due to cumbersome programming effort and lack of operator skill. One significant factor prohibiting the widespread application of robots by small and medium enterprises (SMEs) is the high cost and necessary skill of programming and re-programming robots to perform diverse tasks. This paper discusses an Augmented Reality (AR) assisted robot programming system (ARRPS) that provides faster and more intuitive robot programming than conventional techniques. ARRPS is designed to allow users with little robot programming knowledge to program tasks for a serial robot. The system transforms the work cell of a serial industrial robot into an AR environment. With an AR user interface and a handheld pointer for interaction, users are free to move around the work cell to define 3D points and paths for the real robot to follow. Sensor data and algorithms are used for robot motion planning, collision detection and plan validation. The proposed approach enables fast and intuitive robotic path and task programming, and allows users to focus only on the definition of tasks. The implementation of this AR-assisted robot system is presented, and specific methods to enhance the performance of the users in carrying out robot programming using this system are highlighted.     

用于虚拟现实系统的眼动交互技术综述

眼动人机交互利用眼动特点可以增强用户的沉浸感和提高舒适度,在虚拟现实（VR）系统中融入眼动交互技术对VR系统的普及起到至关重要的作用,已成为近年来的研究热点。对VR眼动交互技术的原理和类别进行阐述,分析了将VR系统与眼动交互技术结合的优势,归纳了目前市面上主流VR头显设备及典型的应用场景。在对有关VR眼动追踪相关实验分析的基础上,总结了VR眼动的研究热点问题,包括微型化设备、屈光度矫正、优质内容的匮乏、晕屏与眼球图像失真、定位精度、近眼显示系统,并针对相关的热点问题展望相应的解决方案。     

Vision‐tangible interactive display method for mixed and virtual reality: Toward the human‐centered editable reality

Building a human‐centered editable world can be fully realized in a virtual environment. Both mixed reality (MR) and virtual reality (VR) are feasible solutions to support the attribute of edition. Based on the current development of MR and VR, we present the vision‐tangible interactive display method and its implementation in both MR and VR. We address the issue of MR and VR together because they are similar regarding the proposed method. The editable mixed and virtual reality system is useful for studies, which exploit it as a platform. In this paper, we construct a virtual reality environment based on the Oculus Rift, and an MR system based on a binocular optical see‐through head‐mounted display. In the MR system about manipulating the Rubik's cube, and the VR system about deforming the virtual objects, the proposed vision‐tangible interactive display method is utilized to provide users with a more immersive environment. Experimental results indicate that the vision‐tangible interactive display method can improve the user experience and can be a promising way to make the virtual environment better.     

A Survey on Sketch Based Content Creation: from the Desktop to Virtual and Augmented Reality

Sketching is one of the most natural ways for representing any object pictorially. It is however, challenging to convert sketches to 3D content that is suitable for various applications like movies, games and computer aided design. With the advent of more accessible Virtual Reality (VR) and Augmented Reality (AR) technologies, sketching can potentially become a more powerful yet easy-to-use modality for content creation. In this state-of-the-art report, we aim to present a comprehensive overview of techniques related to sketch based content creation, both on the desktop and in VR/AR. We discuss various basic concepts related to static and dynamic content creation using sketches. We provide a structured review of various aspects of content creation including model generation, coloring and texturing, and finally animation. We try to motivate the advantages that VR/AR based sketching techniques and systems can offer into making sketch based content creation a more accessible and powerful mode of expression. We also discuss and highlight various unsolved challenges that current sketch based techniques face with the goal of encouraging future research in the domain.     

Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device

Augmented reality (AR)-based programming using the demonstration method has been widely studied. However, studies on AR-based programming for remote robots are lacking because of the limitation of human–computer interaction. This paper proposes an AR-based robot teleoperation system and method using RGB-D imaging and an attitude teaching device. By sending the color and depth images of the remote robot environment to the local side, the operators can complete the teleoperation of the robot at the local side. First, the operators select key positions on the motion path of the robot endpoint from color images via a mouse, and the computer calculates the 3D coordinates of these key points in the robot base coordinate system to complete the position teaching process. In the robot attitude teaching process, the AR technology is used to superimpose the virtual robot model onto the color images of the robot teleoperation environment, so as to make the virtual robot endpoint to move along the teaching path. An operator can use the portable attitude teaching device designed in this study to control the robot movement parameters, such as the attitude and motion speed, during the movement of the virtual robot. After the position and attitude teaching processes, the robot movement trajectory can be generated. To make the base coordinate system of the virtual model consistent with that of the physical robot, we propose an online AR registration method, which does not require manually placing the AR registration marker. The proposed AR-based robot teleoperation system can quickly and easily complete robot teleoperation at the local side.     

虚拟现实技术在柔性上肢康复机器人中的应用

根据末端牵引式和外骨骼式上肢康复机器人的特性,研制了一种新型的柔性上肢康复机器人。机器人的康复训练系统结合了主动和被动模式的要素,将虚拟现实（VR）技术引入上肢康复机器人,通过现实环境中的光学3D位置捕获以及VR环境中的3D位置感知,设计了虚拟动态模型交互性节点和碰撞检测实验,实现虚拟现实交互,提高虚拟模型运动实时效果和上肢康复训练精度。VR和新颖的康复机器人的集成为具有特定任务的患者提供了有效的训练。     

Teaching and Learning Crystal structures through Virtual Reality based systems

Teaching and learning through Virtual Reality(VR) is an emerging technology in the last few years. In this article, the development and use of a VR based teaching–learning system for crystal structures are discussed. The VR system is designed as a lab environment where a user can do experiments related to crystal structures. The VR system is designed in Unity,¹ and Oculus Rift S² is used as a VR headset. Currently, the system consists of three phases; in the first phase user can visualize the crystal lattice structures, wherein the second one a user can visualize the light interaction with the crystal lattice structure using a virtual torch ray. The third phase is the X-ray Diffraction (XRD) experiment. In this phase, users can perform the XRD experiment in the lab environment by taking a random crystal from a crystal dispenser machine and placing it in the X-ray machine which identifies the chosen crystalline material and analyses the unit cell. The incident ray colour changes when there is a peak found in the crystal for a better understanding of the user. There is also an interactive display where users can increase/decrease the angles of the radiation and also lock and unlock the experiment to view the diffraction plot for the crystal structure. In many cases, it was found that XRD and the crystal structure is available in the course syllabus but there are no experiments to enhance their learning. Therefore an experiment with 39 participants was performed where the maximum participants are new to crystallography. The study was conducted in two phases; in the first phase, participants are asked to watch video tutorials of the topic followed by questionnaires; in the second phase participants are asked to do the VR based experiment and followed by questionnaires related to overall study and experiment. From the analysis of the study we found that everyone found VR based teaching methods are better than traditional book/video studies. Study results give an average score of 56.74% in comparison to VR based learning approach with an average score of 93.81%. Participants who took part in the experiment found the experience interactive and motivating and found it helpful to learn elusive concepts, which can be learned when simulated. For example, one participant wrote: “The VR experience was surreal and was easy to control. Lucid user experience. Got a view of XRD like never seen before”.     

Tomorrow''s space: Journey to the virtual worlds

The new technology of Virtual Reality (VR) provides a revolutionary way to improve the coupling of the human operator to the computing machine. In a VR system, the goal is to remove the distinction between the system and the user's environment, as compared to the conventional relationship between the user and a computer. Thus, to be able to immerse the user in his artificial world, the VR system shall make use simultaneously of the human senses and of his cognitive capacities. In practice, the VR system is the integration of computer graphics and various sensor and actuator technology to create the illusion of immersion in a computer generated reality.

VR has therefore been identified as a good candidate for the Technological Research Programme of the European Space Agency (ESA). In 1991, the Agency has initiated research activities in the field of VR to identify its potential applications for European space programmes. A generic VR system has been developed allowing the Agency to evaluate the performance of VR technology and to define research activities for the coming years. The aims are to build a European expertise in VR technology and to demonstrate the applicability of VR to space programmes.     

Toward AI-enabled augmented reality to enhance the safety of highway work zones: Feasibility,requirements, and challenges

Highway work zones are considered among the most hazardous working environments. In 2018 alone, 124 workers lost their lives to fatal accidents. The lack of predictive safety systems that notify workers of upcoming dangers in advance is a major reason to blame in the highway maintenance and operation community. This article presents an integrative design framework for bringing recent advances in Augmented Reality (AR) and Artificial Intelligence (AI) to enhance the safety of highway workers through real-time multimodal notifications on-spot. To this end, this article conceptualizes and co-designs three major pillars: (1) AR user interface design for multimodal notification, (2) real-time AI at the edge for vehicle detection/classification from distance, and (3) real-time wireless communication in work zone setting to enable latency-aware operation between AI and AR components. Our early results demonstrate that we can achieve 24.83 FPS end-to-end execution latency on the Xavier AGX Jetson board with 48.7% mAP on BDD100K dataset, and a real-time communication covering 120 meters with an average latency of 5.1 ms at the farthest distance. Our mixed-method user research also reveals an acceptable level of excitement and engagement from the body of highway workers toward both the proposed technology and the designed user interface. Overall, this article provides a proof-of-concept toward AI-enabled AR safety systems in highway work zones.     

基于VR的潜艇航行训练模拟器视景系统设计与实现

该文提出了基于VR(虚拟现实)的潜艇航行训练模拟器视景系统的体系结构设计方法、给出了软件实现的具体过程,总结了系统的主要特点。海试结果表明,该VR视景系统能充分模拟艇员日常训练的现实环境。     

增强现实技术综述

增强现实技术是将计算机渲染生成的虚拟场景与真实世界中的场景无缝融合起来的一种技术,它通过视频显示设备将虚实融合的场景呈现给用户,使人们与计算机之间的交互更加的自然,同时具有广泛的应用前景,因此成为近年来的一个研究热点。随着跟踪注册技术的进步、计算机性能的飞速发展、深度摄像机的普及,以及Light Field投影技术在增强现实中的应用,增强现实技术逐渐成为下一代人机交互的发展方向。该文章首先概述了增强现实的主要研究内容和发展情况,并详细介绍了增强现实的关键技术、开发工具,然后分类概述了增强现实应用案例。     

Teleporting through virtual environments: benefits of navigational feedback and practice

Virtual Reality - Virtual environments (VEs) can be infinitely large, but movement of the virtual reality (VR) user is constrained by the surrounding real environment. Teleporting has become a...     

基于约束物理系统和运动捕捉的人体运动仿真

为了能在基于PC平台的虚拟现实中实现更具真实感的人体运动仿真,该文提出一种改进的约束求解方法,将人体建模为基于约束关节树的骨骼结构。约束模型的求解采用基于线性互补问题的求解方法,同时结合相应的运动捕捉数据。实验结果表明,该方法所获得的运动数据不仅能满足交互式要求的帧速率,而且由于算法基于人体运动捕捉原始数据,使得虚拟人体与环境进行实时交互运动的过程更加真实。仿真结果以每帧的三维图像表示,可以形象地显示每秒60帧的人体运动。