Development and Evaluation of a Virtual Reality Training System Based on Cognitive Task Analysis: The Case of CNC Tool Length Offsetting

This article reports on the development and evaluation of a virtual reality training system (VRTS) for a specific machining task. A cognitive task analysis of expert machinists was conducted to examine whether this can be effective in developing a VRTS concerning tool length offsetting for a machining center. This analysis provided the necessary information for development and calibration of such a system. Subsequently, the effectiveness of the VRTS was evaluated by conducting an experiment with 29 mechanical engineering students. The VRTS set‐up comprised a video projection of the machining center and a physical mock‐up of its interface. The system demonstrated positive training transfer for the toll length offsetting task in terms of task accomplishment and of time to complete the task. No positive transfer was observed in terms of task accuracy, probably due to perceptual biases induced by the detailed specification of the VRTS. The present work provides evidence that cognitive task analysis was effective in identifying a number of key skills pertaining to the tool length offsetting task and in implementing ways to facilitate training in such tasks in a virtual environment. This article also demonstrates that even for tasks that include subtle perceptual skills VRTS may be beneficial regardless of the level of physical fidelity, provided that the cognitive organization of a task is adequately mapped in the system.     

配电线路维护机器人虚拟现实仿真系统设计

针对配电线路维护机器人遥显示和在线作业轨迹规划的应用需要,设计了一套机器人虚拟现实三维仿真系统.通过将机械臂姿态信息实时传输到虚拟现实环境中,实现了作业场景3D遥显示功能;根据机器人带电作业需求,设计了一种基于虚拟现实环境的机械臂轨迹规划运动学仿真软件,可在三维虚拟空间测试机械臂作业路径的可行性,具有直观、安全和高效的...     

视触觉融合的增强现实三维注册方法

增强现实技术是近年来人机交互领域的研究热点。在增强现实环境下加入触觉感知,可使用户在真实场景中看到并感知到虚拟对象。为了实现增强现实环境下与虚拟对象之间更加自然的交互,提出一种视触觉融合的三维注册方法。基于图像视觉技术获得三维注册矩阵;借助空间转换关系求解出触觉空间与图像空间的转换关系;结合两者与摄像头空间的关系实现视触觉融合的增强现实交互场景。为验证该方法的有效性,设计了一个基于视触觉增强现实的组装机器人项目。用户可触摸并移动真实环境中的机器人零件,还能在触摸时感受到反馈力,使交互更具真实感。     

Object interaction and task programming by demonstration in visuo-haptic augmented reality

A visuo-haptic augmented reality system is presented for object manipulation and task learning from human demonstration. The proposed system consists of a desktop augmented reality setup where users operate a haptic device for object interaction. Users of the haptic device are not co-located with the environment where real objects are present. A three degrees of freedom haptic device, providing force feedback, is adopted for object interaction by pushing, selection, translation and rotation. The system also supports physics-based animation of rigid bodies. Virtual objects are simulated in a physically plausible manner and seem to coexist with real objects in the augmented reality space. Algorithms for calibration, object recognition, registration and haptic rendering have been developed. Automatic model-based object recognition and registration are performed from 3D range data acquired by a moving laser scanner mounted on a robot arm. Several experiments have been performed to evaluate the augmented reality system in both single-user and collaborative tasks. Moreover, the potential of the system for programming robot manipulation tasks by demonstration is investigated. Experiments show that a precedence graph, encoding the sequential structure of the task, can be successfully extracted from multiple user demonstrations and that the learned task can be executed by a robot system.     

虚拟现实技术下分拣机器人嵌入式遥控系统设计

为了解决当前遥控系统控制分拣机器人执行既定任务时存在目标轨迹跟踪效果不佳、跟踪误差较大,任务执行成功率较低、失误率较高以及系统响应延迟时间较长等缺点,提出并设计了基于虚拟现实技术的分拣机器人嵌入式遥控系统,研究通过分析虚拟现实系统组成结构基础上,将虚拟现实技术与分拣机器人技术有机结合,依据模块化、标准化、开放性和可用性原则,利用虚拟现实技术设计了具有临场感的操作指令输入输子系统和分拣机器人具有真实感的虚拟场景仿真子系统的嵌入式遥控系统;并给出了嵌入式遥控系统的基本功能单元和具体操作流程;依据该流程采用QNX Neutrino系统作为分拣机器人本体控制器,同时采用工业现场总线EtherCAT作为系统网络通信支撑,设计了嵌入式遥控系统软件控制程序,完成了基于虚拟现实技术的分拣机器人嵌入式遥控系统设计。模拟实验结果验证了设计系统的有效性,获得了较好的目标跟踪效果,减小了跟踪误差,提高了分拣机器人任务执行成功率,同时提高了系统效率。     

虚拟现实技术在柔性上肢康复机器人中的应用

根据末端牵引式和外骨骼式上肢康复机器人的特性,研制了一种新型的柔性上肢康复机器人。机器人的康复训练系统结合了主动和被动模式的要素,将虚拟现实（VR）技术引入上肢康复机器人,通过现实环境中的光学3D位置捕获以及VR环境中的3D位置感知,设计了虚拟动态模型交互性节点和碰撞检测实验,实现虚拟现实交互,提高虚拟模型运动实时效果和上肢康复训练精度。VR和新颖的康复机器人的集成为具有特定任务的患者提供了有效的训练。     

柔索牵引式力觉交互机器人控制策略

为了让航天员在没有太空真实环境的地面上模拟太空环境进行虚拟作业训练,设计了一种与虚拟现实（VR）技术相结合的柔索牵引式力觉交互机器人．首先,根据微重力环境中物体的运动特性设计机器人的构型,建立移动平台、驱动单元、人推物体运动过程的动力学模型并进行运动学分析．然后,针对系统冗余驱动及力控制任务,提出一种复合控制策略,即以柔索长度变化为速度控制内环,力的外环控制为力／速混合控制．最后,分别进行单柔索加载和人机系统力觉交互仿真分析,分析结果表明该控制策略可以使柔索驱动单元降低10%的恒力跟随误差并能稳定地跟随余弦力的变化,验证了该控制策略对多余力抑制的有效性．     

Human responses to augmented virtual scaffolding models

This study investigated the effect of adding real planks, in virtual scaffolding models of elevation, on human performance in a surround-screen virtual reality (SSVR) system. Twenty-four construction workers and 24 inexperienced controls performed walking tasks on real and virtual planks at three virtual heights (0, 6 m, 12 m) and two scaffolding-platform-width conditions (30, 60 cm). Gait patterns, walking instability measurements and cardiovascular reactivity were assessed. The results showed differences in human responses to real vs. virtual planks in walking patterns, instability score and heart-rate inter-beat intervals; it appeared that adding real planks in the SSVR virtual scaffolding model enhanced the quality of SSVR as a human - environment interface research tool. In addition, there were significant differences in performance between construction workers and the control group. The inexperienced participants were more unstable as compared to construction workers. Both groups increased their stride length with repetitions of the task, indicating a possibly confidence- or habit-related learning effect. The practical implications of this study are in the adoption of augmented virtual models of elevated construction environments for injury prevention research, and the development of programme for balance-control training to reduce the risk of falls at elevation before workers enter a construction job.     

基于虚拟现实的机器人离线编程技术的研究

介绍了一个基于虚拟现实技术的机器人离线编程系统.对系统结构作了描述,讨 论了机器人虚拟仿真环境建模技术、基于虚拟现实设备的机器人任务生成技术、碰撞检测算 法.该系统建立了基于虚拟现实技术的高级人机交互接口,使机器人离线编程更加快速、方便 和直观.     

Human responses to augmented virtual scaffolding models

This study investigated the effect of adding real planks, in virtual scaffolding models of elevation, on human performance in a surround-screen virtual reality (SSVR) system. Twenty-four construction workers and 24 inexperienced controls performed walking tasks on real and virtual planks at three virtual heights (0, 6 m, 12 m) and two scaffolding-platform-width conditions (30, 60 cm). Gait patterns, walking instability measurements and cardiovascular reactivity were assessed. The results showed differences in human responses to real vs. virtual planks in walking patterns, instability score and heart-rate inter-beat intervals; it appeared that adding real planks in the SSVR virtual scaffolding model enhanced the quality of SSVR as a human – environment interface research tool. In addition, there were significant differences in performance between construction workers and the control group. The inexperienced participants were more unstable as compared to construction workers. Both groups increased their stride length with repetitions of the task, indicating a possibly confidence- or habit-related learning effect. The practical implications of this study are in the adoption of augmented virtual models of elevated construction environments for injury prevention research, and the development of programme for balance-control training to reduce the risk of falls at elevation before workers enter a construction job.     

Agent-customized training for human learning performance enhancement

Training individuals from diverse backgrounds and in changing environments requires customized training approaches that align with the individual learning styles and ever-evolving organizational needs. Scaffolding is a well-established instructional approach that facilitates learning by incrementally removing training aids as the learner progresses. By combining multiple training aids (i.e. multimodal interfaces), a trainer, either human or virtual, must make real-time decisions about which aids to remove throughout the training scenario. A significant problem occurs in implementing scaffolding techniques since the speed and choice of removing training aids must be strongly correlated to the individual traits of a specific trainee. We detail an agent-based infrastructure that supports the customization of scaffolding routines as triggered by the performance of the trainee. The motivation for this agent-based approach is for integration into a training environment that leverages augmented reality (AR) technologies. Initial experiments using the simulated environment have compared the proposed adaptive approach with traditional static training routines. Results show that the proposed approach increases the trainees’ task familiarity and speed with negligible introduction of errors.     

Enhancing human indoor cognitive map development and wayfinding performance with immersive augmented reality-based navigation systems

Augmented reality (AR) is an interactive experience where computer-generated perceptual information is superimposed into the real-world environment. Most existing research in AR-based wayfinding has focused on the technological aspects of developing AR-based software or devices to realize navigation. No previous investigations have focused on understanding the impact of immersive augmented reality (IAR)–based systems on human wayfinding performance from the cognitive perspective. Aimed at investigating the influence of IAR-based systems on people’s cognitive map development and their subsequent wayfinding performance as well as the effect of using three-dimensional (3D) layout models in IAR environments in addition to superimposed guideposts, an experiment was carried out in a building with a complex floor plan. A total of 54 university students were evenly divided into three groups: a control group with no IAR assistance, a second group using an IAR-based navigation system that includes only superimposed guideposts, and a third group using an IAR-based navigation system that includes both guideposts and a 3D layout model. Each participant was asked to conduct a spatial exploration task in the environment, sketch a floor map based on their spatial cognition, and perform a wayfinding task to find eight specific locations in the building. An analysis of the participants’ performance and responses to a number of self-evaluation questionnaires collected in the experiment indicates that IAR technology can help people develop their cognitive maps more effectively and can substantially improve their wayfinding performance with a much lower workload. A second finding is that adding a 3D layout model can enhance the effect of an IAR-based navigation system in terms of cognitive map development. The findings from this research extend the existing knowledge about IAR-based navigation and further verify that AR technology has the potential to reduce human workload for cognitive tasks. The results also could support its more effective application in various scenarios that require assisted wayfinding and cognitive map training, such as emergency evacuation drills.     

Modelling and design of an augmented reality differential drive mobility aid in an enabled environment

In this paper the authors discuss the modelling and design of an augmented reality platform for disabled wheeled mobility studies. This design consists of a virtual environment with two degrees of freedom motion platform and integrated ground contact force feedback. Differential drive mobility users continue to be in touch with reality on their own mobility aid while interacting with virtual objects. The major domain related to the differential drive mobility of the disabled members of society which coincides with the use of manual wheelchairs, electric wheelchairs and mobility scooters. In order to account for environmental and dynamic effects, the wheeled mobility user needs to map the intended trajectory into the virtual world. Motion and inertia force feedback produced on the augmented simulator give the users a haptic sensory stimulus input regarding spatial movement and ground contact forces. The main objective is to model and design an augmented reality platform with real world kinematic and dynamic properties to place a wheeled mobility user closer to real world encounters. The use of the designed augmented reality platform will be beneficial to disabled wheeled mobility users in need of occupational therapist training and evaluation.     

Egocentric depth judgments in optical, see-through augmented reality

A fundamental problem in optical, see-through augmented reality (AR) is characterizing how it affects the perception of spatial layout and depth. This problem is important because AR system developers need to both place graphics in arbitrary spatial relationships with real-world objects, and to know that users would perceive them in the same relationships. Furthermore, AR makes possible enhanced perceptual techniques that have no real-world equivalent, such as X-ray vision, where AR users are supposed to perceive graphics as being located behind opaque surfaces. This paper reviews and discusses protocols for measuring egocentric depth judgments in both virtual and augmented environments, and discusses the well-known problem of depth underestimation in virtual environments. It then describes two experiments that measured egocentric depth judgments in AR. Experiment I used a perceptual matching protocol to measure AR depth judgments at medium and far-field distances of 5 to 45 meters. The experiment studied the effects of upper versus lower visual field location, the X-ray vision condition, and practice on the task. The experimental findings include evidence for a switch in bias, from underestimating to overestimating the distance of AR-presented graphics, at ~ 23 meters, as well as a quantification of how much more difficult the X-ray vision condition makes the task. Experiment II used blind walking and verbal report protocols to measure AR depth judgments at distances of 3 to 7 meters. The experiment examined real-world objects, real-world objects seen through the AR display, virtual objects, and combined real and virtual objects. The results give evidence that the egocentric depth of AR objects is underestimated at these distances, but to a lesser degree than has previously been found for most virtual reality environments. The results are consistent with previous studies that have implicated a restricted field-of-view, combined with an inability for observers to sc     

Transfer of training from virtual reality

This experiment compared the value of real-world training, virtual reality training, and no training in the transfer of learning to the same task performed in real-world conditions. Results provide no evidence of transfer from a virtual reality training environment to a real-world task. There was no significant difference between the virtual reality training group and the group that received no training on the task. The group that received real-world training performed significantly better than both of the other two groups. The results question the utility of virtual training and suggest that in the present configuration, individuals learn performance characteristics specific only to the virtual reality context. Needed improvements to virtual reality for the purpose of enabling the transfer of training are indicated.     

Establishing the Usability of a Virtual Training System for Assembly Operations within the Automotive Industry

Virtual training systems deliver training within a virtual environment (VE) using virtual reality (VR) or augmented reality (AR) technologies. However, to be fully accepted as a valid tool for training within the automotive industry, evidence is required on the ability of these systems to deliver effective and efficient training to the relevant users. This paper aims to investigate the effectiveness and efficiency of the first prototype of the virtual training system (VTS) developed within the VISTRA (Virtual Simulation and Training of Assembly and Service Processes in Digital Factories) project (FP7‐ICT‐285176), using real end users from the OPEL automotive plant in Rüsselsheim, Germany. Two separate and independent studies were employed that used objective and subjective methods of investigation to establish performance and usability measures. The objective results show that virtual training was effective in reducing error during task performance when compared to traditional training. The subjective results concluded that the opinions of the participants were mainly positive concerning the overall use of the VTS for assembly operation training; however, a number of issues were highlighted and reported to the developers for further advancement of the system.     

虚拟手交互中约束类物体操作研究

虚拟手技术是虚拟现实技术中的一种交互方式，是指通过手势识别技术将人的真实手映射到虚拟环境中并与虚拟物体进行交互操作的人机交互方式。主要应用于虚拟训练、虚拟手术和虚拟装配等领域。对虚拟手交互的约束类对象进行了研究，提出了基于运动学的约束类物体操作方法，解决了阀门类有约束物体交互中虚拟手姿态不真实、操作不可控的问题。设计实验验证了基于运动学的方法能以符合实际操作的效果完成虚拟现实系统中的操作任务，可以应用到虚拟训练、虚拟装配等仿真平台。     

增强现实场景光源的实时检测方法和真实感渲染框架

首先通过检测视频定位标记表面辐照度的变化和真实光源之间的关系建立模型;然后在交互过程中迭代计算真实场景中光源的强度和方向,并将它集成到一个高效的基于场景管理的增强现实应用开发框架中．实验结果表明,使用该算法自动生成的虚拟光源可以近似地逼近真实场景的光照,在具有一个或多个真实光源的增强现实环境中使虚拟物体和真实物体能产生近似一致的光照效果．     

Training in virtual environments: transfer to real world tasks and equivalence to real task training

Virtual environments (VEs) are extensively used in training but there have been few rigorous scientific investigations of whether and how skills learned in a VE are transferred to the real world. This research aimed to measure and evaluate what is transferring from training a simple sensorimotor task in a VE to real world performance. In experiment 1, real world performances after virtual training, real training and no training were compared. Virtual and real training resulted in equivalent levels of post-training performance, both of which significantly exceeded task performance without training. Experiments 2 and 3 investigated whether virtual and real trained real world performances differed in their susceptibility to cognitive and motor interfering tasks (experiment 2) and in terms of spare attentional capacity to respond to stimuli and instructions which were not directly related to the task (experiment 3). The only significant difference found was that real task performance after training in a VE was less affected by concurrently performed interference tasks than was real task performance after training on the real task. This finding is discussed in terms of the cognitive load characteristics of virtual training. Virtual training therefore resulted in equivalent or even better real world performance than real training in this simple sensorimotor task, but this finding may not apply to other training tasks. Future research should be directed towards establishing a comprehensive knowledge of what is being transferred to real world performance in other tasks currently being trained in VEs and investigating the equivalence of virtual and real trained performances in these situations.     

Training in virtual environments: transfer to real world tasks and equivalence to real task training

Virtual environments (VEs) are extensively used in training but there have been few rigorous scientific investigations of whether and how skills learned in a VE are transferred to the real world. This research aimed to measure and evaluate what is transferring from training a simple sensorimotor task in a VE to real world performance. In experiment 1, real world performances after virtual training, real training and no training were compared. Virtual and real training resulted in equivalent levels of post-training performance, both of which significantly exceeded task performance without training. Experiments 2 and 3 investigated whether virtual and real trained real world performances differed in their susceptibility to cognitive and motor interfering tasks (experiment 2) and in terms of spare attentional capacity to respond to stimuli and instructions which were not directly related to the task (experiment 3). The only significant difference found was that real task performance after training in a VE was less affected by concurrently performed interference tasks than was real task performance after training on the real task. This finding is discussed in terms of the cognitive load characteristics of virtual training. Virtual training therefore resulted in equivalent or even better real world performance than real training in this simple sensorimotor task, but this finding may not apply to other training tasks. Future research should be directed towards establishing a comprehensive knowledge of what is being transferred to real world performance in other tasks currently being trained in VEs and investigating the equivalence of virtual and real trained performances in these situations.