Control gain adaptation in virtual reality mediated human–telerobot interaction

The Internet connects millions of computers worldwide, and provides a new potential working environment for remote‐controlled telerobotic systems. The main limitation of using the Internet in this application is random delays between communicating nodes, which can cause disturbances in human–machine interaction and affect telepresence experiences. This is particularly important in systems integrating virtual reality technology to present interfaces. Telepresence, or the sense of presence in a remote environment, hypothetically is positively related to teleoperation task performance. This research evaluated the effect of constant and random network (communication) delays on remote‐controlled telerover performance, operator workload, and telepresence experiences. The research also assessed the effect of using a system gain adaptation algorithm to offset the negative impact of communication delays on the various response measures. It was expected that with gain adaptation, system stability, performance, and user telepresence experiences would improve with a corresponding decrease in workload. Results indicated that gain adaptation had a significant effect on the performance measures. The study demonstrated that gain adaptation could reduce deterioration in telepresence experiences and improve user performance in teleoperated and telerobotic control. © 2005 Wiley Periodicals, Inc. Hum Factors Man 15: 259–274, 2005.     

基于虚拟现实的临场感遥控作业系统的研究动向

本文阐述了虚拟现实技术在临场感遥控作业系统中的研究和应用背景、状况和 内容．综述了临场感遥控作业虚拟现实系统的最近进展．重点介绍了它在监控和协作控制、 劣质视觉反馈的遥作及具有通信时延的遥作方面的研究动向．指出临场感遥控作业虚拟现实 系统是今后临场感技术研究和发展的主潮流,并对我国今后在这方面的研究应遵循的基本原 则提出了建议．     

虚拟现实力觉临场感遥控作业系统的研究进展

本文阐述了虚拟现实技术在力觉临场感遥控作业系统中的研究和应用背景、状况和主要内容,综述了虚拟现实力觉临场遥控作业实系统的国内外最近研究进展,并介绍了目前研究亟需解决的关键技术,指出虚拟现实力觉临场感遥控作业系统的研究对临场感技术以至于非确定性环境下作业的第三代机器人的发展有着非常重要的理论意义和应用价值。     

Stereo Viewing and Virtual Reality Technologies in Mobile Robot Teleguide

The use of 3-D stereoscopic visualization may provide a user with higher comprehension of remote environments in teleoperation when compared with 2-D viewing, in particular, a higher perception of environment depth characteristics, spatial localization, remote ambient layout, faster system learning, and decision performance. Works in the paper have demonstrated how stereo vision contributes to the improvement of the perception of some depth cues, often for abstract tasks, while it is hard to find works addressing stereoscopic visualization in mobile robot teleguide applications. This paper intends to contribute to this aspect by investigating the stereoscopic robot teleguide under different conditions, including typical navigation scenarios and the use of synthetic and real images. This paper also investigates how user performance may vary when employing different display technologies. Results from a set of test trials run on seven virtual reality systems, from laptop to large panorama and from head-mounted display to Cave automatic virtual environment (CAVE), emphasized few aspects that represent a base for further investigations as well as a guide when designing specific systems for telepresence.      

空间力觉临场感遥控作业系统的时延无条件稳定性分析

空间遥控作业系统中存在很长的通信时延,造成遥控作业系统的不稳定和操作性能的降低,为此,从建立力觉临场感遥控作业系统的时延动力学方程出发,利用差分微分方程对系统的无条件稳定性进行分析,实验结果表明该分析方法是行之有效的。     

A virtual environment-based system for the navigation of underwater robots

Efficient teleoperation of underwater robot requires clear 3D visual information of the robot's spatial location and its surrounding environment. However, the performance of existing telepresence systems is far from satisfactory. In this paper, we present our virtual telepresence system for assisting tele-operation of an underwater robot. This virtual environment-based telepresence system transforms robot sensor data into 3D synthetic visual information of the workplace based on its geometrical model. It provides the operators with a full perception of the robot's spatial location. In addition, we propose a robot safety domain to overcome the robot's location offset in the virtual environment caused by its sensor errors. The software design of the system and how a safety domain can be used to overcome robot location offset in virtual environment will be examined. Experimental tests and its result analysis will also be presented in this paper.     

Situation awareness and driving performance in a simulated navigation task

The objective of this study was to identify task and vehicle factors that may affect driver situation awareness (SA) and its relationship to performance, particularly in strategic (navigation) tasks. An experiment was conducted to assess the effects of in-vehicle navigation aids and reliability on driver SA and performance in a simulated navigation task. A total of 20 participants drove a virtual car and navigated a large virtual suburb. They were required to follow traffic signs and navigation directions from either a human aid via a mobile phone or an automated aid presented on a laptop. The navigation aids operated under three different levels of information reliability (100%, 80% and 60%). A control condition was used in which each aid presented a telemarketing survey and participants navigated using a map. Results revealed perfect navigation information generally improved driver SA and performance compared to unreliable navigation information and the control condition (task-irrelevant information). In-vehicle automation appears to mediate the relationship of driver SA to performance in terms of operational and strategic (navigation) behaviours. The findings of this work support consideration of driver SA in the design of future vehicle automation for navigation tasks.     

High-fidelity bilateral teleoperation systems and the effect of multimodal haptics.

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.     

VR-Based Teleoperation for Robot Compliance Control

Robots governed by remote human operators are excellent candidates for work in hazardous or uncertain environments such as nuclear plants or outer space. For successful teleoperation, it is important to let the operator feel physically present at the remote site. When the telerobotic system is used to execute compliance tasks in which simultaneous control of both position and force may be demanded and inevitable contact with environments is encountered, information about the interactions between the robot manipulator and the environment are especially crucial for the operator to make proper decisions. This paper proposes a VR-based telerobotic system for such compliance tasks. The proposed system provides both visual and haptic information. A local intelligence controller, capable of surface tracking and force regulation, is equipped on the robot manipulator to tackle the time-delay problem usually present in teleoperation and to share control load with the operator. The proposed telerobotic system is developed in a virtual environment due to recent gains in the capabilities and popularity of virtual reality to generate realistic telepresence. Experiments based on the surface-tracking and peg-in-hole compliance tasks demonstrate the effectiveness of the proposed system.     

Workload Management in Teleoperation of Unmanned Ground Vehicles: Effects of a Delay Compensation Aid on Human Operators’ Workload and Teleoperation Performance

ABSTRACT

Workload management is of critical concern in the teleoperation of unmanned vehicles because teleoperation is often employed in high-risk industries wherein high workload can lead to sub-optimal task performance and can harm human operators’ long-term well-being. This study aimed to assess the detrimental effects of time delays in teleoperation on operators’ workload and performance, and how a delay compensation aid mitigated such effects. We conducted a human-in-the-loop experiment with 36 participants using a dual-task teleoperation platform, where participants drove a simulated High Mobility Multipurpose Wheeled Vehicle (HMMWV) and performed a one-back memory task under three conditions: the delay condition, the delay with compensation aid condition, and the ideal no delay condition. A model-free predictor was used as the compensation aid. Results indicate that with a time delay of 0.8-s participants’ workload increased and performance degraded significantly. Moreover, the model-free predictor mitigated the detrimental effects of time delay on workload and task performance. Our findings suggest that participants are more sensitive in their perceived workload compared to the objective and physiological measures of workload. In addition, without any delay compensation algorithms, continuous teleoperation may not be ideal for operations with long time delays.     

Design issues of mobile haptic interfaces

As a novel approach to force‐reflecting telepresence, the concept of mobile haptic interfaces (MHI) is presented. An MHI actively follows the locomotion of an operator, who is no longer bound to be stationary during teleoperation. Thus, operator locomotion can be used as an input for locomotion control of a real teleoperator or control of locomotion in a virtual environment while keeping the advantage of force‐reflection. The article focuses on basic design issues and presents a prototype MHI for haptic exploration of extended virtual environments. © 2003 Wiley Periodicals, Inc.     

The contribution of virtual reality to research on sensory feedback in remote control

Here we consider research on the kinds of sensory information most effective as feedback during remote control of machines, and the role of virtual reality and telepresence in that research. We argue that full automation is a distant goal and that remote control deserves continued attention and improvement. Visual feedback to controllers has developed in various ways but autostereoscopic displays have yet to be proven. Haptic force feedback, in both real and virtual settings, has been demonstrated to offer much to the remote control environment and has led to a greater understanding of the kinesthetic and cutaneous components of haptics, and their role in multimodal processes, such as sensory capture and integration. We suggest that many displays using primarily visual feedback would benefit from the addition of haptic information but that much is yet to be learned about optimizing such displays.     

Does situation awareness add to the validity of cognitive tests?

OBJECTIVE: Does adding situation awareness (SA) to a battery of cognitive tests improve prediction? BACKGROUND: Identifying variables that predict skilled performance in a complex task aids in understanding the nature of skill and also aids in the selection of operators to perform that task. SA is thought to be an important predictor of performance. SA is often thought to be based on underlying cognitive mechanisms. METHOD: Three performance measures taken from the Federal Aviation Administration's (FAA's) Air Traffic Scenarios Test, the low-fidelity simulation component of the FAA's controller selection battery, were used as criterion variables in a hierarchical regression. After predicting performance based on a battery of cognitive (e.g., intelligence, working memory, spatial memory) and noncognitive tests (e.g., cognitive style, personality, demographics), we added measures of SA. RESULTS: SA did provide increases in prediction, but only when measured with the Situation Present Assessment Method, an on-line query method. When the same questions were asked off line, SA did not enter the model in two cases and improved prediction by only 2% in the third. CONCLUSION: Thus, some measures of SA do show incremental validity, even against a backdrop of a large number of cognitive variables. APPLICATION: On-line measures of SA can be a worthwhile addition to standard batteries of tests used to predict performance in cognitively oriented industrial tasks.     

网络化制造环境下虚拟遥现驱动的制造资源建模研究

远程制造资源的虚拟遥现和仿真是实现化制造的核心技术之一。探讨了虚拟制造系统的组织过程，着重分析了任务驱动的网络化制造模式下分布式制造场所和设备的远程虚拟遥现和组建方法，采用B／S结构实现了远程制造资源的VRML建模及EAI控制，并建立了相应的实验模型。同时提出了远程制造资源能力评估的一些基本方法。     

Presence,workload and performance effects of synthetic environment design factors

There remains a limited understanding of factors in presence and its relation to performance. This research examined a range of synthetic environment (SE) design features (viewpoint, auditory cue type and visual background) suspected to influence presence, and evaluated differences in presence, workload and task performance caused by manipulations of the factors and task difficulty in a virtual-reality-based basketball free-throw task. Thirty-two research participants were also required to perform secondary-monitoring tasks to assess attention allocation to the virtual and (surrounding) real environments, as an indicator of presence. Analysis of variance results demonstrated immersiveness (viewpoint) and auditory cue type to significantly influence the sense of subjective presence and perceptions of workload. Virtual task performance was significantly affected by task difficultly. This study also provided further evidence of significant positive relations between presence and workload, but no evidence of a correlation of objective presence and performance. These results have general applicability for the design of multimodal SE-based interfaces for real-world tasks, such as telerobot control.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Innovative Entrepreneurial Teams: The Give and Take of Trust and Conflict

Entrepreneurship research lacks empirical evidence about interactions between entrepreneurial team members . This paper examines the role of trust ( cognitive and affective ) and conflict ( task and relationship ) on team performance ( effectiveness and efficiency ) of innovative entrepreneurial teams . Data originated from 88 incubator‐based entrepreneurial teams in Austria . Results indicate that cognitive trust is the cornerstone of innovative entrepreneurial team performance . To maximize efficiency , such teams would rely on high cognitive trust and low task conflict . Nonetheless , the guidelines for being effective centre around high cognitive trust coupled with low task and relationship conflict .     

Effects of haptic feedback,stereoscopy, and image resolution on performance and presence in remote navigation

Traditionally, the main goal of teleoperation has been to successfully achieve a given task as if performing the task in local space. An emerging and related requirement is to also match the subjective sensation or the user experience of the remote environment, while maintaining reasonable task performance. This concept is often called “presence” or “(experiential) telepresence,” which is informally defined as “the sense of being in a mediated environment.” In this paper, haptic feedback is considered as an important element for providing improved presence and reasonable task performance in remote navigation. An approach for using haptic information to “experientially” teleoperate a mobile robot is described. Haptic feedback is computed from the range information obtained by a sonar array attached to the robot, and delivered to a user's hand via a haptic probe. This haptic feedback is provided to the user, in addition to stereoscopic images from a forward-facing stereo camera mounted on the mobile robot. The experiment with a user population in a real-world environment showed that haptic feedback significantly improved both task performance and user-felt presence. When considering user-felt presence, no interaction among haptic feedback, image resolution, and stereoscopy was observed, that is, haptic feedback was effective, regardless of the fidelity of visual elements. Stereoscopic images also significantly improved both task performance and user-felt presence, but high-resolution images only significantly improved user-felt presence. When considering task performance, however, it was found that there was an interaction between haptic feedback and stereoscopy, that is, stereoscopic images were only effective when no force feedback was applied. According to the multiple regression analysis, haptic feedback was a higher contributing factor to the improvement in performance and presence than image resolution and stereoscopy.