Telerobotic systems design based on real‐time CORBA

A new class of telerobotic applications is making its way into research laboratories, fine arts or science museums, and industrial installations. Virtual laboratories and remote equipment maintenance are examples of these applications, which are built exploiting distributed computing systems and Internet technologies. Distributed computing technologies provide several advantages to telerobotic applications, such as dynamic and multiuser access to remote resources and arbitrary user locations. Nonetheless, building these applications remains a substantial endeavor, especially when performance requirements must be met. The aim of this paper is to investigate how mainstream and advanced features of the CORBA object‐oriented middleware can be put to work to meet the requirements of novel telerobotic applications. We show that Real‐Time CORBA extensions and asynchronous method invocation of CORBA services can be relied upon to meet performance and functional requirements, thereby enabling teleoperation on local area networks. Furthermore, CORBA services for concurrency control and large‐scale data distribution enable geographic‐scale access for robot teleprogramming. Limitations in the currently available implementations of the CORBA standard are also discussed, along with their implications. The effectiveness and suitability for telerobotic applications of several CORBA mechanisms are tested first individually and then by means of a software framework exploiting CORBA services and ensuring component‐based development, software reuse, low development cost, fully portable real‐time and communication support. A comprehensive telerobotic application built based on the framework is described in the paper and evaluated on both local and wide area networks. The application includes a robot manipulator and several sensory subsystems under concurrent access by multiple competing or collaborating operators, one of which is equipped with a multimodal user interface acting as the master device. © 2005 Wiley Periodicals, Inc.     

Internet-Based Robotic System Using CORBA as Communication Architecture

In this paper, we propose the Internet-based robotic system that uses Common Object Request Broker Architecture (CORBA) to implement networking connections between a client and a remote robotic system. The client can transparently invoke a method on a server across the network without any need to know where the application servers are located, or what programming language and operating system are used. This lets the system overcome the shortcomings of the other typical Internet robotic system. To cope with time delays on the communication path, we have implemented the robot control server, which allows the user to control the telerobotic system at a task-level. We have also implemented the live image feedback server, which provides live image feedback for a remote user. The proposed system gives the users the ability to operate the remote robotic system to retrieve and manipulate the desired tableware or other things to support the aged and disabled over the Internet by using the intuitive user interface.     

Situation awareness and attention allocation measures for quantifying telepresence experiences in teleoperation

This study assessed the utility of measures of situation awareness (SA) and attention allocation for quantifying telepresence, the sense of being present at a remote site, in a teleoperation task scenario. Attention and SA have been identified as cognitive constructs potentially underlying telepresence. The motivation for this research was to establish an objective measure of telepresence and investigate the relationship between telepresence and teleoperation performance. Twenty‐four research participants performed a virtual ordnance disposal task at varying levels of difficulty (LODs). The task involved locating, identifying, and disposing of virtual land mines in an outdoor environment using a simulated remote‐control rover with a robotic arm. Performance, SA, and attention allocation were recorded along with subjective assessments of telepresence. Results demonstrated LOD effects on performance and telepresence. Regression analysis revealed LOD and attention to explain significant portions of the variance in telepresence. Results of the study provide further evidence that telepresence may share a relationship with performance, and that cognitive constructs, such as attention and SA, may serve as alternative, objective measures of telepresence. © 2004 Wiley Periodicals, Inc. Hum Factors Man 14: 51–67, 2004.     

基于神经网络的力觉临场感系统的预测控制

针对力觉临场感系统传输通道中存在时交通讯时廷，造成系统不稳定和操作性能降低的问题，利用前向神经网络建立主机械手、从机械手和环境的模型，并通过神经网络模型预测主机械手速度和从机械手受力，以消除或减少通讯时廷对系统的影响。实验结果表明了该方法的有效性。     

利用远程网络技术的机器人遥操作系统分析

本文系统地讨论了如何构建一个基于远程网络（Int ernet）技术的机器人遥操作系统．通过描述此系统,指出了构建系统中所遇到的问题,并 提出了解决此问题的基本策略与方法．     

Adaptive Control for State Synchronization of Nonlinear Haptic Telerobotic Systems with Asymmetric Varying Time Delays

In this paper, we introduce a new adaptive controller design scheme for nonlinear telerobotic systems with varying time delays where the delays and their variation rates are unknown. The designed controller has the ability to synchronize the state behaviors of the local and the remote robots. In this paper, asymptotic stability in the presence of varying time delays is of interest. Using the proposed controller, asymptotic stability of the bilateral telerobotic system subject to any bounded yet unknown varying delay with a bounded yet unknown rate of change can be guaranteed. Besides the varying time delay, the proposed adaptive controller has the ability to adapt to the parameter variations in the local and the remote robots’ dynamics. It is shown that position and velocity errors between the local and the remote manipulators converge to the zero asymptotically, thus ensuring teleoperation transparency. Experimental and simulation results with a pair of PHANToM haptic devices and a pair of planar manipulators under varying time delays in the communication channel demonstrate the effectiveness of the proposed scheme.     

A telepresence mobile robot controlled with a noninvasive brain-computer interface

This paper reports an electroencephalogram-based brain-actuated telepresence system to provide a user with presence in remote environments through a mobile robot, with access to the Internet. This system relies on a P300-based brain-computer interface (BCI) and a mobile robot with autonomous navigation and camera orientation capabilities. The shared-control strategy is built by the BCI decoding of task-related orders (selection of visible target destinations or exploration areas), which can be autonomously executed by the robot. The system was evaluated using five healthy participants in two consecutive steps: 1) screening and training of participants and 2) preestablished navigation and visual exploration telepresence tasks. On the basis of the results, the following evaluation studies are reported: 1) technical evaluation of the device and its main functionalities and 2) the users' behavior study. The overall result was that all participants were able to complete the designed tasks, reporting no failures, which shows the robustness of the system and its feasibility to solve tasks in real settings where joint navigation and visual exploration were needed. Furthermore, the participants showed great adaptation to the telepresence system.     

Dynamic virtual environment to test teleoperated systems with time delay communications

This paper presents a dynamic virtual environment tool for training operators and to prove different control schemes in telerobotic systems, and describes virtual reality environments used in teleoperated robotic systems. In the presented tool, the kinematic and dynamic model of the remote environment which is manipulating the operator is considered. The paper also describes how time delays in the communication channel can be easily added to the simulator, in order to analyze their effects in the teleoperated system. Finally, some experimental results achieved with this virtual teleoperated system are shown. With the presented dynamic simulator, different control schemes designed to overcome the time delay problem could be tested. © 2005 Wiley Periodicals, Inc.     

A Neural Net Predictive Control for Telerobots with Time Delay

This paper extends the Smith Predictor feedback control structure to unknown robotic systems in a rigorous fashion. A new recurrent neural net predictive control (RNNPC) strategy is proposed to deal with input and feedback time delays in telerobotic systems. The proposed control structure consists of a local linearized subsystem and a remote predictive controller. In the local linearized subsystem, a recurrent neural network (RNN) with on-line weight tuning algorithm is employed to approximate the dynamics of the time-delay-free nonlinear plant. The remote controller is a modified Smith predictor for the local linearized subsystem which provides prediction and maintains the desirable tracking performance. Stability analysis is given in the sense of Lyapunov. The result is an adaptive compensation scheme for unknown telerobotic systems with time delays, uncertainties, and external disturbances. A simulation of a two-link robotic manipulator is provided to illustrate the effectiveness of the proposed control strategy.     

VR-Based Teleoperation for Robot Compliance Control

Robots governed by remote human operators are excellent candidates for work in hazardous or uncertain environments such as nuclear plants or outer space. For successful teleoperation, it is important to let the operator feel physically present at the remote site. When the telerobotic system is used to execute compliance tasks in which simultaneous control of both position and force may be demanded and inevitable contact with environments is encountered, information about the interactions between the robot manipulator and the environment are especially crucial for the operator to make proper decisions. This paper proposes a VR-based telerobotic system for such compliance tasks. The proposed system provides both visual and haptic information. A local intelligence controller, capable of surface tracking and force regulation, is equipped on the robot manipulator to tackle the time-delay problem usually present in teleoperation and to share control load with the operator. The proposed telerobotic system is developed in a virtual environment due to recent gains in the capabilities and popularity of virtual reality to generate realistic telepresence. Experiments based on the surface-tracking and peg-in-hole compliance tasks demonstrate the effectiveness of the proposed system.     

Stereoscopic human interfaces

This article focuses on the use of stereoscopic video interfaces for telerobotics. Topics concerning human visual perception, binocular image capturing, and stereoscopic devices are described. There is a wide variety of video interfaces for telerobotic systems [1]. Choosing the best video interface depends on the telerobotic application requirements. Simple monoscopic cameras are good enough for watching remote robot movements or for teleprogramming a sequence of commands. However, when operators seek precise robot guidance or wish to manipulate objects, a better perception of the remote environment must be achieved, for which more advanced visual interfaces are required. This implies a higher degree of telepresence, and, therefore, the most suitable visual interface has to be chosen. The aim of this article is to describe the two main aspects using stereoscopic interfaces: · the capture of binocular video images, according to the disparity limits in human perception · the proper selection of the visualization interface for stereoscopic images.     

Distributed robotics over the Internet

We present a communication framework to enable control and collaboration between multiple users over the Internet. We first discuss standard Internet protocols and extensions known as middleware and technologies in the context of Internet telerobotics. A protocol and framework suitable for collaborative telerobotic control are then introduced and discussed. Finally, an example of how the framework might be used for a simple telerobotic system is presented. The system has been tested locally but is not yet freely available on the Internet     

A tele‐experiment on rover motor control via internet

Modern Internet and multimedia techniques enable experiments with remote hardware. This way engineering students can gather practical experience even in areas where their own university does not have the related equipment in place. The focus of this paper is on an experiment to develop a motor controller for a remote mobile robot. Implementation details of such an experiment are addressed with emphasis placed on the subject of tuning a PID controller for steering and speed adaptation of a rover. The complete experiment is conceived as a self‐explaining learning unit on the web. The learning unit comprises material on theory, exercises to provide feedback to the student about his learning progress, simulations to study motor control effects, and finally the experiment with remote robot hardware. This paper presents software techniques to realize a robust system implementation, the educational considerations necessary to present the experiment material properly for the tele‐education context, and experiences obtained during the use of this remote experiment in previously taught classes. © 2005 Wiley Periodicals, Inc.     

一种基于Internet 力觉临场感遥操作无源控制算法

将Internet技术引入到力觉临场感遥操作系统是目前一个具有广阔应用前景的研究方向。由此，针对基于Internet力觉临场感遥操作系统所面临的变时延稳定控制问题，提出了一种基于波积分传输的无源控制算法。经过仿真，证明本算法可保证系统的无源性，并获得较好的操作性。     

Scheduling communication in multithreaded programs: experimental results

When the critical path of a communication session between end points includes the actions of operating system kernels, there are attendant overheads. Along with other factors, such as functionality and flexibility, such overheads motivate and favor the implementation of communication protocols in user space. When implemented with threads, such protocols may hold the key to optimal communication performance and functionality. Based on implementations of reliable user‐space protocols supported by a threads framework, we focus on our experiences with internal threads' scheduling techniques and their potential impact on performance. We present scheduling strategies that enable threads to do both application‐level and communication‐related processing. With experiments performed on a Sun SPARC‐5 LAN environment, we show how different scheduling strategies yield different levels of application‐processing efficiency, communication latency and packet‐loss. This work forms part of a larger study on the implementation of multiple thread‐based protocols in a single address space, and the benefits of coupling protocols with applications. Copyright © 2005 John Wiley & Sons, Ltd.     

ViMediaNet: an emulation system for interactive multimedia based telepresence services

The telepresence service (TPS) leverages advanced communication, virtual reality, and cooperative web technologies to provide remote engaged users an immersive experience of being fully present, which are widely used in teleconferencing, remote surgery, and hazardous environment exploration. The common feature of TPS-based applications lies in that the users’ experiences heavily depend on the quality of interactive multimedia services, that is, most deliveries of multimedia contents are mission-critical, as well as time-critical, to ensure the control and operation accuracy for telerobots. Therefore, the delivery performance of multimedia contents over heterogeneous networks becomes the primary concern in the TPS system design, especially for those delay sensitive applications, like remote surgery and environment exploration. In this paper, we incorporate video processing tools with the network simulator to design and implement a joint emulation system, named as the virtual media network (ViMediaNet), through which, the TPS designer can readily deploy the TPS system over a heterogeneous network virtualized by the network emulator, investigate the critical network performance metrics in the multimedia content delivery, such as transmission delay and packet jitter, and observe control results in real time. To verify the capability of ViMediaNet in emulating the real-setting TPS system, we take a mobile device based vision navigation system as an example to evaluate performance impacts of the rate adaptation technology in the telerobot control. The experimental results reveal that the adaptive video streaming is a promising solution to significantly enhance the control accuracy of telerobots in TPS systems.     

Online Avatar Motion Adaptation to Morphologically-similar Spaces

In avatar-mediated telepresence systems, a similar environment is assumed for involved spaces, so that the avatar in a remote space can imitate the user's motion with proper semantic intention performed in a local space. For example, touching on the desk by the user should be reproduced by the avatar in the remote space to correctly convey the intended meaning. It is unlikely, however, that the two involved physical spaces are exactly the same in terms of the size of the room or the locations of the placed objects. Therefore, a naive mapping of the user's joint motion to the avatar will not create the semantically correct motion of the avatar in relation to the remote environment. Existing studies have addressed the problem of retargeting human motions to an avatar for telepresence applications. Few studies, however, have focused on retargeting continuous full-body motions such as locomotion and object interaction motions in a unified manner. In this paper, we propose a novel motion adaptation method that allows to generate the full-body motions of a human-like avatar on-the-fly in the remote space. The proposed method handles locomotion and object interaction motions as well as smooth transitions between them according to given user actions under the condition of a bijective environment mapping between morphologically-similar spaces. Our experiments show the effectiveness of the proposed method in generating plausible and semantically correct full-body motions of an avatar in room-scale space.     

Prediction-based methods for teleoperation across delayed networks

The remote nature of telepresence scenarios can be seen as a strongpoint and also as a weakness. Although it enables the remote control of robots in dangerous or inaccessible environments, it necessarily involves some kind of communication mechanism for the transmission of control signals. This communication mechanism necessarily involves adverse network effects such as delay. Three mechanisms aimed at improving the effects of network delay are presented in this paper: (1) Motion prediction to partially compensate for network delays, (2) force prediction to learn a local force model, thereby reducing dependency on delayed force signals, and (3) haptic data compression to reduce the required bandwidth of high frequency data. The utilized motion prediction scheme was shown to improve operator performance, but had no influence on operator immersion. The force prediction provided haptic feedback through synchronous forces from the local model, thereby stabilizing the control loop. The developed haptic data compression scheme reduced the number of packets sent across the network by 90%, while improving the quality of the haptic feedback.     

Human-centered control scheme for delayed bilateral teleoperation of mobile robots

Teleoperation task performance strongly depends on how well the human operator’s commands are executed. In this paper, we propose a control scheme for delayed bilateral teleoperation of mobile robots that considers user’s commands execution in order to achieve a high-performance teleoperation system in some important aspects like time to complete the task, safety, and operator dependence. We describe some evaluation metrics that allow us to address these aspects and a quantitative metric is proposed and incorporated in the control scheme to compensate wrong commands. A force feedback is applied to the master at the local site as a haptic cue. In addition, the system stability is analyzed taking into consideration the master and remote robot dynamic models and the asymmetric time-varying delays of the communication channel. Multiple human-in-the-loop simulations were carried out and the results of the evaluation metrics were discussed. Additionally, we present experiments where a user teleoperates a mobile robot via the Internet connection between Argentina and Italy.     

虚拟仿真技术在网络闭环控制中的应用

Internet上数据传输的不确定延时妨碍了远程被控对象和操作者之间迅捷而透明的交互,严重限制了网络远程控制系统的性能和应用。为了解决网络延时问题,提出一种基于虚拟仿真的网络三闭环控制结构。该方法依照开环系统实现闭环控制的思想,在客户端构造虚拟被控对象模型,使虚拟仿真系统与实际系统的运行状态相似甚至相同,同时又使操作者能依据虚拟仿真系统一端的运行情况决定下一步的控制指令,从而准确地对实际系统发出所需的控制指令,最终达到远程实时控制的目的。仿真结果证明了所提方法的有效性和可行性。