Human-centered control scheme for delayed bilateral teleoperation of mobile robots

Teleoperation task performance strongly depends on how well the human operator’s commands are executed. In this paper, we propose a control scheme for delayed bilateral teleoperation of mobile robots that considers user’s commands execution in order to achieve a high-performance teleoperation system in some important aspects like time to complete the task, safety, and operator dependence. We describe some evaluation metrics that allow us to address these aspects and a quantitative metric is proposed and incorporated in the control scheme to compensate wrong commands. A force feedback is applied to the master at the local site as a haptic cue. In addition, the system stability is analyzed taking into consideration the master and remote robot dynamic models and the asymmetric time-varying delays of the communication channel. Multiple human-in-the-loop simulations were carried out and the results of the evaluation metrics were discussed. Additionally, we present experiments where a user teleoperates a mobile robot via the Internet connection between Argentina and Italy.     

Teleoperation of Mobile Robots With Time-Varying Delay

This paper proposes a stable control scheme for teleoperation of mobile robots with visual feedback in presence of time-varying delay. The proposed control scheme is based on using a model of the human operator to combine (on the remote site) the velocity command generated by the human operator in a delayed time instant, the received information (which stimulates the operator) in such moment, and the current state of the remote site to set the velocity reference, which is applied to the mobile robot. In addition, the proposed control scheme does not modify the information sent from the remote site to the local site. On the other hand, the proposed scheme uses estimated parameters of the human operator and a fictitious force, which is calculated using data fusion from ultrasonic sensors and optical flow field taken from panoramic images. To illustrate the performance and stability of the proposed control structure, several teleoperation experiments between Argentina and Brazil linked via Internet are shown.     

Passive Bilateral Teleoperation With Constant Time Delay

We propose a novel control framework for bilateral teleoperation of a pair of multi-degree-of-freedom nonlinear robotic systems under constant communication delays. The proposed framework uses the simple proportional-derivative control, i.e., the master and slave robots are directly connected via spring and damper over the delayed communication channels. Using the controller passivity concept, the Lyapunov–Krasovskii technique, and Parseval's identity, we can passify the combination of the delayed communication and control blocks altogether robustly, as long as the delays are finite constants and an upper bound for the round-trip delay is known. Having explicit position feedback through the delayed P-action, the proposed framework enforces master–slave position coordination, which is often compromised in the conventional scattering-based teleoperation. The proposed control framework provides humans with extended physiological proprioception, so that s/he can affect and sense the remote slave environments mainly relying on her/his musculoskeletal systems. Simulation and experiments are performed to validate and highlight properties of the proposed control framework.     

PD-like controller for delayed bilateral teleoperation of wheeled robots

This paper proposes a proportional derivative (PD)-like controller applied to the delayed bilateral teleoperation of wheeled robots with force feedback in face of asymmetric and varying-time delays. In contrast to bilateral teleoperation of manipulator robots, in these systems, there is a mismatch between the models of the master and slave (mobile robot), problem that is approached in this work, where the system stability is analysed. From this study, it is possible to infer the control parameters, depending on the time delay, necessary to assure stability. Finally, the performance of the delayed teleoperation system is evaluated through tests where a human operator drives a 3D simulator as well as a mobile robot for pushing objects.     

Teleoperation in the presence of varying time delays and sandwich linearity in actuators

In this paper, a novel control scheme is proposed to guarantee global asymptotic stability of bilateral teleoperation systems that are subjected to time-varying time delays in their communication channel and sandwich linearity in their actuators. This extends prior art concerning control of nonlinear bilateral teleoperation systems under time-varying time delays to the case where the local and the remote robots’ control signals pass through saturation or similar nonlinearities that belong to a class of systems we name sandwich linear systems. Our proposed controller is similar to the proportional plus damping (P+D) controller with the difference that it takes into account the actuator saturation at the outset of control design and alters the proportional term by passing it through a nonlinear function; thus, we call the proposed method as nonlinear proportional plus damping (nP+D). The asymptotic stability of the closed-loop system is established using a Lyapunov–Krasovskii functional under conditions on the controller parameters, the actuator saturation characteristics, and the maximum values of the time-varying time delays. To show the effectiveness of the proposed method, it is simulated on a variable-delay teleoperation system comprising a pair of planar 2-DOF robots subjected to actuator saturation. Furthermore, the controller is experimentally validated on a pair of 3-DOF PHANToM Premium 1.5A robots, which have limited actuation capacity, that form a teleoperation system with a varying-delay communication channel.     

Adaptive passive control with varying time delay

This paper presents a passive control scheme for a force reflecting bilateral teleoperation system with a varying time communication delay. To improve the stability and performance of the system, the master and slave must be coupled dynamically via a transmission network through which the force and velocity are communicated bilaterally. However, the time delay caused by various factors, such as the transmission distance, network congestion, and communication bandwidth, is a long-standing impediment to bilateral control that can destabilize the system. In this study, we investigated how a varying time delay affects the stability of a teleoperation system. A new optimal adaptive approach based on a passive control scheme was designed bilaterally for both the master and slave sites. Extra variables were transmitted together with the wave variables in the scattering system. The proposed scheme achieved both passive control, and an acceptable tracking performance. The tracking performance was demonstrated using a computer simulation of varying time delays in a bilateral teleoperation system.     

Experimental Comparison Study of Control Architectures for Bilateral Teleoperators

A detailed experimental comparison study of several published algorithms for motion and force control of bilateral teleoperators, with emphasis on Internet-based teleoperation, is presented. The study investigates the effects of data losses, communication delays, and environmental constraints on a teleoperation system for different control techniques, which are based on wave variables, Smith predictors, and recent algorithms on synchronization. The controllers are compared on stability, transparency, and complexity using two identical nonlinear robots coupled via a stochastic network model that allowed transmission round-trip delays and data-loss rates to range from 8 to 1088 ms and 0% to 50%, respectively. A total of 18 subjects, which were distributed among 26 experiments with the aims of regulating the effects of the operators learning process and dynamic properties, participated in this study. Overall, the comparison study reports a deteriorating effect in the performance (i.e., larger position errors and lower fidelity of contact information) from delays and data losses. Yet, the effect of data losses is less critical when compared with time delays. In addition, the preference for a particular control framework is shown to strongly depend on the operational conditions of the system, such as the characteristics of the coupling channel, the specifics of the remote task, and the computational capabilities of the manipulators.      

Nonlinear bilateral teleoperators under event‐driven communication with constant time delays

Bilateral teleoperation systems provide a platform for human operators to remotely manipulate slave robots in engaging various tasks in remote environments. Most of the previous studies in bilateral teleoperation were developed under continuous transmission or periodic communication with fixed data exchanging rates. This paper presents control schemes for bilateral teleoperation systems using nonperiodic event‐driven communication. By using P‐like and PD‐like controllers, this study proposes triggering conditions for teleoperators to reduce network access frequency so that robots only transmit output signals when necessary. Stability and position tracking of the control system are studied, and nonzero minimum interevent time is guaranteed. The proposed event‐driven teleoperation is studied with a velocity estimator to avoid the requirement of velocity information in the controller and triggering condition. Without velocity measurements, the boundedness of tracking errors and stability are ensured for teleoperation systems under event‐driven communication. Simulations and experiments are illustrated to validate the performance of the proposed event‐driven teleoperation systems.     

Integrated adaptive robust control for multilateral teleoperation systems under arbitrary time delays

With the increasing industrial requirements such as bigger size object, stable operation, and complex task, multilateral teleoperation systems extended from traditional bilateral teleoperation are widely developed. In this paper, the integrated control design is developed for multilateral teleoperation systems, where n master manipulators are operated by human to remotely control n slave manipulators cooperatively handling a target object. For the first time, the control objectives of multilateral teleoperation including stability, synchronization, transparency, and internal force distribution are clarified systematically. A novel communication architecture is proposed to cope with communication delays, where the estimated environmental parameters are transmitted from the slave side to the master, to replace the traditional environmental force measurement in the communication channel. A kind of nonlinear adaptive robust control technique is used to deal with nonlinearities, unknown parameters, and modeling uncertainties existing in the master, slave, and environmental dynamics, so that the excellent tracking performance is achieved in both master and slave sides. The coordinated motion/force control is designed in the slave side by the optimal internal force distribution among n slave manipulators, and the impedance control is designed in the master side to realize the target transparency behavior. In summary, the proposed control algorithm can achieve the guaranteed robust stability, the excellent synchronization and transparency performance, and the optimal internal force distribution simultaneously for multilateral teleoperation systems under arbitrary time delays and various modeling uncertainties. The simulation is carried out on a 2‐master/2‐slave teleoperation system, and the results show the effectiveness of the proposed control design. Copyright © 2015 John Wiley & Sons, Ltd.     

Bilateral teleoperation of a group of mobile robots for cooperative tasks

A visual and force feedback-based teleoperation scheme is proposed for cooperative tasks. The bilateral teleoperation system includes a haptic device, an overhead camera and a group of wheeled robots. The commands of formation and average velocities of the multiple robots are generated by the operator through the haptic device. The state of the multiple robots and the working environment is sent to the human operator. The received information contains the feedback force through the haptic device and visual information returned by a depth camera. The feedback force based on the difference between the desired and actual average velocities is presented. The wave variable method is employed in the bilateral teleoperation of multiple mobile robots with time delays. The effectiveness of the bilateral teleoperation system is demonstrated by experiments. The robots in the slave side are able to follow the commands from the master side to interact with the environments, including moving in different formations and pushing a box. The results show that the scheme enables the operator to manipulate a group of robots to complete cooperative tasks freely.     

Semi-Autonomous Bilateral Teleoperation of Hexapod Robot Based on Haptic Force Feedback

With the widespread use of multi-legged robots in various applications, new challenges have arisen in terms of designing their control systems, one of which is posed by the multiple degrees of freedom of the robotic legs. This paper proposes a novel method for the bilateral teleoperation control of a hexapod robot by using a semi-autonomous strategy. In this teleoperation system, the body velocities of the slave robot and the displacements of the master robot are mapped to each other. The angular velocities of the joints of the legs rely on independent planning to achieve a horizontal movement. A controller is designed based on the difference between the expected velocity and the actual velocity of the body, and the difference is fed back to the operator in the form of haptic force. Therefore, the transparency of the control system is guaranteed by increasing the damping compensation both in the master and slave robots. In addition, the stability of the bilateral teleoperation control system of the hexapod robot is guaranteed by passivity theory, and the proposed method is verified by conducting semi-physical simulation experiments.     

High-fidelity bilateral teleoperation systems and the effect of multimodal haptics.

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.     

A Telepresence-Guaranteed Control Scheme for Teleoperation Applications of Transferring Weight-Unknown Objects

Currently, most teleoperation work is focusing on scenarios where slave robots interact with unknown environments. However, in some fields such as medical robots or rescue robots, the other typical teleoperation application is precise object transportation. Generally, the object’s weight is unknown yet essential for both accurate control of the slave robot and intuitive perception of the human operator. However, due to high cost and limited installation space, it is unreliable to employ a force sensor to directly measure the weight. Therefore, in this paper, a control scheme free of force sensor is proposed for teleoperation robots to transfer a weight-unknown object accurately. In this scheme, the workspace mapping between master and slave robot is firstly established, based on which, the operator can generate command trajectory on-line by operating the master robot. Then, a slave controller is designed to follow the master command closely and estimate the object’s weight rapidly, accurately and robust to unmodeled uncertainties. Finally, for the sake of telepresence, a master controller is designed to generate force feedback to reproduce the estimated weight of the object. In the end, comparative experiments show that the proposed scheme can achieve better control accuracy and telepresence, with accurate force feedback generated in only 500 ms.      

基于时域无源性控制的六足机器人双边触觉遥操作

随着六足机器人研究工作的深入,针对其遥操作系统的开发面临诸多挑战.为了弥补松软接触条件对系统可控性及稳定性的影响,提出一种基于时域无源性控制(time-domain passivity control,TDPC)的六足机器人双边触觉遥操作方法.其主从两端采取位置-速度的交互模式,通过分析足-地柔性接触的作用机理,构建无源观测器和无源控制律以补偿足底滑移所导致环境系统的潜在有源性,采用速度跟踪模式设计基于触觉力反馈的系统控制架构,并利用Llewellyn准则确定控制律参数的稳定范围.最后,搭建半物理仿真实验平台并验证所提出的双边触觉遥操作方法在松软地形条件下能够保证六足机器人遥操作系统的稳定,且兼具较好的持续跟踪能力.     

A Composite State Convergence Scheme for Bilateral Teleoperation Systems

State convergence is a novel control algorithm for bilateral teleoperation of robotic systems. First, it models the teleoperation system on state space and considers all the possible interactions between the master and slave systems. Second, it presents an elegant design procedure which requires a set of equations to be solved in order to compute the control gains of the bilateral loop. These design conditions are obtained by turning the master-slave error into an autonomous system and imposing the desired dynamic behavior of the teleoperation system. Resultantly, the convergence of master and slave states is achieved in a well-defined manner. The present study aims at achieving a similar convergence behavior offered by state convergence controller while reducing the number of variables sent across the communication channel. The proposal suggests transmitting composite master and slave variables instead of full master and slave states while keeping the operator’s force channel intact. We show that, with these composite and force variables; it is indeed possible to achieve the convergence of states in a desired way by strictly following the method of state convergence. The proposal leads to a reduced complexity state convergence algorithm which is termed as composite state convergence controller. In order to validate the proposed scheme in the absence and presence of communication time delays, MATLAB simulations and semi-real time experiments are performed on a single degree-of-freedom teleoperation system.      

Adaptive Control for State Synchronization of Nonlinear Haptic Telerobotic Systems with Asymmetric Varying Time Delays

In this paper, we introduce a new adaptive controller design scheme for nonlinear telerobotic systems with varying time delays where the delays and their variation rates are unknown. The designed controller has the ability to synchronize the state behaviors of the local and the remote robots. In this paper, asymptotic stability in the presence of varying time delays is of interest. Using the proposed controller, asymptotic stability of the bilateral telerobotic system subject to any bounded yet unknown varying delay with a bounded yet unknown rate of change can be guaranteed. Besides the varying time delay, the proposed adaptive controller has the ability to adapt to the parameter variations in the local and the remote robots’ dynamics. It is shown that position and velocity errors between the local and the remote manipulators converge to the zero asymptotically, thus ensuring teleoperation transparency. Experimental and simulation results with a pair of PHANToM haptic devices and a pair of planar manipulators under varying time delays in the communication channel demonstrate the effectiveness of the proposed scheme.     

A novel adaptive robust control architecture for bilateral teleoperation systems under time‐varying delays

Bilateral teleoperation technology has caused wide attentions due to its applications in various remote operation systems. The communication delay becomes one of the main challenging issues in the teleoperation control design. Meanwhile, various nonlinearities, parameter variations, and modeling uncertainties existing in manipulator and environment dynamics need to be considered carefully in order to achieve good control performance. In this paper, a globally stable nonlinear adaptive robust control algorithm is developed for bilateral teleoperation systems to deal with these control issues. Namely, the unknown dynamical parameters of the environmental force are estimated online by the improved least square adaptation law. A novel communication structure is proposed where only the master position signal is transmitted to the slave side for the tracking design, and the online estimators of the environmental parameters are transmitted from the slave to the master to replace the traditional environmental force measurement. Because the estimated environmental parameters are not power signals, the passivity problem of the communication channel and the trade‐off limitation between the transparency performance and robust stability in traditional teleoperation control are essentially avoided. The nonlinear adaptive robust control is subsequently developed to deal with nonlinearities, unknown parameters, and modeling uncertainties of the master, slave, and environmental dynamics, so that the guaranteed transient and steady‐state transparency performance can be achieved. The experiments on two voice‐coil motor‐driven manipulators are carried out, and the comparative results verify that the proposed control algorithm achieves the excellent control performance and the guaranteed robust stability simultaneously under time delays. Copyright © 2014 John Wiley & Sons, Ltd.     

Experimental comparison of backdrivability for time-delayed telerobotics

After stability, transparency is the major goal in teleoperation system design. This transparency goal of the overall system depends on the master/slave manipulator backdrivability. However, time delay in communication channel severely affects the backdrivability of a bilateral teleoperation system in practice. This study investigates the effects of communication delays on the backdrivability of a teleoperation system for wave-variable-based control techniques. The controllers are compared on position and force tracking performance using two identical linear robots coupled via network model that allowed random transmission round-trip delays. Overall, the comparison study reports a deteriorating effect in the system backdrivable performance (i.e., larger position errors and lower fidelity of contact information) from delays. In addition, wave-variable-based controller with position compensation is shown to make better system backdrivability.     

On tracking performance in bilateral teleoperation

This paper addresses the problem of steady-state position and force tracking in bilateral teleoperation. Passivity-based control schemes for bilateral teleoperation provide robust stability against network delays in the feedback loop and velocity tracking, but do not guarantee steady-state position and force tracking in general. Position drift due to data loss and offset of initial conditions is a well-known problem in such systems. In this paper, we introduce a new architecture, which builds upon the traditional passivity-based configuration by using additional position control on both the master and slave robots, to solve the steady-state position and force-tracking problem. Lyapunov stability methods are used to establish the range of the position control gains on the master and slave sides. Experimental results using a single-degree-of-freedom master/slave system are presented, showing the performance of the resulting system.     

预估控制下的实时网络遥操作移动机器人

构建了能使操作者通过Internet远程实时控制的移动机器人系统．为了补偿网络时延和抵消其对遥操作系统的影响,基于我们以前提出的改进型Smith预估器原理,采用了预估控制策略．为了保证系统稳定性和透明性,基于主从端的传感器信息交换,设计了一个动态模型管理器,其中模型和力反馈误差调节通过模糊控制实现．除了力反馈外,为了增强遥操作的实时性,引入了预估的虚拟显示．为了精确地预测网络时延,提出了一个新颖的时钟同步算法．为了降低时延抖动,结合我们提出的两个算法,实现了数据缓冲策略．最后,通过长距离的网络遥操作实验验证了系统和控制策略的实用性和有效性．