Nonlinear bilateral teleoperators with non-collocated remote controller over delayed network

Remote robots in bilateral teleoperation systems are utilized to accomplish various missions in different locations, which are generally far away from local robots collocated with a human operator. In order to enhance flexibility of teleoperators with extensive applicability, this paper proposed a novel control framework, where the remote controller is non-collocated with the robot in the environment. In contrast to traditional teleoperation systems, the remote robot only needs to send out sensory information and receive control commands from the local side. Stability and transparency of the proposed teleoperators are studied for PD-like controller with fixed time delays, and P-like controller with time-varying delays. If the control gains are contingent to upper bounds of time delays, then the system is stable with guaranteed position tracking and force reflection. Numerical simulations and experiments were conducted to demonstrate the effectiveness of the proposed control algorithms in bilateral teleoperation.     

Gain scheduler middleware: a methodology to enable existing controllers for networked control and teleoperation - part I: networked control

Conventionally, in order to control an application over a data network, a specific networked control or teleoperation algorithm to compensate network delay effects is usually required for controller design. Therefore, an existing controller has to be redesigned or replaced by a new controller system. This replacement process is usually costly, inconvenient, and time consuming. In this paper, a novel methodology to enable existing controllers for networked control and teleoperation by middleware is introduced. The proposed methodology uses middleware to modify the output of an existing controller based on a gain scheduling algorithm with respect to the current network traffic conditions. Since the existing controller can still be utilized, this approach could save much time and investment cost. Two examples of the middleware applied for networked control and teleoperation with IP network delays are given in these two companion papers. Part I of these two companion papers introduces the concept of the proposed middleware approach. Formulation, delay modeling, and optimal gain finding based on a cost function for a case study on DC motor speed control with a proportional-integral (PI) controller are also described. Simulation results of the PI controller shows that, with the existence of IP network delays, the middleware can effectively maintain the networked control system performance and stabilize the system. Part II of this paper will cover the use of the proposed middleware concept for a mobile robot teleoperation.     

End-to-end congestion control for the Internet: delays andstability

Under the assumption that queueing delays will eventually become small relative to propagation delays, we derive stability results for a fluid flow model of end-to-end Internet congestion control. The theoretical results of the paper are intended to be decentralized and locally implemented: each end system needs knowledge only of its own round-trip delay. Criteria for local stability and rate of convergence are completely characterized for a single resource, single user system. Stability criteria are also described for networks where all users share the same round-trip delay. Numerical experiments investigate extensions to more general networks. Through simulations, we are able to evaluate the relative importance of queueing delays and propagation delays on network stability. Finally, we suggest how these results may be used to design network resources     

Teleoperation of a robot manipulator using a vision-based human-robot interface

Remote teleoperation of a robot manipulator by a human operator is often necessary in unstructured dynamic environments when human presence at the robot site is undesirable. Mechanical and other contacting interfaces used in teleoperation require unnatural human motions for object manipulation tasks or they may hinder human motion. Previous vision-based approaches have used only a few degrees of freedom for hand motion and have required hand motions that are unnatural for object manipulation tasks. This paper presents a noncontacting vision-based method of robot teleoperation that allows a human operator to communicate simultaneous six-degree-of-freedom motion tasks to a robot manipulator by having the operator perform the three-dimensional human hand-arm motion that would naturally be used to complete an object manipulation task. A vision-based human-robot interface is used for communication of human motion to the robot and for feedback of the robot motion and environment to the human operator. Teleoperation under operator position control was performed with high accuracy in object placement on a target. Semi-autonomous traded and shared control using robot-vision guidance aided in achieving a more accurate positioning and orientation of the end-effector for object gripping tasks.     

Stochastic stabilization for bilateral teleoperation over networks with probabilistic delays

This paper studies the bilateral teleoperation over communication networks. Specifically, the network-induced random delays are modeled as being from a finite set, each delay in the set having a probability of occurrence. To fully utilize the stochastic information inherent with the delays, a novel design scheme combining the probability information and pole placement is proposed to achieve better tracking performance. The teleoperation problem is first formulated as the stabilization of an error dynamic system where the error is the difference between the states of the master and slave. Then, by constructing a Lyapunov function, a sufficient condition to guarantee the input-to-state stability is established in terms of linear matrix inequalities. The simulation results and comparison show a decrease in tracking error with the new design method.     

Design the Remote Control System With the Time-Delay Estimator and the Adaptive Smith Predictor

In real applications, a remote control system is generally an integration of different networks consisting of a commercial network for message transmission and an industrial network to control the remote hardware through a communication gateway. Since the induced time-delay in network control system (NCS) may cause system instability, this paper proposes a remote NCS structure by implementing the adaptive Smith predictor with an online time-delay estimator. As the delay in a commercial network Ethernet is significantly time-varying depending on the number of end-users, the delay is estimated in this paper by processing the online measurement of the round-trip time (RTT) between the application layers of the server and the client. The adaptive Smith predictor control scheme is developed by directly applying the estimated time-delay. To prove the feasibility of the proposed remote control system, the developed design has been applied to an AC 400 W servo motor tested from a 15 km distance. The experimental results indicate that the significantly improved stability and motion accuracy can be reliably achieved by applying the proposed approach.      

Experimental Analysis of an Internet-Based Bilateral Teleoperation System With Motion and Force Scaling Using a Model Predictive Controller

This paper presents an experimental investigation of a bilateral generalized predictive controller for scaled teleoperation systems subject to slave force feedback, variable transmission-time delays, and packet losses. The originality of the approach proposed mainly lies in its capacity to take into account explicitly the slave force feedback in the predictive algorithm. Experimental results show the system stability with the proposed approach using a frequency-domain technique. Several configurations of scaling factors have been used; it is shown that the stability conditions strongly depend of the external environment. Another experimental result verify the robust performances of the approach in terms of tracking behavior with both strong variations of time delays and packet losses in the communication network. For the experimentations, the communication network is the Internet using user-datagram protocol, while the slave robot is a 6-DOF anthropomorphic robot with a force sensor.      

Time signal distribution in communication networks based onsynchronous digital hierarchy

A new method that uses round-trip paths to accurately measure transmission delay for submicrosecond time synchronization in synchronous digital hierarchy (SDH) networks is proposed. Since the proposed system uses virtual container-3 (VC-3) signal as the time reference signals, existing SDH equipment can be used without any changes. The feature of this method is that it separately measures the initial round-trip path delay and subsequent variations in round-trip path delay. The initial uncertain delay due to the administrative unit-3 (AU-3) pointer operations in SDH equipment is determined by controlling the reference clock of the equipment. The characteristics of the equipment delay caused by the pointer operations are also described. In an experiment with actual SDH equipment, the error in initial delay determination was suppressed to 30 ns. The delay variation measurement of the round-trip path is not affected by the pointer operation in the proposed method. The precision of delay variation measurement can be better than 10 ns     

大时滞网络拥塞控制器的设计

几乎现有的AQM算法都忽略了大时滞对网络拥塞控制系统稳定性的负面影响,以致稳定性、响应性和鲁棒性在大时滞网络中大大降低.如PI、REM等AQM算法在大时滞环境下表现出剧烈的队列震荡和频繁的空队列情况,这些现象直接导致了链路利用率的低下和延时抖动的增大.而面向大时滞网络拥塞控制的DC-AQM算法不够恰当的参数配置使得系统输出偏离了控制的目标,出现了很高的分组丢弃概率.为解决大时滞网络拥塞控制的稳定性问题,本文基于内模控制原理,提出一种新的鲁棒AQM控制器IMC-PID来补偿网络时滞对系统稳定性的影响.仿真分析表明,随着网络时滞的增大,IMC-PID在综合性能上胜过其他算法,获得了较高的链路利用率和较低的延时抖动.     

Cooperative teleoperation of a multirobot system with force reflection via Internet

With the rapid development of information technology, the Internet has evolved from a simple data-sharing media to an amazing information world where people can enjoy various services. Recently, the use of the Internet has been expanded to the field of automation, i.e., using the Internet as a tool to control equipment located at remote sites. This work presents a system that enables multiple operators at different sites to cooperatively control multiple robots with real-time force reflecting via the Internet. To overcome instability and reliability problems caused by random time delay of the Internet communication, we adopt an event as the reference for the controller design. To improve real-time efficiency and reduce the complexity of the controller, a distributed approach is proposed for the control of remote robots, so that the time delay in one control loop does not affect performance of the others. A vision-based method is developed to monitor and render interactions between the robots. The usefulness and effectiveness of the developed method and system have been verified by teleoperation experiments on a two-robots cooperative system among Hong Kong, mainland China, and the U.S.     

Robust power control of CDMA cellular radio systems with time-varying delays

Power control is an important factor to increase communication link quality and system capacity in the direct-sequence code-division multiple access (DS-CDMA) cellular radio systems. The Smith prediction filter can achieve the unbiased asymptotic tracking about a desired target signal to interference noise ratio (SINR) under the fixed round-trip delay. However, it is sensitive to the variation of round-trip delay. In order to track the desired SINR with the time-varying round-trip delay, a multiple-mode Smith prediction filter, which combines the multiple Smith predictors with a likelihood function, is proposed for the power control of CDMA systems. The proposed scheme can compensate for the unknown time-varying round-trip delay. Simulation results show that the performance of the proposed multiple-mode power control method is robust to time-varying round-trip delay in the CDMA cellular radio systems.     

Lightweight digital video stabilization for small-size robot

Though constrained by payload and processing, small robots have gained applications in collecting visual information from the scene. Typically these small-size robots do not carry data loggers and send the video information to a hand-held device at a remote location for visual observations. Due to sophisticated processing and control limitations from mechatronics resources, the video captured by the robot is subjected to the effects of unintended motion, which requires digital methods for video stabilization. For a lightweight solution for video stabilization, we avoid use of any external hardware and develop a Singular Value Decomposition (SVD) based digital algorithm that avoids explicit feature tracking and motion estimation during stabilization. The process involves identifying a subspace with minimal dimensions that contains information of intentional motion alone. This work identifies the minimal subspace for video stabilization using the sliding window geometry method for practical implementation. Further, a shape-preserving filter is utilized to remove perturbations induced by the unintended motions, thereby resulting in the reconstruction of the stabilized video sequence. Experimental results on two different small-size robots viz spherical robot and Unmanned Aerial Vehicle (UAV) in indoor and outdoor settings, respectively, show quality outcomes without any change in parameters of the proposed filter design. Performance comparison with existing methods on the quality of stabilized video shows that the proposed stabilization method overcomes the non-availability of features for tracking due to large amplitudes and limited onboard resources. With the proposed video stabilization method, there is a potential for wider applicability of small-size robots in remote visual observations.     

Stability of Multi-Path Dual Congestion Control Algorithms

This paper investigates fair, scalable, stable congestion controls which achieve high bandwidth utilization over networks operating multi-path routing. It aims to take advantage of path diversity to achieve efficient bandwidth allocation without causing instability. We develop a multi-path extension to the dual algorithm, which takes into consideration path diversity when evaluating fairness. This algorithm is shown to be globally stable in the absence of propagation delays and a sufficient condition for local stability, for the case when heterogeneous propagation delays are present, is found. The sufficient condition we present is decentralized in the following sense: the gain parameter for each dynamic variable is restricted by the average round-trip time of packets passing through the link or source it represents, but not by the round-trip times of any other packets. This leads to a highly scalable parameter choice scheme. Gain parameters are calculated from local information which is independent of the state of the algorithm, and our delay stability condition is satisfied. The models considered apply to networks consisting of arbitrary interconnections of sources and links with arbitrary heterogeneous propagation delays.     

Depth-enhanced mobile robot teleguide based on laser images

3D stereoscopic visualization may provide a user with higher comprehension of remote environment in teleoperation when compared to 2D viewing. Works in the literature have addressed the contribution of stereo vision to improve perception of some depth cues often for abstract tasks, and it is hard to find contributions specifically addressing mobile robot teleguide. The authors of this paper have investigated stereoscopic viewing in mobile robot teleguide based on video images in a previous work and pointed out advantages of stereo viewing in this type of application as well as shortcomings inherent to the use of visual sensor, e.g. image transmission delay. The proposed investigation aims at testing mobile robot teleguide based on a different sensor: the laser sensor. The use of laser is expected to solve some problems related to visual sensor while maintaining the advantage of having stereoscopic visualization of a remote environment. A usability evaluation is proposed to assess system performance. The evaluation runs under the same setup of the previous study so to have an experimental outcome comparable to the previous one. The evaluation involves several users and two different 3D visualization technologies. The results show a strong improvement in users’ performance when mobile robot teleguide based on laser sensor is (depth-) enhanced by stereo viewing. Some differences are detected between the use of laser and visual sensor which are discussed.     

Bilateral robot system on the real-time network structure

This paper presents a bilateral robot system, which is driven by the static friction-free drive system and implemented on the real-time network structure. The goal is to realize a force reflecting bilateral teleoperation with haptic impression transmission over computer networks. The paper considers two subjects relating to the bilateral robot. The first is static friction, which degrades the performance of manipulation and results in a poor haptic impression. A new transmission mechanism named twin drive system developed by the authors resolves this problem. The transmission mechanism, which resembles the differential gear of automobiles, is essentially free of static friction. This static-friction-free motion greatly contributes to the broad range of motion control applications. The second subject is the time delay of the network, which may cause serious problems such as instability of the feedback system. To avoid such delay, the authors developed a new real-time network protocol stack (RTNP). The detailed mechanism of the twin drive system and architecture of the RTNP are presented, and the control scheme and experimental results are also shown.     

A switching command-based whole-body operation method for humanoid robots

This paper introduces a switching command-based whole-body operation method for humanoid robots. Humanoid robots are biped machines possessing multiple degrees of freedom (DOF). Due to the complexity of their multi-DOF structure, and the difficulty in maintaining postural stability, whole-body operation of humanoid robots is fundamentally different from traditional fixed-base manipulators or stable-base mobile manipulators. By studying the shifts in locus of attention between human body joints during task execution, we developed a switching command-based operation method that allows the operator to select only the necessary points of the humanoid robot's body for manipulation. Whole-body motion satisfying the desired movements of the selected points is generated using an inverse-kinematics motion generation scheme. This switching operation method enables flexible whole-body operation of humanoid robots using simple input devices. The proposed whole-body operation method is implemented as a teleoperation system using two 3-DOF joysticks to operate a 30-DOF humanoid robot (HRP-1S) developed in the Humanoid Robotics Project (HRP) of the Ministry of Economy, Trade, and Industry of Japan. Experiments teleoperating HRP-1S confirmed the effectiveness of our method.     

A real-time explicit mapping and teleoperation control method for humanoid robots with posture constraints

In most industrial robot applications, on-site robot operation is required. However, this is not feasible for certain harsh operating conditions, such as high temperatures and intense radiation. In such instances, teleoperation based on machine vision can be applied, and adding posture constraints helps control the robot to complete more complex tasks. The general idea is to use quadratic programming or other optimization methods to solve the problem. In order to improve accuracy and ensure real-time performance, a real-time explicit mapping and teleoperation control (REMTC) is proposed. Based on the principle of "included angle minimization" (IAM), the incorrect mapping problem which might be caused by the traditional method is solved. In addition, considering that the analytical inverse solution algorithm is not as universal as the general optimization framework, an improved structure transformation method is proposed, so that the algorithm can be applied to both Spherical- Revolute-Spherical (SRS) and non-SRS robots. Finally, the experiment verifies the effectiveness of the algorithm in real-time performance, accuracy and reliability.     

Development of a multi-behavior based mobile robot for remotesupervisory control through the Internet

We review the networked mobile robot systems and suggest taxonomy based on the three levels of control commands. The performance analysis result shows that direct control has potential difficulty for implementation due to the unpredicted transmission delay of the network. To tackle this problem, we have suggested the behavior-programming control concept to avoid disturbances of the Internet latency. For this purpose, primitive local intelligence of the mobile robot is grouped into motion planner, motion executor, and motion assistant, where each of a group is treated as an agent. They are integrated by centralized control architecture based on multi-agent concept, communicated through a center information memory. The event-driven concept is applied on the robot to switch the behaviors to accommodate the unpredicted mission autonomously. We have successfully demonstrated experimentally the feasibility and reliability for system through a performance comparison with direct remote control     

On Extending the Wave Variable Method to Multiple-DOF Teleoperation Systems

It is well known that providing a human operator with contact force information can significantly improve task performance in a teleoperation system. Unfortunately, time delay is a serious problem for such systems. Even a small time delay in a bilateral teleoperation system will generally degrade the system's performance and cause instability. Consequently, without some form of compensation for time delay, latencies in a teleoperation system would preclude the use of force feedback. Fortunately, there are approaches based on scattering theory and passivity that can compensate for time delay and allow the use of force feedback in teleoperation systems with latencies. In particular, the wave variable method is a passivity-based approach that guarantees stability for any fixed time delay. Since its introduction, the wave variable method has been augmented with predictors to compensate for variable time delay. The wave variable formalism has also been extended to multiple-DOF systems by replacing scalar damping constants with a family of impedance matrices. In this paper, the authors generalize this last approach to include a larger family of impedance matrices. The paper includes a complete derivation of the extended family of impedance matrices as well as simulation and experimental results to illustrate the approach.      

Gain scheduler middleware: a methodology to enable existing controllers for networked control and teleoperation-part II: teleoperation

This paper is the second of two companion papers. The foundation for the external gain scheduling approach to enable an existing controller via middleware for networked control with a case study on a proportional-integral (PI) controller for dc motor speed control over IP networks was given in Part I. Part II extends the concepts and methods of the middleware called gain scheduler middleware (GSM) in Part I to enable an existing controller for mobile robot path-tracking teleoperation. By identifying network traffic conditions in real-time, the GSM will predict the future tracking performance. If the predicted tracking performance tends to be degraded over a certain tolerance due to network delays, the GSM will modify the path-tracking controller output with respect to the current traffic conditions. The path-tracking controller output is modified so that the robot will move with the fastest possible speed, while the tracking performance is maintained in a certain tolerance. Simulation and experimental results on a mobile robot path-tracking platform show that the GSM approach can significantly maintain the robot path-tracking performance with the existence of IP network delays.