Cooperative modalities in robotic tele-rehabilitation using nonlinear bilateral impedance control

A nonlinear model reference adaptive bilateral impedance controller is proposed that can accommodate various cooperative tele-rehabilitation modes for patient–therapist interaction using a multi-DOF tele-robotic system. In this controller, two reference impedance models are implemented for the master and slave robots using new model reference adaptive control laws for the nonlinear bilateral teleoperation system. “Hand-over-hand” and “adjustable-flexibility” are two modes of patient–therapist cooperation that are realized using the proposed strategy. The Lyapunov-based stability proof guarantees the patient’s and the therapist’s safety during the cooperation and interaction with robots, even in the presence of modeling uncertainties of the multi-DOF teleoperation system. The performance of the proposed bilateral impedance controller is experimentally investigated for upper-limb tele-rehabilitation in the two mentioned cooperation modes.     

基于广义扩张状态观测器的遥操作系统同步控制

针对机械臂遥操作系统中存在的时变时延问题,提出了基于广义扩张状态观测器的控制方法,实现了遥操作系统稳定并且主从机械臂关节角位置同步的控制目标。首先通过反馈线性化,将遥操作系统的主从机械臂动力学模型转化为一个关于位置跟踪误差和时延的状态空间模型。针对该多输入多输出的干扰不匹配模型,设计了广义扩张状态观测器和相应的控制律,从而消除了时变时延以及其它扰动引起的不确定性对系统的影响,并对系统进行稳定性和抗扰性分析。最后,通过仿真验证了所设计的控制方法的有效性。     

Integrated adaptive robust control for multilateral teleoperation systems under arbitrary time delays

With the increasing industrial requirements such as bigger size object, stable operation, and complex task, multilateral teleoperation systems extended from traditional bilateral teleoperation are widely developed. In this paper, the integrated control design is developed for multilateral teleoperation systems, where n master manipulators are operated by human to remotely control n slave manipulators cooperatively handling a target object. For the first time, the control objectives of multilateral teleoperation including stability, synchronization, transparency, and internal force distribution are clarified systematically. A novel communication architecture is proposed to cope with communication delays, where the estimated environmental parameters are transmitted from the slave side to the master, to replace the traditional environmental force measurement in the communication channel. A kind of nonlinear adaptive robust control technique is used to deal with nonlinearities, unknown parameters, and modeling uncertainties existing in the master, slave, and environmental dynamics, so that the excellent tracking performance is achieved in both master and slave sides. The coordinated motion/force control is designed in the slave side by the optimal internal force distribution among n slave manipulators, and the impedance control is designed in the master side to realize the target transparency behavior. In summary, the proposed control algorithm can achieve the guaranteed robust stability, the excellent synchronization and transparency performance, and the optimal internal force distribution simultaneously for multilateral teleoperation systems under arbitrary time delays and various modeling uncertainties. The simulation is carried out on a 2‐master/2‐slave teleoperation system, and the results show the effectiveness of the proposed control design. Copyright © 2015 John Wiley & Sons, Ltd.     

Spatial Motion Constraints Using Virtual Fixtures Generated by Anatomy

This paper describes a spatial-motion-constraints-generation approach for a human-machine collaborative surgical-assistant system from registered computer tomography models. We extend constrained optimization formulation incorporating task goals, anatomy-based constraints, "no fly zones," etc. We use a fast potential-collision-constraint-detection method based on a 3-D surface model and covariance tree data structure. These boundary constraints, along with task behaviors and joint limits, serve as constraint conditions for constrained robot control. We are able to follow a complex path inside a human skull, phantom represented by a surface model composed of 99 000 vertices and 182 000 triangles in real time. Our approach enables real-time task-based control of a surgical robot in a precise interactive minimally invasive surgery task. We illustrate our approach based on two example tasks which are analogous to the procedures in endoscopic sinus surgery, and analyze the user's performance on both teleoperation and cooperative control for one of the example tasks. The experimental results show that a robotic assistant employing our approach on spatial motion constraints can assist the user in skilled manipulation tasks, while maintaining desired properties. Our approach is equally applicable to teleoperative and cooperative controlled robots     

外骨骼式遥操作主手设计及主从异构映射算法研究

为了更好地促进机器人适应复杂的遥操作任务,开发了能够精确获取人体上肢运动信息的外骨骼式遥操作主手,并通过异构映射算法,实现对6自由度协作机械臂的遥操作.首先,基于人体仿生结构,设计了可穿戴式8自由度外骨骼主手(臂部7自由度和手部1自由度);其次,通过改进的D-H(Denavit-Hartenberg)方法建立遥操作系统的运动学模型,基于Matlab的机器人工具箱进行了工作空间仿真,并设计主从异构映射算法;最后,实验验证外骨骼主手在遥操作系统中的可操作性,以及工作空间异构映射算法的可行性.实验表明,外骨骼主手能够控制从端机械手臂,且保证末端位置和姿态一致,可在大范围工作空间内复现人体上肢精细运动,主从跟随误差达2 mm,工作空间类似于直径1.08 m的半球形.因此,可穿戴式的外骨骼主手使操作者能更加直观地参与到遥操作系统当中,辅助操作者更加高效地完成精细复杂任务.     

Experimental Comparison Study of Control Architectures for Bilateral Teleoperators

A detailed experimental comparison study of several published algorithms for motion and force control of bilateral teleoperators, with emphasis on Internet-based teleoperation, is presented. The study investigates the effects of data losses, communication delays, and environmental constraints on a teleoperation system for different control techniques, which are based on wave variables, Smith predictors, and recent algorithms on synchronization. The controllers are compared on stability, transparency, and complexity using two identical nonlinear robots coupled via a stochastic network model that allowed transmission round-trip delays and data-loss rates to range from 8 to 1088 ms and 0% to 50%, respectively. A total of 18 subjects, which were distributed among 26 experiments with the aims of regulating the effects of the operators learning process and dynamic properties, participated in this study. Overall, the comparison study reports a deteriorating effect in the performance (i.e., larger position errors and lower fidelity of contact information) from delays and data losses. Yet, the effect of data losses is less critical when compared with time delays. In addition, the preference for a particular control framework is shown to strongly depend on the operational conditions of the system, such as the characteristics of the coupling channel, the specifics of the remote task, and the computational capabilities of the manipulators.      

Parts-probability-based vehicle detection

Detecting vehicles is important in aerial surveillance. Traditional methods used classifiers to detect vehicles, but a single classifier was limited to detecting vehicles of only one intensity and orientation. Therefore, the task required the use of multiple classifiers of different intensities and orientations. To solve this problem, we first used a latent Dirichlet allocation （LDA） model that improved the previous approaches to vehicle detection. Previous text modeling approaches have been generative. They could be used to build probability models of vehicles in different intensities and from various orientations simultaneously using unlabeled data. Using a probability model, we can detect vehicles in a region with high probability. Next, we used a parts-probability model that improves the LDA model. The model effectively encodes spatial structure among visual words by adding spatial relationships among vehicle parts as priors of words. A parts probability model represents a vehicle hierarchically according to parts appearances and a vehicle＇s features within the parts to enforce spatial coherency. Then, we used our model to detect vehicles from a collection of images and demonstrate its performs more effectively.     

Human control model in teleoperation rendezvous

This paper proposes a human control model in teleoperation rendezvous on the basis of human information processing （perception, judgment, inference, decision and response）. A predictive display model is introduced to provide the human operator with predictive information of relative motion. By use of this information, the longitudinal and lateral control models for the operator are presented based on phase plane control method and fuzzy control method, and human handling qualities are analyzed. The integration of these two models represents the human control model. Such a model can be used to simulate the control process of the human operator, which teleoperates the rendezvous with the aid of predictive display. Experiments with human in the loop are carried out based on the semi-Physical simulation system to verify this human control model. The results show that this human control model can emulate human operators＇ performance effectively, and provides an excellent way for the analysis, evaluation and design of the teleoperation rendezvous system.     

Modeling and control of cooperative teleoperation systems

This paper presents a multilateral control architecture for teleoperation in multimaster/multislave environments. The proposed framework incorporates flow of position and force information between all master and slave robots, rather than merely between corresponding units. Within this architecture, cooperative performance measures are defined to enhance coordination among the operators and the robots for achieving the task objectives. A /spl mu/-synthesis-based methodology for cooperative teleoperation control is also introduced. This approach guarantees robust stability of cooperative teleoperation in the presence of dynamic interaction between slave robots, as well as unknown passive operators and environment dynamics. It also improves task coordination by optimizing relevant performance objectives. Experiments carried out with a two-master/two-slave single-axis system demonstrate the effectiveness of the proposed approach.     

多机械手系统建模的一般方法

以双机械手协调作业为例,介绍了多机械手系统建模的一般方法.对于不同数目的机械手以及不同的作业任务,通过定义广义变量,建立关于广义变量的运动学和动力学方程,进而获得多机械手系统的统一模型.它为多机械手系统的控制和仿真提供了基础.     

Adaptive wavenet controller design for teleoperation systems with variable time delays using singular perturbation method

The main goal of controller design in teleoperation systems is to achieve stability and optimal operation in presence of factors such as time delays, system disturbances and modeling errors. This paper proposes a new method of controller design based on wavenet with singular perturbation method for the bilateral teleoperation of robots through the internet. The wavenet controller could overcome the variable time delay in teleoperation system. This new method introduces a reduced-order structure for control and stability of teleoperation systems. By using singular perturbation method, teleoperation system is decomposed into two fast and slow subsystems. This method is a step towards reduced-order modeling. In this method, we use a feedback linearization method in master subsystem and a wavenet controller for slave subsystem. In wavenet controller, we used a learning method so that the system was Lyapunov stable. As the stability of the model is highly dependent on the learning of the system, we use Lyapunov stability in this method. It has been tried to reduce the tracking error between the master and the slave subsystems. In this structure the position of master-slave are compared together and controlling signal is applied to the slave so that they can track each other in the least possible time. In all schemes the effectiveness of the system is shown through the simulations and they have been compared with each other.     

预估控制下的实时网络遥操作移动机器人

构建了能使操作者通过Internet远程实时控制的移动机器人系统．为了补偿网络时延和抵消其对遥操作系统的影响,基于我们以前提出的改进型Smith预估器原理,采用了预估控制策略．为了保证系统稳定性和透明性,基于主从端的传感器信息交换,设计了一个动态模型管理器,其中模型和力反馈误差调节通过模糊控制实现．除了力反馈外,为了增强遥操作的实时性,引入了预估的虚拟显示．为了精确地预测网络时延,提出了一个新颖的时钟同步算法．为了降低时延抖动,结合我们提出的两个算法,实现了数据缓冲策略．最后,通过长距离的网络遥操作实验验证了系统和控制策略的实用性和有效性．     

半张量积理论在网络遥操作系统中的应用

研究了网络遥操作系统的半张量积建模问题.网络遥操作系统中存在人机交互以及大时延事件,这使得系统中既有离散事件动态又有连续动态,当系统的离散事件动态与底层连续动态相互作用耦合强烈时,系统的性能、行为分析变得更为复杂.本文通过引入半张量积给出了一种新的半张量积混杂建模方法,它把一类特定事件系统建模为布尔系统,通过半张量积方法得到类似于离散动态系统的方程.它将系统中离散事件和连续动态相互耦合的地方表达为半张量积的形式,最后经过扩张简化得到完整统一的系统数学模型,它将有利于进一步分析、计算.本文给出的半张量积建模方法和步骤具有一定的通用性和一般性.此外本文还对基于半张量积的混杂网络遥操作系统模型进行了基于事件控制和时间控制的仿真.仿真结果表明,半张量积建模方法完全可信、可行,基于事件控制的方法在网络遥操作控制中明显优于基于时间控制的方法.     

Advanced teleoperation and control system for industrial robots based on augmented virtuality and haptic feedback

There are some industrial tasks that are still mainly performed manually by human workers due to their complexity, which is the case of surface treatment operations (such as sanding, deburring, finishing, grinding, polishing, etc.) used to repair defects. This work develops an advanced teleoperation and control system for industrial robots in order to assist the human operator to perform the mentioned tasks. On the one hand, the controlled robotic system provides strength and accuracy, holding the tool, keeping the right tool orientation and guaranteeing a smooth approach to the workpiece. On the other hand, the advanced teleoperation provides security and comfort to the user when performing the task. In particular, the proposed teleoperation uses augmented virtuality (i.e., a virtual world that includes non-modeled real-world data) and haptic feedback to provide the user an immersive virtual experience when remotely teleoperating the tool of the robot system to treat arbitrary regions of the workpiece surface. The method is illustrated with a car body surface treatment operation, although it can be easily extended to other surface treatment applications or even to other industrial tasks where the human operator may benefit from robotic assistance. The effectiveness of the proposed approach is shown with several experiments using a 6R robotic arm. Moreover, a comparison of the performance obtained manually by an expert and that obtained with the proposed method has also been conducted in order to show the suitability of the proposed approach.     

基于虚拟环境的多操作者多机器人协作遥操作系统

建立了基于虚拟环境的多操作者多机器人协作遥操作系统,采用3层C/S结构的控制系统软件构架.根据实验系统平台工作场景对分布式虚拟环境进行建模,以实现多操作者对多机器人的分布式控制.结合多操作者多机器人遥操作系统的结构特点,采用本地滞后原理对分布式虚拟环境的一致性控制进行了研究,解决了遥操作系统一致性控制中时钟同步、滞后时...     

Bilateral teleoperation of a group of mobile robots for cooperative tasks

A visual and force feedback-based teleoperation scheme is proposed for cooperative tasks. The bilateral teleoperation system includes a haptic device, an overhead camera and a group of wheeled robots. The commands of formation and average velocities of the multiple robots are generated by the operator through the haptic device. The state of the multiple robots and the working environment is sent to the human operator. The received information contains the feedback force through the haptic device and visual information returned by a depth camera. The feedback force based on the difference between the desired and actual average velocities is presented. The wave variable method is employed in the bilateral teleoperation of multiple mobile robots with time delays. The effectiveness of the bilateral teleoperation system is demonstrated by experiments. The robots in the slave side are able to follow the commands from the master side to interact with the environments, including moving in different formations and pushing a box. The results show that the scheme enables the operator to manipulate a group of robots to complete cooperative tasks freely.     

Teleoperation Control of a Position‐Based Impedance Force Controlled Mobile Robot by Neural Network Learning: Experimental Studies

Effective haptic performance in teleoperation control systems can be achieved by solving two major problems: the time‐delay in communication channels and the transparency of force control. The time‐delay in communication channels causes poor performance and even instability in a system. The transparency of force feedback is important for an operator to improve the performance of a given task. This article suggests a possible solution for these two problems through the implementation of a teleoperation control system between the master haptic device and the slave mobile robot. Regulation of the contact force in the slave mobile robot is achieved by introducing a position‐based impedance force control scheme in the slave robot. The time‐delay problem is addressed by forming a Smith predictor configuration in the teleoperation control environment. The configuration of the Smith predictor structure takes the time‐delay term out of the characteristic equation in order to make the system stable when the system model is given a priori. Since the Smith predictor is formulated from exact linear modeling, a neural network is employed to identify and model the slave robot system as a nonlinear model estimator. Simulation studies of several control schemes are performed. Experimental studies are conducted to verify the performance of the proposed control scheme by regulating the contact force of a mobile robot through the master haptic device.     

Exponential synchronization of nonlinear multi‐agent systems with time delays and impulsive disturbances

The problem of cooperative synchronization of nonlinear multi‐agent systems with time delays is investigated in this paper. Compared with the existing works about synchronization (or consensus) of multi‐agent systems, the method in this paper provides a more general framework by considering nonlinear multi‐agent systems with time delays and impulsive disturbances. The model in this paper is sufficiently general to include a class of delayed chaotic systems. Based on the Lyapunov stability theory and algebraic graph theory, sufficient conditions are presented to guarantee the cooperative exponential synchronization for these multi‐agent delayed nonlinear systems. These conditions are expressed in terms of linear matrix inequalities, which can easily be checked by existing software tools. It is seen that the Lyapunov functions must be constructed based on the graph topology to prove synchronization. The well‐known master–slave (drive‐response) synchronization of two chaotic delayed systems is a special case of this paper, and therefore, the results in this paper are also useful for practical applications in secure communication. Simulation results verify the effectiveness of the proposed synchronization control algorithm. Copyright © 2015 John Wiley & Sons, Ltd.     

Control of Teleoperators with Communication Time Delay through State Convergence

This paper describes a new control method of teleoperation systems with communication time delay. This method models the teleoperation system in the state space, considering all the possible interactions that could appear in the operator‐master‐slave‐environment set, and it uses the Taylor expansion to model the time delay. The control system allows that the slave manipulator follows the master in spite of the time delay in the communication channel. The tracking is achieved by state convergence between the master and the slave. The method is also able to establish the desired dynamics of this convergence and the dynamics of the slave manipulator. Furthermore, a simple design procedure is provided to obtain the control system gains. These control gains are calculated solving a set of seven equations. The control method is robust to the uncertainty of the design parameters, so it is not necessary to obtain good estimations of these parameters. Simulations and experiments with a one DOF teleoperation system are presented to verify the control method. © 2004 Wiley Periodicals, Inc.     

Experimental evaluation of teleoperation system with force-free control and visual servo control by human operator perception

This research considers the teleoperation of an articulated robot arm by means of force-free control and visual servo control (VSC) over communication channels using the Internet technology. A semi-autonomous type teleoperation system contains a human supervisory control and VSC schemes and switches one scheme to another for accomplishing the required task accurately. The main investigation is carried out to find how it effectively improves the accuracy and the effectiveness of the teleoperation after provision of a visual feedback channel to the system. The system accuracy, effectiveness, repeatability and handleability based on the human operator’s skills and operator’s cognitive aspects are evaluated using experimental results and statistical data analysis. Effectiveness of the statistical analysis is assured by increasing the number of experiment data and assuming environmental factors, and implicit variables maintain to be unchanged. Correlation coefficients are calculated to find out how controlled input parameters are related to the successful output given by the system.