High-fidelity bilateral teleoperation systems and the effect of multimodal haptics.

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.     

Position referenced force augmentation in teleoperated hydraulic manipulators operating under delayed and lossy networks: A pilot study

Position error between motions of the master and slave end-effectors is inevitable as it originates from hard-to-avoid imperfections in controller design and model uncertainty. Moreover, when a slave manipulator is controlled through a delayed and lossy communication channel, the error between the desired motion originating from the master device and the actual movement of the slave manipulator end-effector is further exacerbated. This paper introduces a force feedback scheme to alleviate this problem by simply guiding the operator to slow down the haptic device motion and, in turn, allows the slave manipulator to follow the desired trajectory closely. Using this scheme, the master haptic device generates a force, which is proportional to the position error at the slave end-effector, and opposite to the operator’s intended motion at the master site. Indeed, this force is a signal or cue to the operator for reducing the hand speed when position error, due to delayed and lossy network, appears at the slave site. Effectiveness of the proposed scheme is validated by performing experiments on a hydraulic telemanipulator setup developed for performing live-line maintenance. Experiments are conducted when the system operates under both dedicated and wireless networks. Results show that the scheme performs well in reducing the position error between the haptic device and the slave end-effector. Specifically, by utilizing the proposed force, the mean position error, for the case presented here, reduces by at least 92% as compared to the condition without the proposed force augmentation scheme. The scheme is easy to implement, as the only required on-line measurement is the angular displacement of the slave manipulator joints.     

基于STM32的力觉反馈装置及用于主从遥操作实现

在遥操作系统中为了增强现实及实现本地力觉信号再现功能以提高精细化操作的目的 ,设计了用于人机交互功能的力反馈装置;该装置为单自由度结构,基于步进电机驱动;利用STM32微控制器采集触觉力信号以及关节位移信号,通过设计基于力误差的控制律调整位置变量实现输出力信号与标准力信号的匹配;为了验证该力反馈装置进行了标准力信号再现实验;并且利用该力反馈装置作为主机械手与单自由度从机械手搭建遥操作装置,进行了力、位置双边跟踪实验验证,实现了主、从机械手力、位置协同一致的目的.     

Automatic pick-and-place using a magnetically navigated microrobot and a motorized micromanipulator

This paper demonstrates the automatic pick-and-place of a small object in 2D using a magnetically navigated microrobot (MNM) and a motorized micromanipulator (MM). A master/slave control mechanism is used in the manipulation process. The MM is the master manipulator. The MNM is the slave manipulator. To avoid damaging the object by large holding force and to maintain successful holding, a position-based impedance control algorithm is implemented to the slave side. The feedback force to the impedance controller is obtained from an off-board force determination mechanism which overcomes the disadvantages of installing an on-board force sensor on the MNM. The performance of the proposed manipulation system was examined experimentally by transporting a hard-shell object to its desired destinations with predefined holding force. To the authors knowledge, this is the first work reported using a magnetically navigated microrobot to complete manipulation tasks with a screw type manipulator. The proposed system has potential utility in microinjection if the MNM was scaled down to proper size.

     

A haptic interface for computer-integrated endoscopic surgery and training

Haptic feedback has the potential to provide superior performance in computer-integrated surgery and training. This paper discusses the design of a user interface that is capable of providing force feedback in all the degrees of freedom (DOFs) available during endoscopic surgery. Using the Jacobian matrix of the haptic interface and its singular values, methods are proposed for analysis and optimization of the interface performance with regard to the accuracy of force feedback, the range of applicable forces, and the accuracy of control. The haptic user interface is used with a sensorized slave robot to form a master–slave test-bed for studying haptic interaction in a minimally invasive environment. Using the master–slave test-bed, teleoperation experiments involving a single degree of freedom surgical task (palpation) are conducted. Different bilateral control methods are compared based on the transparency of the master–slave system in terms of transmitting the critical task-related information to the user in the context of soft-tissue surgical applications.     

Master-slave intelligent robot telepresence system

In this paper, the analysis and design of master-slave intelligent robot telepresence system are discussed. When the operator acts on the master manipulator, the position and attitude information of the master manipulator are gathered by the computer. After calculating and coordinate transforming, the data are send to the computer of the slave manipulator. Then the slave manipulator-PUMA562 robot follows the master manipulator's movement precisely. Six-dimension force/toque sensor(lord cell) is mounted on the slave manipulator. As the master manipulator and the toque on the slave manipulator are different in structure, the force and the slave manipulator should be send to the master manipulator computer and dissociated by the master manipulator computer. Proper ratio of the force on the master manipulator and the force on the slave manipulator is selected, and distribute to the master manipulator joints. So that the operator could feel the force from the master manipulator, which is obtained by the motors of the joints. The proposed control scheme is introduced to a prototype master-slave system and the experimental results show the validity of the proposed scheme.     

Task-based and stable telenanomanipulation in a nanoscale virtual environment

In a haptic interface system with a nanoscale virtual environment (NVE) using an atomic force microscope, not only is stability important, but task-based performance (or fidelity) is crucial. In this paper, we introduce a nanoscale virtual coupling (NSVC) concept and explicitly derive the relationship between performance, stability, and scaling factors of velocity (or position) and force. An available scaling factor region is represented based on Llewellyn's absolute stability criteria and the physical limitation of the haptic device. For the stable haptic interface, the sampled time passivity controller is implemented in the NVE. Experiments have been performed for telenanomanipulation tasks, such as positioning, indenting, and nanolithography with guaranteed stability in the NVE. Note to Practitioners-This paper suggests methods and control schemes for the task-based and stable telenanomanipulation in the nanoscale virtual environment (NVE). The proposed task-based and stable telenanomanipulation in the NVE can be used for an augmented human machine interface for the manipulation of nanoscale objects with the atomic force microscope (AFM). In addition, it is beneficial for learning or performing nanoscale tasks, such as nanolithography, nanoindenting, nanofabrication, and cell manipulation. Also, the interaction with the NVE using haptic device provides a useful tool for researchers in a variety of disciplines, such as biology, chemistry, and physics. Moreover, it may even be applied to educational purposes. In future research, the developed stable haptic interface would be integrated with the AFM system as a slave manipulator for telenanomanipulation experiments, such as pushing a nanoparticle with precise positioning and nanoassembly.     

Teleoperation Control of a Position‐Based Impedance Force Controlled Mobile Robot by Neural Network Learning: Experimental Studies

Effective haptic performance in teleoperation control systems can be achieved by solving two major problems: the time‐delay in communication channels and the transparency of force control. The time‐delay in communication channels causes poor performance and even instability in a system. The transparency of force feedback is important for an operator to improve the performance of a given task. This article suggests a possible solution for these two problems through the implementation of a teleoperation control system between the master haptic device and the slave mobile robot. Regulation of the contact force in the slave mobile robot is achieved by introducing a position‐based impedance force control scheme in the slave robot. The time‐delay problem is addressed by forming a Smith predictor configuration in the teleoperation control environment. The configuration of the Smith predictor structure takes the time‐delay term out of the characteristic equation in order to make the system stable when the system model is given a priori. Since the Smith predictor is formulated from exact linear modeling, a neural network is employed to identify and model the slave robot system as a nonlinear model estimator. Simulation studies of several control schemes are performed. Experimental studies are conducted to verify the performance of the proposed control scheme by regulating the contact force of a mobile robot through the master haptic device.     

A teleoperation system for micro positioning with haptic feedback

This paper presents the research work on a 1 Degree of Freedom (DOF) macro-micro teleoperation system which enables human operator to perform complex task in micro environment such as cell insertion with the capability of haptic feedback. To reach submicron resolution, a nano-motion piezo actuator was used as the slave robot and a servo DC motor was used as the master robot. Force sensors were implemented at both ends for haptic feedback and a microscope equipped with camera was employed for real-time visual feedback. The hysteresis nonlinearity of the piezo motor was modeled using LuGre friction model and compensated for. A Sliding Mode Based Impedance Controller (SMBIC) was designed at the slave side to ensure position tracking while an impedance force controller was designed at the master side to ascertain tracking of the force. Control parameters were chosen based on Llewellyn stability criteria such that the entire system stays stable against parameter uncertainties and constant time delay. The experimental results demonstrated capability of the proposed control frameworks in desirable tracking of the position and force signals while the entire system remained stable. The results of this study can be used for complex tasks in micron environment such as cell insertion.     

遥控系统理想性能的实现和实验研究

本文对主从遥控系统理想性能的实现进行了理论分析和实验研究。对于位置／力控制的遥控机器人，理想性能就是要实现主从操作器的位置跟踪和力跟踪。文中首先通过分析得到了一组控制规律，利用这组控制规律就可以实现理想性能，而且这组控制规律由于引入了力微分信号而比较简单。接着以无源性为基础，对系统的稳定性进行了研究。最后通过实验可知在从操作器同环境有或无作用的情况下，主从遥控系统都可以实现理想性能。     

Lyapunov stable displacement-mode haptic manipulation of hydraulic actuators: theory and experiment

In this article, a stable control scheme is designed and experimentally evaluated for haptic-enabled teleoperated control of hydraulic actuators. At the actuator (slave) side, the controller allows the hydraulic actuator to have a stable position tracking. At the master side, the haptic device provides a kind of ‘feel’ of telepresence to the operator by creating a force that acts like a virtual spring, coupling the displacement of the haptic device to the displacement of the hydraulic actuator. In free motion, this virtual spring restricts the operator's hand to move fast when the slave manipulator is behind/ahead in terms of tracking the master manipulator's displacement. On the other hand, when interacting with the environment, the constrained force imposed on the hydraulic actuator is indirectly reflected through this virtual spring force. Extension of Lyapunov's stability theory to non-smooth systems is first employed to prove the stability of the resulting control system. Effectiveness of the controller is then validated via experimental studies. It is shown that the control scheme performs well in terms of both positioning the hydraulic actuator and providing a haptic feel to the operator. The control scheme is easy to implement since very little knowledge about system parameters is needed and the required on-line measurements are actuator's supply and line pressures and displacement.     

A delayed force-reflecting haptic controller for master–slave neurosurgical robots

This paper presents a new force-reflecting control system for master–slave haptic devices. This controller has been implemented and tested on the robotic systems for minimally invasive neurosurgery developed by our Research Group. Robot-assisted surgery is a very valuable treatment, since it allows benefits of high precision, accuracy, and repeatability of robotic devices. The proposed controller is meant to be used for master–slave haptic robotic surgery, but it can be used for any device that provides haptic feedback. The new controller merges the paradigms of force reflection (FR) control and delayed reference control. Unlike the FR control, the proposed solution enhances the safety since it does not allow an unwanted motion of the slave device once the operator releases the haptic controller. Experimental tests are provided to show the capabilities and the performance of the controller. Closed-loop stability is investigated both theoretically and experimentally. The analytic results on stability impose a limit on the ratio between the measured contact force and the sampling frequency of the closed-loop controller.     

State and Impedance Reflection Based Control Interface for Bilateral Telerobotic System with Asymmetric Delay

In this paper, we investigate state and imped-ance reflection based robust control strategy for bilateral shared telerobotic system under unsymmetrical time varying delay. Shared input for both master and slave robot is designed by combining delayed position and position-velocity signals with impedance reflection properties of the interaction between slave and environment and between human and master robot manipulator. Adaptive control algorithm is proposed to estimate the interaction properties between human and master manipulator and between slave and remote environment. Then, the delayed estimated interaction properties are reflected back to the master and slave robot manipulator to match with the estimated impedance properties of the interaction between human and remote environment. We combine robust term with adaptive control term to deal with the uncertainty associated with gravity loading vector, unmodeled dynamic and external disturbance. The stability conditions with time varying delays are derived by using Lyapunov-Krasovskii functional. Experimental results are given to demonstrate the validity of the proposed design for real-time applications.     

Nonlinear adaptive control of master–slave system in teleoperation

This paper is devoted to the nonlinear control design problem to achieve stability of master–slave manipulators in teleoperation system and its transparency in the sense of motion/force tracking. Nonlinear adaptive controllers are bilaterally designed for both master and slave sites to guarantee the stability of whole system and motion tracking performance. Global boundedness of the overall adaptive system and asymptotic motion (velocity/position) tracking are established. Especially, the concept of “virtual master manipulator” is introduced to increase degree of freedom of control design for force tracking performance. The resulting force tracking error depends only on the acceleration of the designed virtual master manipulator. Accurate dynamic parameters of manipulators, their acceleration information as well as models of human operator and environment are not required in the control design. Another important feature of our approach is the relaxation for the trade-off between motion and force tracking performances.     

基于距离传感器的双边遥操作

为了改善双边遥操作的力反馈性能,本文根据从端操作臂上的传感器检测的目标距离信息,设计了新 的PD 双边控制器．证明了系统的稳定性条件,并通过单自由度双边遥操作实验系统,对提出的控制方法进行了实 验验证．实验结果表明,当从端操作臂靠近目标时,主端操作臂产生了逐渐增大的反馈力．这种控制策略为操作者 安全实现遥操作任务提供了有效手段．     

Accurate force reflection method for a multi-d.o.f. haptic interface using instantaneous restriction space without a force sensor in an unstructured environment

This paper proposes an accurate force reflection method for a multi-d.o.f. haptic interface without a force sensor. Sensorless force reflection is possible using position–position (p–p) architecture. However, the conventional p–p architecture in the literature has limitations representing constraint space when it is applied to a multi-d.o.f. haptic interface in that it gives an inaccurate force direction. This paper demonstrates the limitation of the conventional p–p architecture through an example and proposes a novel force reflection method using the instantaneous restriction space (IRS) concept. The IRS can be calculated using the Jacobian and joint angle error of a slave manipulator. Since the proposed method has the form of an impedance two-port architecture in the sense of data flow, it can be easily combined with previous well-known results of two-port haptic display frameworks. The proposed method is especially useful when the slave manipulator collides with unexpected obstacles during motion, even though the slave does not have a force sensor. The performance of the proposed method is evaluated through experiments.     

A rod-mass method of soft tissue modeling and three dimensional force vector estimation for tele-surgery training

In a tele-surgery training system, the transparency is extremely important so as to ensure the success of the operation and the safety of soft objects. Due to current technique limits, it is difficult to mount force sensors at the end of the slave manipulator. In this paper, we propose a novel rod-mass algorithm and construct the model of soft objects. Through the modeling process, the accurate three dimensional contact force vector between the end of the manipulator and the soft object can be estimated in real time. A virtual spring using Hooke s law is introduced to the novel mass–spring method. Applying an impedance model, the three dimensional contact force estimates can be calculated from the deformation of masses’ positions and velocities. In order to verify our methods, a virtual reality interaction platform is constructed including the Omni master manipulator, a four joints manipulators, a virtual reality display, and the soft object’s model. Numerical simulations and experiments are performed to verify the accuracy and the feasibility of soft objects grasping. Results show the high effectiveness and efficiencies of our methods.     

Construction Methodology for a Remote Ultrasound Diagnostic System

In the present paper, we describe a method for constructing a remote ultrasound diagnostic system. Remote diagnosis can be realized using a communication network. We have developed a master–slave type remote medical system to diagnose shoulder diseases, such as dialysis-related amyloid arthropathy (DRAA), by ultrasonographic images. Proper positioning, orientation, and contact force between the ultrasound probe and the affected area of the patient are required in order to acquire proper diagnostic images. Safety and manipulability are also required when operating the remote medical system through a communication network. Therefore, the system has impedance control capability for positioning of the master and slave manipulators in order to convey the contact force and enhance manipulability. In addition, the system has continuous-path control capability for the orientation of the slave manipulator in order to realize smooth and accurate motion of the ultrasound probe, even if the sampling rate of the transmission of the orientation data of the master manipulator is not sufficient. The results of remote diagnostic experiments demonstrated that a healthcare professional could diagnose real patients through a communication network using the constructed system.      

Proposal of an Index for the Operator’s Haptic Sensation in a Master-Slave System with Force-Feedback Function

This article proposes an index to estimate the operator’s haptic sensation of the contact between the slave device and the environment in operating master–slave systems with force feedback function. The index value is derived from the velocity information of the master device before and after contact, which is hypothesized to represent the intensity of haptic sensation stimuli presented to the operator. Two characteristics of this index are discussed by means of psychophysics experiment, which are the statistical characteristics of the index value for different operators, and how the change in the operator’s haptic sensation is reflected on the index value. The index is validated by another psychophysics experiment. The experimental results show that the performance of operator’s haptic sensation can be predicted correctly based on the proposed index value. This index is expected to be applied in the parameter design of bilateral-control systems with force feedback function.     

Dual arm coordination and control

A generalized master/slave technique and experimental results for coordinated control of two arms rigidly grasping an object is described in this paper. Usually master/slave refers to the case when one arm is controlled in position mode while the second arm is controlled in force mode. In generalized master/slave control, each arm at its tool frame has six degrees of freedom; each degree of freedom can be in either position or force control mode. An interactive program has been developed to allow a user the flexibility to select appropriate control modes for a given experiment. This interface also allows for control gain adjustments. This paper presents the results of several experiments performed on this system to demonstrate its capabilities such as transporting an object with or without induced internal forces and movement of a constrained object. The system is further developed to achieve a so-called shared control mode in which an operator species the free motion trajectory for a point on the object of manipulation via a joystick while the autonomous control system is used for coordination and control of the arms.