Multi-Fingered Bimanual Haptic Interface Robot with Three-Directional Force Display

This paper introduces a bimanual haptic interface robot and presents results from its trial operation. Our aim in developing a bimanual haptic interface is to display high-precision three-directional forces at all 10 fingertips of both hands of the operator. By installing two five-fingered robot hands and two robot arms, we construct a bimanual haptic interface. A haptic interface that consists of robot hands and robot arms can provide multi-point contact between the operator and a virtual environment. However, there is the risk that robot hands and robot arms will collide while an operator is manipulating the haptic interface. To solve this problem, we also propose a collision avoidance control law for the multi-fingered bimanual haptic interface. Finally, to determine the validity of the proposed interface, we carry out several experiments. These results show the validity and great potential of the proposed bimanual haptic interface.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

Haptic force control based on impedance/admittance control aided by visual feedback

This paper demonstrates a haptic device for interaction with a virtual environment. The force control is added by visual feedback that makes the system more responsive and accurate. There are two popular control methods widely used in haptic controller design. First, is impedance control when user motion input is measured, and then, the reaction force is fed back to the operator. The alternative method is admittance control, when forces exerted by user are measured and motion is fed back to the user. Both, impedance and admittance control are also basic ways for interacting with a virtual environment. In this paper, several experiments were performed to evaluate the suitability of force-impedance control for haptic interface development. The difference between conventional application of impedance control in robot motion control and its application in haptic interface development is investigated. Open loop impedance control methodology is implemented for static case and a general-purpose robot under open loop impedance control was developed as a haptic device, while a closed loop model based impedance control was used for haptic controller design in both static and dynamic case. The factors that could affect to the performance of a haptic interface are also investigated experimentally using parametric studies. Experimental results for 1 DOF rotational motion and 2 DOF planar translational motion systems are presented. The results show that the impedance control aided by visual feedback broaden the applicability of the haptic device and makes the system more responsive and accurate.

Control aspects of a robotic haptic interface for kinesthetic knee joint simulation

A new 6 DOF (degrees of freedom) haptic device is presented for display of the dynamic properties of a virtual human knee in orthopaedic training. In order to achieve the high impedance requirements in this application an industrial robot has been selected and extended with a PC-based high performance controller hardware and redundant safety features. Two control architectures—a force-command and motion-command control—are presented and both implemented in a 1 DOF application. With a new method for evaluation and comparison of controller performance in terms of impedance error the experimental results reveal that motion-command provides better accuracy for display of the high impedances specific to the human knee application.     

A human–robot interface for mobile manipulator

This paper presents a remote manipulation method for mobile manipulator through operator’s gesture. In particular, a track mobile robot is equipped with a 4-DOF robot arm to grasp objects. Operator uses one hand to control both the motion of mobile robot and the posture of robot arm via scheme of gesture polysemy method which is put forward in this paper. A sensor called leap motion (LM), which can obtain the position and posture data of hand, is employed in this system. Two filters were employed to estimate the position and posture of human hand so as to reduce the inherent noise of the sensor. Kalman filter was used to estimate the position, and particle filter was used to estimate the orientation. The advantage of the proposed method is that it is feasible to control a mobile manipulator through just one hand using a LM sensor. The effectiveness of the proposed human–robot interface was verified in laboratory with a series of experiments. And the results indicate that the proposed human–robot interface is able to track the movements of operator’s hand with high accuracy. It is found that the system can be employed by a non-professional operator for robot teleoperation.     

Development and Evaluation of a 7-DOF Haptic Interface

With the development of human robot interaction technologies, haptic interfaces are widely used for 3D applications to provide the sense of touch. These interfaces have been utilized in medical simulation, virtual assembly and remote manipulation tasks. However, haptic interface design and control are still critical problems to reproduce the highly sensitive touch sense of humans. This paper presents the development and evaluation of a 7-DOF (degree of freedom) haptic interface based on the modified delta mechanism. Firstly, both kinematics and dynamics of the modified mechanism are analyzed and presented. A novel gravity compensation algorithm based on the physical model is proposed and validated in simulation. A haptic controller is proposed based on the forward kinematics and the gravity compensation algorithm. To evaluate the control performance of the haptic interface, a prototype has been implemented. Three kinds of experiments:gravity compensation, static response and force tracking are performed respectively. The experimental results show that the mean error of the gravity compensation is less than 0.7 N and the maximum continuous force along the axis can be up to 6 N. This demonstrates the good performance of the proposed haptic interface.      

Telerobotic visual servoing

This paper addresses the problem of integrating the human operator with autonomous robotic visual tracking and servoing modules. A CCD camera is mounted on the end-effector of a robot and the task is to servo around a static or moving rigid target. In manual control mode, the human operator, with the help of a joystick and a monitor, commands robot motions in order to compensate for tracking errors. In shared control mode, the human operator and the autonomous visual tracking modules command motion along orthogonal sets of degrees of freedom. In autonomous control mode, the autonomous visual tracking modules are in full control of the servoing functions. Finally, in traded control mode, the control can be transferred from the autonomous visual modules to the human operator and vice versa. This paper presents an experimental setup where all these different schemes have been tested. Experimental results of all modes of operation are presented and the related issues are discussed. In certain degrees of freedom (DOF) the autonomous modules perform better than the human operator. On the other hand, the human operator can compensate fast for failures in tracking while the autonomous modules fail. Their failure is due to difficulties in encoding an efficient contingency plan.     

Design of a Haptic Interface for a Gastrointestinal Endoscopy Simulation

This paper presents a new design and analysis of a haptic interface for a gastrointestinal endoscopy simulation. The gastrointestinal endoscopy is a procedure in which the digestive tract and organs of a patient are diagnosed and treated using a long and flexible endoscope. The developed haptic interface incorporates two degrees of freedom (DOF), each of which is necessary to describe the movements of an endoscope during the actual endoscopy procedures. The haptic interface has a translational motion mechanism to implement the insertion movement of the endoscope, and a rotational motion mechanism to implement the rotational movement of the endoscope. The endoscope included in the haptic interface is supported by a folding guide to prevent the endoscope from buckling. Force feedback in each direction is provided by wire-driven mechanisms. The developed haptic interface has a workspace, sensitivity, and maximum attainable force and torque enough to simulate the endoscopy procedures such as colonoscopy, upper GI (gastrointestinal) endoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). The developed haptic interface is applied to implementation of a colonoscopy simulation. Performance including force bandwidth is evaluated through experiments and simulation.     

A haptic interface for computer-integrated endoscopic surgery and training

Haptic feedback has the potential to provide superior performance in computer-integrated surgery and training. This paper discusses the design of a user interface that is capable of providing force feedback in all the degrees of freedom (DOFs) available during endoscopic surgery. Using the Jacobian matrix of the haptic interface and its singular values, methods are proposed for analysis and optimization of the interface performance with regard to the accuracy of force feedback, the range of applicable forces, and the accuracy of control. The haptic user interface is used with a sensorized slave robot to form a master–slave test-bed for studying haptic interaction in a minimally invasive environment. Using the master–slave test-bed, teleoperation experiments involving a single degree of freedom surgical task (palpation) are conducted. Different bilateral control methods are compared based on the transparency of the master–slave system in terms of transmitting the critical task-related information to the user in the context of soft-tissue surgical applications.     

两自由度康复训练交互装置

针对手部康复训练设备主要由主动驱动器驱动导致的安全稳定性差、容易造成二次伤害等问题,设计了一种用于手部康复训练的被动驱动交互装置.首先简单介绍了被动驱动器的结构、实现原理,在此基础上提出了一种两自由度手部康复训练交互装置设计方法,该交互装置在支架上设置两根相互垂直并处于同一平面的轴,轴的两端分别连接被动驱动器和数字编码器,手柄在两轴的驱动下能在两维空间内运动并产生两自由度的力.接着分析了手柄受力与被动驱动器输出力的变换方法,最后设计康复游戏,与研制的交互装置配合,开展了脑卒中患者的康复训练试验,试验结果验证了康复训练交互装置对于手部康复训练的有效性.     

基于ARM的多自由度控制装置研究

为满足微小型车辆和机器人控制实时性及功能扩展性的需求,设计了一种控制装置,能实现某小型履带式车前后行进、转向和天线升降、旋转等多个自由度的控制。文中分析了被控对象,提出控制装置的软硬件设计方案,以ARM7为微处理器,以μC／OS-Ⅱ嵌入式实时操作系统为软件平台。介绍了微处理器模块、传感器模块、执行模块和遥控模块,实现了系统的PID控制算法及基于MiniGUI的图形用户界面程序开发。控制装置可满足对控制实时性和智能化的要求,具有操作灵活、界面友好、功耗低、体积小等特点,为小型车辆和机器人的控制系统设计提供了一种新颖的方法。     

Assistant force compensation for hand movement of patients by using exogenous signals and/or neurophysiological signals

Because functional diseases of the brain can cause disabilities related to human movement control, a compensation method was developed for improving the performance of hand movements. The compensation for human hand movements can be carried out by adding an assistant force that is generated from artificial equipment attached to a human arm. From the experiment on visual target tracking, it was found that the tracking trajectory was adequately represented by a dynamic model of the motion of an articulated industrial robot arm, and the different abilities for movement control among healthy people and patients were classified by different model parameters as position loop gain, velocity loop gain, and response delay. Dynamic force compensation was approached by considering the different control features of the patients. The effectiveness of the proposed compensation method was verified in a simulation study on an actual industrial robot arm. A human-machine interface, e.g., a brain-computer interface (BCI), for realizing the control of artificial equipment to compensate for human hand movements is also presented and discussed.     

Haptic display of micro surface undulation based on discrete mechanical stimuli to whole fingers

Humans can perceive a wide and small surface undulation that is hundreds micrometers in height and hundreds millimeters in width by scanning the surface with their whole fingers and palm in the distal and proximal directions. We developed a wearable haptic device that presents a surface undulation to the hand. The device is composed of nine independent stimulator units that control the heights of nine finger pads of the index finger, the middle finger, and the ring finger (three units on each finger) according to the virtual surface. Three experiments are carried out to evaluate the haptic perception by the haptic device. A first experiment shows that the perceived dimensions are diminished as compared to the dimensions applied by the haptic device. On the basis of this result, the applied dimensions are calibrated to match the virtual surface undulation to the real surface undulation. A second experiment shows that the shape of gently-curved surfaces can be estimated with the haptic display. A third experiment shows that the discrimination threshold is not different between the virtual surface undulation and the real surface undulation. These experimental results show the applicability of the haptic device as a haptic interface.     

NC painting robot for narrow areas

Giving a natural operational feeling to a human via a haptic interface requires not only a sophisticated and intuitive mechanical design, but also an appropriate control system design. Most haptic systems, however, implicitly demand that the human gets used to manipulation of the haptic devices before he/she can get the feel of the virtual space and/or telepresence beyond the haptic device. Based on a new concept of a human-in-the-loop system called Human Adaptive Mechatronics (HAM), an assist-control for a force/vision interactive haptic system is discussed in this paper. The proposed assist-control scheme includes online estimation of a operator's control characteristics, and a 'force assist' function implemented as a change in the support ratio according to the identified skill level. We developed a HAM haptic device test system, performed evaluation experiments with this apparatus and analyzed the measured data. It was confirmed that the operator's skill could be estimated sufficiently and that operator's performance was enhanced by the assist-control.     

Human-assisted virtual reality for a magnetic-haptic micromanipulation platform

This paper deals with the development of a virtual reality interface (VRI) for a magnetic-haptic micromanipulation platform (MHMP) (Mehrtash et?al. in IEEE/ASME Trans Mechatron 16(3):459–469, 2011). Our previously developed MHMP has shown a great deal of promise in non-contact micromanipulations. This micromanipulation platform concerns the integration of magnetic actuation technology and a bilateral macro–micro teleoperation. The MHMP has two separate stations: one magnetic microrobotic station and one haptic. The magnetic microrobotic station manipulates micro-sized objects based on the commands from the haptic station. The haptic station uses bilateral communication with the magnetic microrobotic station to allow a human operator the feeling of a micro-domain environment. In this paper, we report a VRI that enables human operators to improve their skills in using the MHMP, before carrying out an actual dexterous task. The VRI is made up of three main components: a haptic station, a simulation engine, and a display unit. The haptic station provides the operator with the force/torque information from virtual or remote environments, and is also used to recognize the operator’s hand motion command. Dynamical computation and control system modeling have been carried out on the simulation engine. Based on the real-time computation, this engine, as the heart of the system, provides force applied to the operator’s hand and the microrobot’s position for the haptic station and the display unit, respectively. The display unit employs 3D computer graphics to demonstrate the micromanipulation tasks and environments. The VRI is also developed in such a way that it can be separately used in parallel with the MHMP for the 3D visualization of a real task by providing multiple virtual viewports. This paper introduces the configuration of the proposed VRI, and reports the result of a preliminary experiment using micromanipulation investigation for validation.     

小型化柔性触觉再现装置

提出了一种小型化的具有实时触觉反馈作用的人机接口装置的设计方法,以控制弹性梁的有效变形长度为核心,保证了触觉再现的实时性和准确性,从而可以在操作者指端大范围再现虚拟对象或远程对象的刚度。介绍了该小型化人机触觉再现接口装置的硬件设计和软件设计。该实时触觉再现接口具有尺寸小、刚度再现范围大,可方便与鼠标相结合等特点,可广泛应用于遥操作机器人和虚拟现实领域。     

A fail-safe tele-autonomous robotic system for nuclear facilities

A fail-safe tele-autonomous robotic system is proposed for use in advanced nuclear reprocessing facilities. The design exploits the technologies developed for space telerobotics. The target system consists of a graphical user interface for an operator to execute robotic tasks, hand controllers for teleoperation, a three-dimensional graphical simulator, and robot control software to drive both the graphical simulation and dual six degree-of-freedom robots to perform tasks using autonomous, teleoperated, and shared control modes. A preliminary design for a safety monitoring system for fail-safe operations is also described.     

Usability Evaluation of VR Interface for Mobile Robot Teleoperation

This article presents the results from research in which 3 different remote control interfaces were compared to assess the impact of interface structure on the performance of the operator for remotely controlled mobile inspection robots. The primary control interface of a mobile robot consists of a head-mounted display, data gloves for gripper control, joystick for movement control of the robot platform, and a motion tracking system for measuring head orientation and hand position. In order to compare different control interfaces, an additional system, based on a Liquid Crystal Display monitor and joystick, was prepared. Results of this study show that the use of virtual reality techniques in the interfaces of mobile inspection robots increases operator productivity, the level of spatial presence, and distance evaluation while facilitating the execution of tasks, as well as improving and speeding up their execution and reducing the operator’s time needed to adapt to the control interface. The latter is achieved with the increased level of intuitive control while ensuring comfort.     

Motion planning and contact control for a tele-assisted hydraulic underwater robot

Implementing tele-assistance or supervisory control for autonomous subsea robots requires atomic actions that can be called from high level task planners or mission managers. This paper reports on the design and implementation of a particular atomic action for the case of a subsea robot carrying out tasks in contact with the surrounding environment.Subsea vehicles equipped with manipulators can have upward of 11 degrees of freedom (DOF), with degenerate and redundant inverse kinematics. Distributed local motion planning is presented as a means to specify the motion of each robot DOF given a goal point or trajectory. Results are presented to show the effectiveness of the distributed versus non-distributed approach, a means to deal with local minima difficulties, and the performance for trajectory following with and without saturated joint angles on a robot arm.Consideration is also given to the modelling of hydraulic underwater robots and to the resulting design of hybrid position/force control strategies. A model for a hydraulically actuated robot is developed, taking into account the electrohydraulic servovalve, the bulk modulus of oil, piston area, friction, hose compliance and other arm parameters. Open and closed-loop control results are reported for simulated and real systems.Finally, the use of distributed motion planning and sequential position/force control of a Slingsby TA-9 hydraulic underwater manipulator is described, to implement an atomic action for tele-assistance. The specific task of automatically positioning and inserting a Tronic subsea mateable connector is illustrated, with results showing the contact conditions during insertion.