Scaled teleoperation system for nano-scale interaction and manipulation

In this paper, a visual and haptic human–machine interface is proposed for teleoperated nano-scale object interaction and manipulation. Design specifications for a bilateral scaled tele-operation system with slave and master robots, sensors, actuators and control are discussed. The Phantom? haptic device is utilized as the master manipulator, and a piezoresistive atomic force microscope probe is selected as the slave manipulator and as topography and force sensors. Using the teleoperation control system, initial experiments are realized for interacting with nano-scale surfaces. It is shown that fine structures can be felt on the operator's finger successfully, and improved nano-scale interaction and manipulation using visual and haptic feedback can be achieved.     

Lyapunov stable displacement-mode haptic manipulation of hydraulic actuators: theory and experiment

In this article, a stable control scheme is designed and experimentally evaluated for haptic-enabled teleoperated control of hydraulic actuators. At the actuator (slave) side, the controller allows the hydraulic actuator to have a stable position tracking. At the master side, the haptic device provides a kind of ‘feel’ of telepresence to the operator by creating a force that acts like a virtual spring, coupling the displacement of the haptic device to the displacement of the hydraulic actuator. In free motion, this virtual spring restricts the operator's hand to move fast when the slave manipulator is behind/ahead in terms of tracking the master manipulator's displacement. On the other hand, when interacting with the environment, the constrained force imposed on the hydraulic actuator is indirectly reflected through this virtual spring force. Extension of Lyapunov's stability theory to non-smooth systems is first employed to prove the stability of the resulting control system. Effectiveness of the controller is then validated via experimental studies. It is shown that the control scheme performs well in terms of both positioning the hydraulic actuator and providing a haptic feel to the operator. The control scheme is easy to implement since very little knowledge about system parameters is needed and the required on-line measurements are actuator's supply and line pressures and displacement.     

基于力觉交互的高速率精准操作技能训练方法

介绍了用力觉交互技术进行手眼协调高速率精准操作的动作技能训练方法。提出了记录播放和轨迹智能导引两种培训模式。在记录培训模式中,采用PD控制的方法使学员被动感受专家的运动信息。在智能导引纠正模式中,学员主动操作交互设备,计算机根据学员的操作情况控制交互设备输出导引力或纠正力。最后,采用Omega 3DOF建立了具备触觉显示和图形显示功能的“信封靶”描绘技能训练系统样机平台,分析了力模型参数对系统稳定性的影响。实验结果证明了培训方法的可行性。     

Design and control of a planar haptic device with passive actuators based on passive force manipulability ellipsoid (FME) analysis

In this paper, we propose an optimal design for a passive haptic device with brakes and its control method. The inability of a brake to generate torque significantly affects the performance of a multi‐DOF haptic device, in that a desired force can be generated only approximately in some workspace and, in some cases, the device may become stuck contrary to the user's intention. In this research, these limitations are analyzed by means of the so‐called passive force manipulability ellipsoid. Through the analysis, performance indices are developed for evaluating the limitations associated with passive haptic devices. Optimization is conducted for a 5‐bar mechanism with redundant actuation, and a coercive force approximation scheme is developed to avoid unsmooth motion during the wall‐following task along the virtual wall. It is experimentally shown that the performance in relation to the limitations is greatly improved for the optimized mechanism. © 2005 Wiley Periodicals, Inc.     

A Survey of Haptic Rendering Techniques

Computer Graphics technologies have developed considerably over the past decades. Realistic virtual environments can be produced incorporating complex geometry for graphical objects and utilising hardware acceleration for per pixel effects. To enhance these environments, in terms of the immersive experience perceived by users, the human's sense of touch, or haptic system, can be exploited. To this end haptic feedback devices capable of exerting forces on the user are incorporated. The process of determining a reaction force for a given position of the haptic device is known as haptic rendering. For over a decade users have been able to interact with a virtual environment with a haptic device. This paper focuses on the haptic rendering algorithms which have been developed to compute forces as users manipulate the haptic device in the virtual environment.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

High performance haptic device for force rendering in textile exploration

The HAPTEX system aims to develop a new multi-sensory environment for the immersive exploration of textiles. HAPTEX is based on a multi-layer/multi-thread architecture that optimizes the computational speed and integrates three different types of sensory feedback: vision, tactile and haptic. Such kind of environment is suitable for demanding VR applications such as the online marketing of novel textiles or garments, however it requires the design of a high performance multi-point haptic interface. The present work focuses on the haptic device design and describes how demanding requirements can be met by integrating on a high performance device a force sensor to achieve closed loop control. The methodology for the dimensioning of a motion based explicit force control on the basis of the dynamic parameters will be discussed and the specific implementation for the HAPTEX system presented.     

Predictive human operator model to be utilized as a controller using linear,neuro-fuzzy and fuzzy-ARX modeling techniques

Modeling human operator's behavior as a controller in a closed-loop control system recently finds applications in areas such as training of inexperienced operators by expert operator's model or developing warning systems for drivers by observing the driver model parameter variations. In this research, first, an experimental setup has been developed for collecting data from human operators as they controlled a nonlinear system. Appropriate reference signals and scenarios were designed according to the system identification and human operator modeling theory, to collect data from subjects. Different modeling schemes, namely ARX models as linear approach, and adaptive-network-based fuzzy inference system (ANFIS) as intelligent modeling approach have been evaluated. A hybrid modeling method, fuzzy-ARX (F-ARX) model, has been developed and its performance was found to be better in terms of predicting human operator's control actions as well as replacing the operator as a stand-alone controller. It has been concluded that F-ARX models can be a good alternative for modeling the human operator.     

A fuzzy logics clustering approach to computing human attention allocation using eyegaze movement cue

Human's attention is an important element in human–machine interface design due to a close relationship between operator's attention and operator's work performance. However, understanding of operator's attention allocation while he or she is performing a task remains a challenging task because attention is generally unobservable, immeasurable, and uncertain. In our previous study, we demonstrated the effectiveness of using operator's eye movement information to understand attention allocation, which has made attention observable. The present paper describes our study which addressed immeasurability and uncertainty of attention. Specifically, we used eye fixation's duration to indicate operator's attention and developed a new computational model for the attention and its allocation using fuzzy logics clustering techniques. Along with the development of this model, we also developed an experiment to verify the effectiveness of the model. The result of the experiment shows that the model is promising.     

Study of factors impacting remote diagnosis performance on a PLC based automated system

In this paper, we present systematically experimental and analytical evaluations on design of remote fault diagnosis systems for a programmable logic controller (PLC) based automated system. In order to investigate the factors of remote architecture, operator's skill level, and fault's effect on diagnosis performance, comprehensive experimental evaluations, statistical analysis and survey were conducted. The experiment compared three levels of remote architectures, two levels of operators’ skill levels on four typical faults in an automated system. Performance evaluation including detection time, amount of information search, number of diagnostic tests, number of asked questions, number of system runs, and performance score, were extracted from the experiment record. Two-stage statistical analysis including (1) analysis of variance (ANOVA) and (2) least significant difference (LSD) paired comparison was conducted on the performance evaluation data. From the statistical analysis results and expert survey, we concluded that: (1) the architecture sophistication eased the diagnosis on the faults that are related to the measurement signals, and (2) the diagnosis performance also increased with the sophistication of the architecture, but (3) operator's skill level did not significantly affect the diagnosis performance. The survey results on troubleshooters’ opinions and preferences about the diagnosis were also summarized, which can be applied for improvement on design of remote diagnosis system. The proposed evaluation approach is systematic; it can be applied on design and evaluation of diagnostics systems on other PLC based automated systems such as heating, ventilation, and air conditioning (HVAC), robotics assembly.     

Haptic force control based on impedance/admittance control aided by visual feedback

This paper demonstrates a haptic device for interaction with a virtual environment. The force control is added by visual feedback that makes the system more responsive and accurate. There are two popular control methods widely used in haptic controller design. First, is impedance control when user motion input is measured, and then, the reaction force is fed back to the operator. The alternative method is admittance control, when forces exerted by user are measured and motion is fed back to the user. Both, impedance and admittance control are also basic ways for interacting with a virtual environment. In this paper, several experiments were performed to evaluate the suitability of force-impedance control for haptic interface development. The difference between conventional application of impedance control in robot motion control and its application in haptic interface development is investigated. Open loop impedance control methodology is implemented for static case and a general-purpose robot under open loop impedance control was developed as a haptic device, while a closed loop model based impedance control was used for haptic controller design in both static and dynamic case. The factors that could affect to the performance of a haptic interface are also investigated experimentally using parametric studies. Experimental results for 1 DOF rotational motion and 2 DOF planar translational motion systems are presented. The results show that the impedance control aided by visual feedback broaden the applicability of the haptic device and makes the system more responsive and accurate.

A graphic modeling and analysis tool for human fault diagnosis tasks

This study presents a graphic modeling and analysis tool for use in constructing an operator's mental model in fault diagnosis tasks. In most automatic and complicated process control systems, human fault diagnosis tasks have become increasingly complex and specialized. The system designer should consider the cognitive process of human operator to avert failure of implement action owing to a lack of compatibility between humans and aiding system interface. Here, an experiment is performed to investigate the nature of human fault diagnosis. A graphic modeling and analysis tool is then proposed to model the continuous process of human fault diagnosis. The approach proposed herein exploits both the line-chart and Petri nets to demonstrate the operator's thoughts and actions. Moreover, results in this study are integrated into an adaptive standard diagnosis model that can assess the operators' mental workload and accurately depict the interactions between human operator and aiding system.Relevance to industryAutomatic intelligent diagnosis systems can not provide satisfactory operating performance. Human diagnosticians are more effective than computer ones. Results in this study offer further insight into an operator behavior in graphic form and also how to design a better aiding system.     

Intelligent manipulator system with nonsymmetric and redundant master-slave

Some work is so complicated and unsteady that it is not possible to use automatic robots, such as FA robots. In such a case, a teleoperated manipulation system is applied. In this research, the authors aim at a reduction in the operator's physical and mental burdens. An artificially intelligent manipulator system has been developed with nonsymmetric and redundant master-slave. This system has five features: (1) a polar coordinates master arm; (2) a highly operational articulated slave arm with 7 degrees of freedom; (3) a nonsymmetric configuration and different degrees of freedom master-slave control; (4) an expert system; and (5) a new master-slave control motion, which makes the operator's task easier with automatic force/position control. The system was experimentally produced and its performance tested and evaluated. A qualitative evaluation was carried out by conducting a comparative test on the conventional master-slave control and the new master-slave control. It was found to be effective in reducing operating time, as well as work-induced fatigue.     

Multi-Modal Human Interactions with an Intelligent Interface Utilizing Images, Sounds, and Force Feedback

One goal of research in the area of human–machine interaction is to improve the ways a human user interacts with a computer through a multimedia interface. This interaction comprises of not only text, graphical animation, stereo sounds, and live video images, but also force and haptic feedback, which can provide more “real” feeling to the user. The force feedback joystick, a human interface device is an input–output device. It not only tracks user's physical manipulation input, but also provides realistic physical sensations of force coordinated with system output. As part of our research, we have developed a multimedia computer game that assimilates images, sounds, and force feedback. We focused on the issues of how to combine these media to allow the user feel the compliance, damping, and vibration effects through the force feedback joystick. We conducted series of human subject experiments that incorporated different combinations of media, including the comparative study of the different performances of 60 human users, aiming to answer the question: What are the effects of force feedback (and associated time delays) when used in combination with visual and auditory information as part of a multi-modal interface? It is hoped that these results can be utilized in the design of enhanced multimedia systems that incorporate force feedback.     

Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator

Although people usually contact a surface with some area rather than a point, most haptic devices allow a user to interact with a virtual object at one point at a time and likewise most haptic rendering algorithms deal with such situations only. In a palpation procedure, medical doctors push and rub the organ's surface, and are provided the sensation of distributed pressure and contact force (reflecting force) for discerning doubtable areas of the organ. In this paper, we suggest real-time area-based haptic rendering to describe distributed pressure and contact force simultaneously, and present a haptic interface system to generate surface properties in accordance with the haptic rendering algorithm. We represent the haptic model using the shape-retaining chain link (S-chain) framework for a fast and stable computation of the contact force and distributed pressure from a volumetric virtual object. In addition, we developed a compact pin-array-type tactile display unit and attached it to the PHANToM^TM haptic device to complement each other. For the evaluation, experiments were conducted with non-homogenous volumetric cubic objects consisting of approximately 500 000 volume elements. The experimental results show that compared to the point contact, the area contact provides the user with more precise perception of the shape and softness of the object's composition, and that our proposed system satisfies the real-time and realism constraints to be useful for a virtual reality application.     

Teleoperation Control of a Position‐Based Impedance Force Controlled Mobile Robot by Neural Network Learning: Experimental Studies

Effective haptic performance in teleoperation control systems can be achieved by solving two major problems: the time‐delay in communication channels and the transparency of force control. The time‐delay in communication channels causes poor performance and even instability in a system. The transparency of force feedback is important for an operator to improve the performance of a given task. This article suggests a possible solution for these two problems through the implementation of a teleoperation control system between the master haptic device and the slave mobile robot. Regulation of the contact force in the slave mobile robot is achieved by introducing a position‐based impedance force control scheme in the slave robot. The time‐delay problem is addressed by forming a Smith predictor configuration in the teleoperation control environment. The configuration of the Smith predictor structure takes the time‐delay term out of the characteristic equation in order to make the system stable when the system model is given a priori. Since the Smith predictor is formulated from exact linear modeling, a neural network is employed to identify and model the slave robot system as a nonlinear model estimator. Simulation studies of several control schemes are performed. Experimental studies are conducted to verify the performance of the proposed control scheme by regulating the contact force of a mobile robot through the master haptic device.     

Energy-based control of a haptic device using brakes.

This paper proposes an energy-based control method of a haptic device with electric brakes. Unsmooth motion is frequently observed in a haptic system using brakes during a wall-following task. Since it is generally known that a haptic system using brakes is passive due to brake's characteristics, its energy behavior has seldom been investigated. However, force distribution at the end effector reveals that the unsmooth motion of a haptic system using brakes represents active behavior of the system in the specific direction. A force control scheme is proposed that computes the gain for smooth motion by considering the energy behavior of a system. Experiments show that smooth wall following is possible with a proposed force control scheme.     

Virtual prototyping of a car turn-signal switch using haptic feedback

The present work describes the virtual prototyping procedure of car turn-signal switches using a dedicated haptic feedback system. Although turn-signal switches have mechanisms with simple geometry, simulating the “feeling” during its operation is still a challenge. In order to allow ergonomics designers to directly assess the comfort of operation, a dedicated haptic system was developed for reproducing the force feedback that would characterize a real turn-signal switch. For the accurate haptic feedback generation and rendering purpose, a detailed computer-aided design model of the switch was developed together with a multibody simulation. For the design purpose a novel method for the synthesis of the cam mechanism was also developed, starting from an imposed force profile that was previously validated on the haptic device, for which the cam geometry is determined. Thus, the proposed approach enables the designer to also design the “feel” of the switch, and directly manufacture a good model, simplifying the usual iterative process necessary to implement the desired force profile. Finally, an example of virtual prototyping of a turn-signal switch is presented demonstrating the usability of the proposed method and the haptic system.     

Force modeling for tooth preparation in a dental training system

Feedback force is very important for novices to simulate tooth preparation by using the haptic interaction system (dental training system) in a virtual environment. In the process of haptic simulation, the fidelity of generated forces by a haptic device decides whether the simulation is successful. A force model computes feedback force, and we present an analytical force model to compute the force between a tooth and a dental pin during tooth preparation. The force between a tooth and a dental pin is modeled in two parts: (1) force to resist human’s operation and (2) friction to resist the rotation of the dental engine. The force to resist the human’s operation is divided into three parts in the coordinates that are constructed on the bottom center of the dental pin. In addition, we also consider the effects of dental-pin type, tooth stiffness, and contact geometry in the force model. To determine the parameters of the force model, we construct a measuring system by using machine vision and a force/torque sensor to track the human’s operations and measure the forces between the dental pins and teeth. Based on the measuring results, we construct the relation between the force and the human’s operation. The force model is implemented in the prototype of a dental training system that uses the Phantom as the haptic interface. Dentists performing virtual operations have confirmed the fidelity of feedback force.     

力反馈主手研究现状及其力控制方法研究

力觉主手作为主从遥操作机器人系统的关键部件,其机构特性和力控制性能直接关系到整个遥操作系统能否完成预期的任务要求。近年来,关于力觉主手及其力控制方法的研究已成为机器人遥操作领域的研究热点。文中详细介绍了力觉主手在机构设计和力控制算法方面研究现状和发展趋势。