A General-Purpose 7 DOF Haptic Device: Applications Toward Robot-Assisted Surgery

A 7 DOF haptic device has been designed and developed with applications towards robot-assisted minimally invasive surgery. The device consists of four degrees of force feedback (X, Y, Z, and grasping) capability and seven degrees of position feedback capability. It has a closed kinematic chain with two halves (user interface and spatial mechanism) that connect together via a universal joint. The user interface contains four degrees of position feedback, namely, the roll, pitch, yaw, and linear motion of the hand and forearm. In addition, a grasping mechanism with two thimbles mounted at the end of the user interface provides force feedback to the fingers of the user. The spatial mechanism provides force feedback to the user interface through a universal joint located at the grasping mechanism. This paper presents the design and development of this haptic device. In addition, a kinematic and workspace analysis of the device has been completed to compute the position of the slave robot and end-effector tool. Friction estimation has been presented to enable a higher transparency of the haptic device. Finally, a simulation of needle insertion into soft tissue was developed to test the device.     

Virtual environments for medical training: graphical and hapticsimulation of laparoscopic common bile duct exploration

We develop a computer-based training system to simulate laparoscopic procedures in virtual environments for medical training. The major hardware components of our system include a computer monitor to display visual interactions between 3D virtual models of organs and instruments together with a pair of force feedback devices interfaced with laparoscopic instruments to simulate haptic interactions. We simulate a surgical procedure that involves inserting a catheter into the cystic duct using a pair of laparoscopic forceps. This procedure is performed during laparoscopic cholecystectomy to search for gallstones in the common bile duct. Using the proposed system, the user can be trained to grasp and insert a flexible and freely moving catheter into the deformable cystic duct in virtual environments. The associated deformations are displayed on the computer screen and the reaction forces are fed back to the user through the force feedback devices. A hybrid modeling approach was developed to simulate the real-time visual and haptic interactions that take place between the forceps and the catheter, as well as the duct; and between the catheter and the duct     

Interactive Dynamic Simulation Schemes for Articulated Bodies through Haptic Interface

This paper describes interactive dynamic simulation schemes for articulated bodies in virtual environments, where user interaction is allowed through a haptic interface. We incorporated these schemes into our dynamic simulator I‐GMS, which was developed in an object‐oriented framework for simulating motions of free bodies and complex linkages, such as those needed for robotic systems or human body simulation. User interaction is achieved by performing push and pull operations with the PHANToM haptic device, which runs as an integrated part of I‐GMS. We use both forward and inverse dynamics of articulated bodies for the haptic interaction by the push and pull operations, respectively. We demonstrate the userinteraction capability of I‐GMS through on‐line editing of trajectories for 6‐dof (degrees of freedom) articulated bodies.     

Control software with supporting features to enhance the quality of tactile feedback

Tactile displays are devices for cutaneous stimulation to be integrated in haptic feedback systems e.g. in robot-assisted minimally invasive surgery. In general, there are severely limited in performance due to the necessary small size. In this work, we have developed a control software with the goal to allow simple hardware to present sensible tactile information to the user. For the development and evaluation of the software including various features to improve tactile feedback, a tactile display with twelve servo-driven pins was used. With the pins moving upwards and downwards, height maps can be presented to the user’s finger. The feedback system runs at a frequency of 50 Hz which generates the sensation of a fluid movement. The supporting features include a simulation of shear forces which give the user information on the movement direction of the sensor. A smoothing algorithm was implemented to prevent jerky pin movements. High effort was put in the generation of well distinguishable vibration patterns. These serve to enhance the presentation of the height maps or even allow a second layer of information.In an evaluation series, the control software and the support functions were extensively tested. The users were capable of distinguishing differences in height as low as 0.05 mm or differences in width smaller than the pin spacing. The task to find an invisible object only with the help of different vibration patterns was solved with great success. In a practical test, the users had to pursuit invisible paths standing out from the surroundings for 1 mm and less using the mouse relying only on tactile feedback. The users showed very good performance here with each user finishing every part of the test. This leads to the conclusion that our control software is an appropriate mean to create sensible tactile feedback even with limited hardware.     

Force display using a hybrid haptic device composed of motors and brakes

In the virtual environment, force feedback to the human operator makes virtual experiences more realistic. However, the force feedback using active actuators such as motors can make the system active and sometimes unstable. To ensure the safe operation and enhance the haptic feeling, system stability should be guaranteed. Both active actuators such as motors and passive ones such as brakes are commonly used for haptic devices. Motors can generate a torque in any direction, but they can make the system active and thus, sometimes unstable during operation. On the other hand, brakes can generate a torque only against their rotation, but they dissipate energy during operation and this dissipation makes the system intrinsically stable. Consequently, motors and brakes are complementary to each other. In this research, a two degree-of-freedom (DOF) haptic device equipped with motors and brakes is designed, in which each DOF is actuated by a pair of motor and brake. Simultaneous operation of motors and brakes is analyzed. Models for some environments, virtual wall contact and frictional effect, are proposed. The results for the hybrid haptic system are compared with those for the active haptic system and the passivity based control system. The experimental results show that the hybrid haptic device is more suited to some applications than the other haptic systems.     

The effect of force saturation on the haptic perception of detail

This paper presents a quantitative study of the effects of maximum capable force magnitude of a haptic interface on the haptic perception of detail. Specifically, the haptic perception of detail is characterized by identification, detection, and discrimination of round and square cross-section ridges, in addition to corner detection tests. Test results indicate that performance, measured as a percent correct score in the perception experiments, improves in a nonlinear fashion as the maximum allowable level of force in the simulation increases. Further, all test subjects appeared to reach a limit in their perception capabilities at maximum-force output levels of 3-4 N, while the hardware was capable of 10 N of maximum continuous force output. These results indicate that haptic interface hardware may be able to convey sufficient perceptual information to the user with relatively low levels of force feedback. The data is compiled to aid those who wish to design a stylus-type haptic interface to meet certain requirements for the display of physical detail within a haptic simulation.     

On the Design of Miniature Haptic Devices for Upper Extremity Prosthetics

We have developed three different versions of a multifunction haptic device that can display touch, pressure, vibration, shear force, and temperature to the skin of an upper extremity amputee, especially the one who has undergone targeted nerve reinnervation (TR) surgery. In TR patients, sensation from the reinnervated skin is projected to the missing hand. This paper addresses the design of the mechanical display, the portion responsible for contact, pressure, vibration, and shear force. A variety of different overall design approaches satisfying the design specifications and the performance requirements are considered. The designs of the fully prototyped haptic devices are compared through open-loop frequency response, closed-loop force response, and tapping response in constrained motion. We emphasize the tradeoffs between key design factors, including force capability, workspace, size, bandwidth, weight, and mechanism complexity.      

Hybrid control algorithm to improve both stable impedance range and transparency in haptic devices

An ideal haptic device should transmit a wide range of stable impedances with maximum transparency. When using active actuators, transparency improvement algorithms tend to decrease the range of attainable impedances. Passive actuators can transmit high impedances stably, but are not sufficient alone for transparency. In this study, a hybrid force control algorithm employing active and passive actuators was developed to improve the stable impedance range and transparency in haptic devices. A new transparency-Z-width plot is proposed as a way to evaluate the stable impedance range and transparency together. The hybrid control algorithm uses parameters to share the torque demand between two actuators with smooth transition. These parameters were determined and an artificial neural network (ANN) was used to extend them to the entire achievable impedance range. The algorithm was tested experimentally on a 1-DOF haptic device. The transparency experiments employed an excitation motor located at the user side of the device to evaluate various algorithms in time and frequency domains. Results showed that the proposed hybrid control algorithm enables simulation of higher range of impedances with higher transparency than the conventional algorithms.     

Pseudo-haptics for rigid tool/soft surface interaction feedback in virtual environments

This paper proposes a novel pseudo-haptic soft surface stiffness simulation technique achieved by displaying the deformation of the soft surface and maneuvering an indenter avatar over a virtual soft surface by means of a touch-sensitive tablet. The visual feedback of the surface deformation and the alterations to the indenter avatar behavior produced by the proposed technique create the illusion of interaction with a hard inclusion embedded in the virtual soft surface. The proposed pseudo-haptics technique is validated with a series of experiments conducted by employing a tablet computer with an S-pen input and a tablet computer with a bare finger input. Tablet computers provide unique opportunities for presenting the pseudo-haptic (indenter avatar speed), haptic (contact reaction force from the device surface) and visual cues (surface information) at the same active point of interaction which facilitates information fusion. Hence, here, we evaluate the performance of tablet computers in identification of hard inclusions within virtual soft objects and compare it with the performance of a touchpad input device. A direct hand-soft surface interaction is used for benchmarking of this study. We found that compared with using a touchpad, both the sensitivity and the positive predictive value of the hard inclusion detection can be significantly improved by 33.3% and 13.9%, respectively, by employing tablet computers. Using tablet computers could produce results comparable to the direct hand-soft surface interaction in detecting hard inclusions in a soft object. The experimental results presented here confirm the potential of the proposed technique for conveying haptic information in rigid tool/soft surface interaction in virtual environments.     

4-inch transparent plates based on thin-film AlN actuators for haptic applications

Numerous applications require tactile interfaces today. In particular, many customers’ applications such as automotive, Smartphone, tablet PC or touch pad can be concerned by high performances, low voltage haptic interfaces which allow the user to interact with its environment by the sense of touch. This technology is already used but with limitations such as high power consumption and limited feedback effect because today a simple vibration is commonly obtained. We chose to work on the squeeze-film effect. It consists in changing the friction between the finger and a plate resonator. It provides high granularity level of haptic sensation. This paper deals with the design, realization and characterization of high performances actuators in order to promote the squeeze-film effect on a 4-inch transparent plate (diagonal of the plate). Using Finite Element Method (FEM) models, we select the best design, able to generate the highest plate displacement amplitude as possible. We built demonstrators using a generic technology based on thin-film Aluminum Nitride (AlN) actuators on glass substrate. Electromechanical characterizations prove that it is possible to obtain the focused substrate vibration amplitude using only 35 V in amplitude. The integration of the thin-film actuator plate in a haptic demonstrator is now ongoing.     

Skin-Mountable Vibrotactile Stimulator Based on Laterally Multilayered Dielectric Elastomer Actuators

Skin-stimulation technology has attracted intense attention for virtual/augmented reality applications and tactile-feedback systems. However, bulky, heavy, and stiff characteristics of existing skin-stimulating devices limit their wearability and comfort, thus disturbing the immersive experience of users. This study presents a new type of thin and lightweight dielectric elastomer actuator for developing a skin-mountable vibrotactile stimulator. A new methodology is suggested to enhance the operating efficiency of dielectric elastomer actuators based on a laterally aligned dielectric multilayer structure (≈900 layer) with short dielectric distance (≈10 µm) and a soft elastomer/ionic liquid composite with low modulus and high dielectric constant. With the improved structural/material properties, the flexible actuator exhibits high displacements at low operating voltage (<200 V) over a wide frequency range (≈800 Hz). Therefore, the finger-band type vibrotactile stimulator based on the laterally multilayered dielectric elastomer actuators can exert indentations that have the ability of stimulating all mechanoreceptors in human skin over the full perception frequency/amplitude range. In addition, the actuator shows a high electromechanical stability for long-term operation due to time-efficient and precise fabrication process using sophisticated photolithography and secondary sputtering. Therefore, this vibrotactile stimulator shows high promise for use in tactile-assistive devices, tactile communications, haptic feedback, and beyond.     

Cognitive feedback for use with FES upper extremity neuroprostheses

This paper describes the development of two sensory substitutions systems that provide cognitive feedback for FES hand grasp restoration neuroprostheses. One system uses an array of five electrodes to provide machine status information and a spatially encoded representation of the command signal that a quadriplegic individual generates to achieve proportional grasp control. Only one electrode site is active at any given instant, and a second informational channel is superimposed on the spatial position channel by modulating the frequency of the stimulus pulses. The frequency modulated feedback channel signals six levels of force developed at the finger tips during prehension activities. The second sensory system is an integral part of an implanted FES system and utilizes a single subdermally placed electrode to display machine status information and a five-level frequency code for feedback of the user generated grasp control signal. The multielectrode feedback system was implemented for laboratory studies using surface mounted electrodes, although its design will ultimately incorporate subdermal electrodes to provide a highly cosmetic and unencumbering system. An evaluation of the effectiveness of grasp force and command signal feedback provided by this multielectrode system in assisting an FES hand system user to regulate grasp force during a laboratory task, showed increased consistency of performance and an economy of grasp effort between 25 and 30%. Alternative strategies for feedback information and coding algorithms are discussed.     

High-frequency acceleration feedback in wave variable telerobotics

The human hand is very sensitive to the high-frequency accelerations produced by tool contact with a hard object, yet most time delayed telerobots neglect this feedback band entirely in order to achieve stability. We present a control architecture that both incorporates this important information and provides the ability to scale and shape it independently of the low-frequency force feedback. Leveraging the clean power flows afforded by wave variables, this augmented controller preserves the passivity of any environment that it renders to the user, but is not subject to the limitations of being passive itself. This architecture guarantees stability in the presence of communication delay while achieving a level of feedback not possible with a passive controller. We show experimentally that this feedback augmentation and shaping can present a high-frequency acceleration profile to the user's hand that is similar to that experienced by the slave end effector. Two simple user studies also show that the feedback augmentation improves the user's perception, performance, and confidence with the given tasks. We anticipate that these natural haptic cues will make teleoperative systems easier to use and thus more widely applicable.     

A model-based and simulation-driven methodology for design of haptic devices

High precision and reliable haptic devices are highly complex products. The complexity that has to be carefully treated in the design process is largely due to the multi-criteria and conflicting character of the functional and performance requirements. These requirements include high stiffness, large workspace, high manipulability, small inertia, low friction, high transparency, as well as cost constraints. The requirements are a basis for creating and assessing design concepts. Concept evaluation relies to a large extent on a systematic usage of kinematic, dynamic, stiffness, friction, and control models. The design process can benefit from a model-based and simulation-driven approach, where one starts from an abstract top-level model that is extended via stepwise refinements and design space exploration into a detailed and integrated systems model that can be physically realized. Such an approach is presented, put in context of the V-model, and evaluated through a test case where a haptic device, based on a Stewart platform, is designed and realized. It can be concluded, based on simulation and experimental results that the performance of this deterministically optimized haptic device satisfies the stated user requirements. Experiences from this case indicate that the methodology is capable of supporting effective and efficient development of high performing haptic devices. However, more test cases are needed to further validate the presented methodology.     

Testing usability of multimodal applications with visually impaired children

Testing multimodal applications with visually impaired children requires specific testing methods and procedures. In this article, we used a SensAble Phantom device to produce haptic feedback together with stereo sound and visual feedback. We also experimented with low-cost haptic devices, such as force-feedback game controllers and tactile-feedback devices. Here we discuss how to conduct usability testing for multimodal applications with visually impaired children.     

Design of Wavy Ag Microwire Array for Mechanically Stable,Multimodal Vibrational Haptic Interface

A vibrotactile interface is an actuator device to convey haptic information intuitively from electronics to users. For the next‐generation of user‐friendly interface applications, the vibrotactile actuator is required to be vibration intensity/frequency controllable, mechanically stable, transparent, and have large scalability. Previously, although these requirements are satisfied via several approaches using a random network film of Ag wires or a mixture with conductive polymers, the random‐network‐based materials only have limited control on material density and uniformity, which in turn hinders precise control over vibration intensity and device transparency. Here, a new approach to assemble large‐scale Ag microwire arrays is demonstrated by involving an evaporative assembly method and is presented to overcome the current limitations. In particular, the 1D wavy structure derived from fractal designs promotes vibration intensity and cycling due to greater areal coverage and improved mechanical stability. Furthermore, by taking advantage of the precisely aligned microwires array, tunable multimode vibration frequencies are obtained by generating two different voltage frequencies. The large‐scale wavy Ag microwire array with precise spatial controllability will be directly adaptable as a user‐friendly interface in electronic applications like wearable devices, computer interfaces, and flexible mobile phones.     

Haptic display with an interface device capable of continuous-time impedance display within a sampling period

Stability of a haptic interface is an important issue in virtual reality because an operator directly touches haptic interface devices. Stability is influenced by the sampling period and the discrete-time property of the control system. For decreasing the sampling system influence, this paper proposes a haptic device with an analog circuit which is placed between the computer and the haptic device. The circuit functions as springs and dampers. The control system can specify stiffness, damping coefficients, and their equilibrium. Since the impedance generated by the electric spring and damper can work continuously within the sampling period, it is effective for making the system more stable. We also develop a control method for displaying a static virtual object with the proposed device. Further, we discuss the effects of the proposed approach analytically, using passivity analysis for a 1-degree-of-freedom display system. Finally, some experimental results in a two-dimensional virtual environment are presented.     

Design of a general purpose 6-DOF haptic interface

A general purpose six degree-of-freedom haptic device is newly designed, which can be used as a general motion commander as well as a force reflector. The structure features the large workspace and easy analysis of a serial structure and the compactness and durability of a parallel structure at the same time. The study analyzed kinematics of the proposed device to obtain the optimal device parameters in terms of kinematics and derived dynamics to form a linear control system through the feedback. In calculating the optimal design factors of the haptic device in terms of kinematics, this study employed the global isotropy index. The proposed structure has high manipulability within the workspace, which is an essential factor for a motion generator. Also the study proposed a force controller featuring an inner nonlinear loop designed to compensate the nonlinear dynamics of the device and an outer linear control loop aimed to compensate un-modeled dynamics and disturbance. For an outer linear controller, the study adopted an LQG/LTR controller that can systematically take into account stability, robustness and frequency characteristics in the design process. The performance of the proposed device was verified with its efficiency through simulation and experiments.     

Haptic holography: a primitive computational plastic

We describe our work on haptic holography, a combination of computational modeling and multimodal spatial display, which allows a person to see, feel, and interact with three-dimensional freestanding holographic images of material surfaces. In this paper, we combine various holographic displays with a force-feedback device to render multimodal images with programmatically prescribed material properties and behavior. After a brief overview of related work which situates visual display within the manual workspace, we describe our holo-haptic approach and survey three implementations, Touch, Lathe, and Poke, each named for the primitive functional affordance it offers. In Touch, static holographic images of simple geometric scenes are reconstructed in front of the hologram plane, and coregistered with a force model of the same geometry. These images can be visually inspected and haptically explored using a handheld interface. In Lathe, a holo-haptic image can be reshaped by haptic interaction in a dynamic but constrained manner. Finally in Poke, using a new technique for updating interference-modeled holographic fringe patterns, we render a holo-haptic image that permits more flexible interactive reshaping of its reconstructed surface. We situate this work within the context of related research and describe the strengths, shortcomings, and implications of our approach.     

A control-system architecture for robots used to simulate dynamicforce and moment interaction between humans and virtual objects

Haptic or kinesthetic feedback is essential in many important virtual reality and telepresence applications. Previous research focuses on simulating static forces such as those encountered when interacting with a stiff object such as a wall. Past studies usually employ custom-made devices that are not readily available to other researchers. Consequently, many of the results found in the haptic feedback literature cannot be replicated independently. With experimental results, the paper demonstrates that “off the shelf,” general purpose robotics equipment can be incorporated into an effective haptic/kinesthetic feedback system. Such a system can accommodate a wide variety of virtual reality applications including training and telerobotics. An admittance control scheme is utilized, which enables the simulation of dynamic force and moment interaction as well as contact with stiff objects. The paper shows that the mechanical deficiencies (e.g., friction, inertia, and backlash) often associated with general purpose manipulators can be overcome with a suitable control system architecture