Hybrid client-server architecture and control techniques for collaborative product development using haptic interfaces

In this paper, a collaborative product development and prototyping framework is proposed by using distributed haptic interfaces along with deformable objects modeling. Collaborative Virtual Environment (CVE) is a promising technique for industrial product development and virtual prototyping. Network control problems such as network traffic and network delay in communication have greatly limited collaborative virtual environment applications. The problems become more difficult when high-update-rate haptic interfaces and computation intensive deformable objects modeling are integrated into CVEs for intuitive manipulation and enhanced realism. A hybrid network architecture is proposed to balance the computational burden of haptic rendering and deformable object simulation. Adaptive artificial time compensation is used to reduce the time discrepancy between the server and the client. Interpolation and extrapolation approaches are used to synchronize graphic and haptic data transmitted over the network. The proposed techniques can be used for collaborative product development, virtual assembly, remote product simulation and other collaborative virtual environments where both haptic interfaces and deformable object models are involved.     

Neuro-cognitively inspired haptic user interfaces

Haptic systems and devices are a recent addition to multimodal systems. These devices have widespread applications such as surgical simulations, medical and procedural training, scientific visualizations, assistive and rehabilitative devices for individuals who have physical or neurological impediments and assistive devices for individuals who are blind. While the potential of haptics in natural human machine interaction is undisputable, the realization of such means is still a long way ahead. There are considerable research challenges to development of natural haptic interfaces. The study of human tactile abilities is a recent endeavor and many of the available systems still do not incorporate the domain knowledge of psychophysics, biomechanics and neurological elements of haptic perception. Development of smart and effective haptic interfaces and devices requires extensive studies that link perceptual phenomena with measurable parameters and incorporation of such domain knowledge in the engineering of haptic interfaces. This paper presents design, development and usability testing of a neuro-cognitively inspired haptic user interface for individuals who are blind. The proposed system design is inspired by neuro-cognitive basis of haptic perception and incorporates the computational aspects and requirements of multimodal information processing system. Usability testing of the system suggests that a biologically inspired haptic user interfaces may form a powerful paradigm for haptic user interface design.

Control systems of haptic man-machine interfaces

Methods for construction of stable control systems for the mechanism of haptic interaction used in haptic man-machine interfaces are considered. These interfaces reconstruct realistic haptic perception of a virtual object by a man. Two approaches to the control synthesis for the interaction mechanism are proposed and analyzed. The first one is based on tracking the open-loop trajectory of motion of the virtual object (body), and the second one is based on tracking the force of interaction with this object. Unlike known papers, the analysis takes into account the nonlinearity of the control system. The analysis is performed for two possible interaction regimes: “pushing” and “holding” of the virtual object. Stability conditions that are easily implemented are obtained.     

Microcontroller-operated anthropomorphic manipulator with haptic feedback

This article presents the design of a cluster-distributed computing system to operate and control an anthropomorphic manipulator with haptic feedback. The system hardware consists of four major building blocks, anthropomorphic manipulator, haptic sensor glove, real-time microcontroller-based cluster distributed computing (CDC) network, and closed-loop controller. The system software architecture consists of three key operation modules, namely, the sensor glove, anthropomorphic manipulator and communication module. It is designed based on a shared file system concept which forms the foundation of the CDC protocol. The key advantages of this system are its novel structure and its comparatively low cost using microcontroller-based distributed cluster networks which allow large-scale expansion/integration.     

Collaborative virtual sculpting with haptic feedback

This paper introduces a haptic virtual environment in which two users can collaboratively sculpt a virtual clay model, working from different physical locations connected by the internet. They view their virtual sculpting tools and the clay model in 3D, feel the tool’s pressure on the clay as they work, and have their hands co-located with the view of the tool and model. Since the sculptors have independent views of the same logical environment, they can work at different zoom levels, and be in different coordinate systems, even spinning ones, at the same time. This provides them with the capability to explore new styles of collaborative creativity, working off each other’s initiative where appropriate. The system was designed to allow unrestrained, asynchronous behaviour by the collaborating sculptors. The paper describes the hardware as well as the algorithms behind the deformability of the clay surface and the communications model enabling the distance collaboration. It gives an explanation of the simple conflict resolution mechanism that haptic feedback facilitates and also reports on the results of a qualitative study into the creativity benefits of such a collaborative system.     

Methods for haptic feedback in teleoperated robot-assisted surgery

Teleoperated minimally invasive surgical robots can significantly enhance a surgeon's accuracy, dexterity and visualization. However, current commercially available systems do not include significant haptic (force and tactile) feedback to the operator. This paper describes experiments to characterize this problem, as well as several methods to provide haptic feedback in order to improve surgeon's performance. There exist a variety of sensing and control methods that enable haptic feedback, although a number of practical considerations, e.g. cost, complexity and biocompatibility, present significant challenges. The ability of teleoperated robot-assisted surgical systems to measure and display haptic information leads to a number of additional exciting clinical and scientific opportunities, such as active operator assistance through "virtual fixtures" and the automatic acquisition of tissue properties.     

Incorporating haptic feedback in simulation for learning physics

The purpose of this study was to investigate the effectiveness of a haptic augmented simulation in learning physics. The results indicate that haptic augmented simulations, both the force and kinesthetic and the purely kinesthetic simulations, were more effective than the equivalent non-haptic simulation in providing perceptual experiences and helping elementary students create multimodal representations of the movements of gears. However, in most cases, force feedback was needed to construct a fully loaded multimodal representation that helps students to comprehend later instruction with less sensory modalities. In addition, the force and kinesthetic simulation was effective in helping to transfer knowledge to new learning situations. These findings suggest that it is important to help elementary students make a solid cognitive grounding with the use of a perceptual anchor.     

Improving contact realism through event-based haptic feedback

Tapping on surfaces in a typical virtual environment feels like contact with soft foam rather than a hard object. The realism of such interactions can be dramatically improved by superimposing event-based, high-frequency transient forces over traditional position-based feedback. When scaled by impact velocity, hand-tuned pulses and decaying sinusoids produce haptic cues that resemble those experienced during real impacts. Our new method for generating appropriate transients inverts a dynamic model of the haptic device to determine the motor forces required to create prerecorded acceleration profiles at the user's fingertips. After development, the event-based haptic paradigm and the method of acceleration matching were evaluated in a carefully controlled user study. Sixteen individuals blindly tapped on nine virtual and three real samples, rating the degree to which each felt like real wood. Event-based feedback achieved significantly higher realism ratings than the traditional rendering method. The display of transient signals made virtual objects feel similar to a real sample of wood on a foam substrate, while position feedback alone received ratings similar to those of foam. This work provides an important new avenue for increasing the realism of contact in haptic interactions.     

Influence of multisensory feedback on haptic accessibility tasks

Environments of a certain nature, such as those related to maintenance tasks can benefited from haptic stimuli by performing accessibility simulation in a realistic manner. Accessibility is defined as the physical feasibility of accessing an element of a 3D model avoiding undesirable collisions. This paper studies the benefits that multisensory systems can provide in performing this kind of tasks. The research is specially focused on the improvements provided by auditory feedback to the user’s performance. We have carried out a user study where participants had to perform an accessibility task with the aid of different combinations of sensorial stimuli. A large haptic interface for aeronautic maintainability has been extended with real-time sound generation capabilities to study this issue. The results of these experiments show that auditory stimuli provide with useful cues to the users helping them to correct trajectories and hence improving their performance.     

A compliant self-adaptive gripper with proprioceptive haptic feedback

Grippers and robotic hands are an important field in robotics. Recently, the combination of grasping devices and haptic feedback has been a promising avenue for many applications such as laparoscopic surgery and spatial telemanipulation. This paper presents the work behind a new self-adaptive, a.k.a. underactuated, gripper with a proprioceptive haptic feedback in which the apparent stiffness of the gripper as seen by its actuator is used to estimate contact location. This system combines many technologies and concepts in an integrated mechatronic tool. Among them, underactuated grasping, haptic feedback, compliant joints and a differential seesaw mechanism are used. Following a theoretical modeling of the gripper based on the virtual work principle, the authors present numerical data used to validate this model. Then, a presentation of the practical prototype is given, discussing the sensors, controllers, and mechanical architecture. Finally, the control law and the experimental validation of the haptic feedback are presented.     

The moderating role of sensation seeking tendency in robotic haptic interfaces

This study examined the interaction effects between haptic force feedback and users’ sensation seeking tendency (i.e. need for sensations) on users’ feelings of presence (i.e. the state in which users experience virtual objects and virtual environments as if they were actual) in robotic haptic interfaces. Users with low sensation seeking tendency felt stronger physical presence and spatial presence in response to force feedback haptic stimuli (versus no force feedback), whereas users with high sensation seeking tendency did not show any difference between the two conditions, thus confirming the moderating role of the users’ sensation seeking tendency in the robotic haptic interface. Theoretical implications for human–computer interaction (HCI) research and managerial implications for the interactive media market are discussed.     

On the enhanced passivity of pneumatically actuated impedance-type haptic interfaces

The stable simulation of high-stiffness surfaces remains a challenge in impedance-type haptic simulations of mechanical environments. In this paper, the authors propose an approach to achieving a stable, high-stiffness surface in a haptic interface by leveraging the open-loop properties of pneumatic actuators. By using the open-loop component of the actuator stiffness as a primary component of stiffness simulation in a haptic interface, the system requires a comparatively small component of simulated stiffness from the closed-loop control of the actuator. A passivity analysis is presented describing how the presence of an open-loop stiffness enhances the range of passivity of haptically simulated high-stiffness surfaces. Experimental results both with and without a human operator are presented that demonstrate the effectiveness of the approach and its enhanced passivity relative to motor-actuated devices.     

Whole-hand kinesthetic feedback and haptic perception in dextrous virtual manipulation

One of the key requirements for a Virtual Reality system is the multimodal, real-time interaction between the human operator and a computer simulated and animated environment. This paper investigates problems related particularly to the haptic interaction between the human operator and a virtual environment. The work presented here focuses on two issues: 1) the synthesis of whole-hand kinesthetic feedback, based on the application of forces (torques) on individual phalanges (joints) of the human hand, and 2) the experimental evaluation of this haptic feedback system, in terms of human haptic perception of virtual physical properties (such as the weight of a virtual manipulated object), using psychophysical methods. The proposed kinesthetic feedback methodology is based on the solution of a generalized force distribution problem for the human hand during virtual manipulation tasks. The solution is computationally efficient and has been experimentally implemented using an exoskeleton force-feedback glove. A series of experiments is reported concerning the perception of weight of manipulated virtual objects and the obtained results demonstrate the feasibility of the concept. Issues related to the use of sensory substitution techniques for the application of haptic feedback on the human hand are also discussed.     

Augmenting graphical user interfaces with haptic assistance for motion-impaired operators

Haptic assistance is an emerging field of research that is designed to improve human–computer interaction (HCI) by reducing error rates and targeting times through the use of force feedback. Haptic feedback has previously been investigated to assist motion-impaired computer users, however, limitations such as target distracters have hampered its integration with graphical user interfaces (GUIs). In this paper two new haptic assistive techniques are presented that utilise the 3DOF capabilities of the Phantom Omni. These are referred to as deformable haptic cones and deformable virtual switches. The assistance is designed specifically to enable motion-impaired operators to use existing GUIs more effectively. Experiment 1 investigates the performance benefits of the new haptic techniques when used in conjunction with the densely populated Windows on-screen keyboard (OSK). Experiment 2 utilises the ISO 9241-9 point-and-click task to investigate the effects of target size and shape. The results of the study prove that the newly proposed techniques improve interaction rates and can be integrated with existing software without many of the drawbacks of traditional haptic assistance. Deformable haptic cones and deformable virtual switches were shown to reduce the mean number of missed-clicks by at least 75% and reduce targeting times by at least 25%.     

Workspace analysis for haptic feedback manipulator in virtual cockpit system

To obtain natural space experience of haptic interaction for users in virtual cockpit systems (VCS), a haptic feedback system and a workspace analysis framework for haptic feedback manipulator (HFM) are presented in this paper. Firstly, improving the classical three-dimensional workspace obtained by the Monte Carlo method, a novel workspace representation method, oriented workspace, is presented, which can indicate both the position and the orientation of the end-effector. Then, aimed at the characters of HFMs, the oriented workspace is divided into the effective workspace and the prohibited area by extracting the control panel area. At last, the effective workspace volume and the control panel area are calculated by the double-directed extremum method, with the accuracy improved by repeatedly adding and extracting boundary points. By simulation, the area in which interactions between the manipulator and users hand performed is determined and accordingly the effective workspace along with its boundary and volume are obtained in a relative high precision, which lay a basis for haptic interaction in VCS.     

Optimizing low-order controllers for haptic systems under delayed feedback

In this paper, a PD controller design for haptic systems under delayed feedback is considered. More precisely, a complete stability analysis of a haptic system where local dynamics are described by some second-order mechanical dynamics is presented. Next, using two optimization techniques ($H_{\infty }$ and stability, margin optimization) an optimal choice for the controller gains is proposed. The derived results are tested on a three degree-of-freedom real-time experimental platform to illustrate the theoretical results.     

A teleoperation system for micro positioning with haptic feedback

This paper presents the research work on a 1 Degree of Freedom (DOF) macro-micro teleoperation system which enables human operator to perform complex task in micro environment such as cell insertion with the capability of haptic feedback. To reach submicron resolution, a nano-motion piezo actuator was used as the slave robot and a servo DC motor was used as the master robot. Force sensors were implemented at both ends for haptic feedback and a microscope equipped with camera was employed for real-time visual feedback. The hysteresis nonlinearity of the piezo motor was modeled using LuGre friction model and compensated for. A Sliding Mode Based Impedance Controller (SMBIC) was designed at the slave side to ensure position tracking while an impedance force controller was designed at the master side to ascertain tracking of the force. Control parameters were chosen based on Llewellyn stability criteria such that the entire system stays stable against parameter uncertainties and constant time delay. The experimental results demonstrated capability of the proposed control frameworks in desirable tracking of the position and force signals while the entire system remained stable. The results of this study can be used for complex tasks in micron environment such as cell insertion.