Force modeling for tooth preparation in a dental training system

Feedback force is very important for novices to simulate tooth preparation by using the haptic interaction system (dental training system) in a virtual environment. In the process of haptic simulation, the fidelity of generated forces by a haptic device decides whether the simulation is successful. A force model computes feedback force, and we present an analytical force model to compute the force between a tooth and a dental pin during tooth preparation. The force between a tooth and a dental pin is modeled in two parts: (1) force to resist human’s operation and (2) friction to resist the rotation of the dental engine. The force to resist the human’s operation is divided into three parts in the coordinates that are constructed on the bottom center of the dental pin. In addition, we also consider the effects of dental-pin type, tooth stiffness, and contact geometry in the force model. To determine the parameters of the force model, we construct a measuring system by using machine vision and a force/torque sensor to track the human’s operations and measure the forces between the dental pins and teeth. Based on the measuring results, we construct the relation between the force and the human’s operation. The force model is implemented in the prototype of a dental training system that uses the Phantom as the haptic interface. Dentists performing virtual operations have confirmed the fidelity of feedback force.     

The impact of haptic augmentation on middle school students’ conceptions of the animal cell

Of the five sensory channels—sight, sound, taste, smell, and touch, it is only our sense of touch that enables us to modify and manipulate the world around us. This article reports the preliminary findings of a systematic study investigating the efficacy of adding haptic feedback to a desktop virtual reality program for use in middle school science instruction. Current technology allows for the simulation of tactile and kinesthetic sensations via sophisticated haptic devices and a computer interface. This research, conducted with 80 middle school students, examined the cognitive and affective impact of this technology on students’ understandings of the structure and function of an animal cell. The results of this work offer valuable insights into the theoretical and practical considerations involved in the development and implementation of haptically augmented virtual reality instructional programs.     

A mobile platform for haptic grasping in large environments

This paper presents methodologies and technologies that are exploited to design and implement the mobile haptic grasper (MHG), i.e. an integrated system consisting of a mobile robot and two grounded haptic devices (HD) fixed on it. This system features two-point contact kinaesthetic interactions while guaranteeing full user’s locomotion in large virtual environment. The workspace of haptic interaction is indefinitely extended, and this is extremely relevant for applications such as virtual grasping, where the global workspace is typically reduced with respect to those of the single-point contact devices. Regarding software architecture, we present the Haptik Library, an open source library developed at the University of Siena which allows to uniformly access HD, that has been used to implement the MHG software.

Simultaneous remote haptic collaboration for assembling tasks

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen’s University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user’s object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.     

Collaborative virtual sculpting with haptic feedback

This paper introduces a haptic virtual environment in which two users can collaboratively sculpt a virtual clay model, working from different physical locations connected by the internet. They view their virtual sculpting tools and the clay model in 3D, feel the tool’s pressure on the clay as they work, and have their hands co-located with the view of the tool and model. Since the sculptors have independent views of the same logical environment, they can work at different zoom levels, and be in different coordinate systems, even spinning ones, at the same time. This provides them with the capability to explore new styles of collaborative creativity, working off each other’s initiative where appropriate. The system was designed to allow unrestrained, asynchronous behaviour by the collaborating sculptors. The paper describes the hardware as well as the algorithms behind the deformability of the clay surface and the communications model enabling the distance collaboration. It gives an explanation of the simple conflict resolution mechanism that haptic feedback facilitates and also reports on the results of a qualitative study into the creativity benefits of such a collaborative system.     

Design and evaluation of haptic effects for use in a computer desktop for the physically disabled

The human–computer interface remains a mostly visual environment with little or no haptic interaction. While haptics is finding inroads in specialized areas such as surgery, gaming, and robotics, there has been little work to bring haptics to the computer desktop, which is largely dominated today by the GUI/mouse relationship. The mouse as an input device, however, poses many challenges for users with physical disabilities, and it is believed that a haptically enhanced interface could have significant impact assisting in target selection. This paper presents a study intended to evaluate haptic effects used with a force feedback mouse on a computer desktop and a prediction algorithm designed to focus those effects on the desired target. Results of the experiment were partially successful and indicated future directions for improvement. The paper introduces the proposed framework and presents experimental results from targeting tasks using differing haptic effects with a group of physically disabled users.     

Design and display of enhancing information in desktop information-rich virtual environments: challenges and techniques

Information-rich virtual environments (IRVEs) have been described as environments in which perceptual information is enhanced with abstract (or symbolic) information, such as text, numbers, images, audio, video, or hyperlinked resources. Desktop virtual environment (VE) applications present similar information design and layout challenges as immersive VEs, but, in addition, they may also be integrated with external windows or frames commonly used in desktop interfaces. This paper enumerates design approaches for the display of enhancing information both internal and external to the virtual worlds render volume. Using standard Web-based software frameworks, we explore a number of implicit and explicit spatial layout methods for the display and linking of abstract information, especially text. Within the VE view, we demonstrate both heads-up-displays (HUDs) and encapsulated scenegraph behaviors we call semantic objects. For desktop displays, which support information display venues external to the scene, we demonstrate the linking and integration of the scene with Web browsers and external visualization applications. Finally, we describe the application of these techniques in the PathSim visualizer, an IRVE interface for the biomedical domain. These design techniques are relevant to instructional and informative interfaces for a wide variety of VE applications.     

Virtual prototyping of a car turn-signal switch using haptic feedback

The present work describes the virtual prototyping procedure of car turn-signal switches using a dedicated haptic feedback system. Although turn-signal switches have mechanisms with simple geometry, simulating the “feeling” during its operation is still a challenge. In order to allow ergonomics designers to directly assess the comfort of operation, a dedicated haptic system was developed for reproducing the force feedback that would characterize a real turn-signal switch. For the accurate haptic feedback generation and rendering purpose, a detailed computer-aided design model of the switch was developed together with a multibody simulation. For the design purpose a novel method for the synthesis of the cam mechanism was also developed, starting from an imposed force profile that was previously validated on the haptic device, for which the cam geometry is determined. Thus, the proposed approach enables the designer to also design the “feel” of the switch, and directly manufacture a good model, simplifying the usual iterative process necessary to implement the desired force profile. Finally, an example of virtual prototyping of a turn-signal switch is presented demonstrating the usability of the proposed method and the haptic system.     

The Responsive Workbench [virtual work environment]

The Responsive Workbench concept was developed as an alternative to the multimedia and virtual reality systems of the past decade. It was recognized that almost nobody wants simulations of their working worlds in a desktop environment. Generally. users want to focus on their tasks rather than on operating the computer. The Responsive Workbench is a virtual working environment that locates virtual objects and control tools on a real “workbench”. The objects-computer-generated stereo images-are projected onto the surface of a workbench. This setting corresponds to the actual work situation in an architect's office, in surgery, and so forth. A human guide uses the virtual working environment while several collaborators watch events through stereo shutter glasses. The participants operate within a nonimmersive virtual environment. Depending on the application, the virtual workbench can integrate various input and output modules, such as motion, gesture, and voice recognition systems. This characterizes the general trend away from the classical human-machine interface. Several guides can work together in similar environments either locally or by using broadband communication networks. A responsive environment, consisting of powerful graphics workstations, tracking systems, cameras, projectors, and microphones, replaces the traditional multimedia desktop workstation     

Experiments in haptic-based authentication of humans

With the rapid advancement of the technological revolution, computer technology such as faster processors, advanced graphic cards, and multi-media systems are becoming more affordable. Haptics technology is a force/tactile feedback technology growing in disciplines linked to human–computer interaction. Similar to the increasing complexity of silicon-based components, haptics technology is becoming more advanced. On the other hand, currently available commercial haptics interfaces are expensive, and their application is mostly dedicated to enormous research projects or systems. However, the trend of the market is forcing haptic developers to release products for use in conjunction with current keyboards and mice technologies. Haptics allows a user to touch, fell, manipulate, create, and/or alter simulated three-dimensional objects in a virtual environment. Most of the existing applications of haptics are dedicated to hone human physical skills such as sensitive hardware repair, medical procedures, handling hazardous substances, etc. These skills can be trained in a realistic virtual world, and describe human behavioural patterns in human–computer interaction environments. The measurement of such psychomotor patterns can be used to verify a person’s identity by assessing unique-to-the-individual behavioural attributes. This paper explores the unique behaviour exhibited by different users interacting with haptic systems. Through several haptic-based applications, users’ physical attributes output data from the haptic interface for use in the construction of a biometric system.

Multimodal object oriented user interfaces in mobile affective interaction

In this paper, we investigate an object oriented (OO) architecture for multimodal emotion recognition in interactive applications through mobile phones or handheld devices. Mobile phones are different from desktop computers since mobile phones are not performing any processing involving emotion recognition whereas desktop computers can perform such processing. In fact, in our approach, mobile phones have to pass all data collected to a server and then perform emotion recognition. The object oriented architecture that we have created, combines evidence from multiple modalities of interaction, namely the mobile device’s keyboard and the mobile device’s microphone, as well as data from emotion stereotypes. Moreover, the OO method classifies them into well structured objects with their own properties and methods. The resulting emotion detection server is capable of using and handling transmitted information from different mobile sources of multimodal data during human-computer interaction. As a test bed for the affective mobile interaction we have used an educational application that is incorporated into the mobile system.     

Passive internet-based crane teleoperation with haptic aids

Crane operation is a challenging task, due to the combined problem of obstacle avoidance and load swing suppression in underactuated conditions. This paper presents a human-machine interface that increases the operator’s perception of a gantry crane’s workspace. With this aim, a virtual environment resembling the workspace is connected with a haptic device. This allows the user to receive not only visual but also tactile feedback, thus increasing maneuvering safety. Additionally, this capability is integrated in a teleoperation setup, adopting a passivity-based control approach that guarantees overall stability. This includes also the design of controllers by means of the IDA-PBC method. Experimental results carried out with a laboratory crane show its feasibility for internet-based teleoperation and demonstrate the improvements on the system performance.     

Sensory-motor enhancement in a virtual therapeutic environment

The sensory-motor skills of persons with neuromuscular disabilities have been shown to be enhanced by intensive and repetitive therapeutic interventions. This paper describes a form of low immersion virtual reality and a prototype, open source system that allow a user with significant physical disability to actively interact with computer-generated objects whose behaviors promote a game-like interaction. Unlike fully immersive and haptic virtual reality, this approach frees the user from head-mounted displays and gloves. It extracts the user’s real-time silhouette from the output of a remote video camera and uses that two-dimensional outline to interact with graphical objects on screen. In contrast to video games that have been modified with specialized interfaces, this virtual interaction system promotes the repetitive use of goal directed movements of the arms and body, which are essential to promote cortical reorganization, as well as discourage unwanted changes in muscle tissue that result in contracture. A prototype system demonstrates the potential of low immersion technology to motivate users and encourage participation in therapy. It also offers the potential of accommodating the sensory-motor skills of individuals with very significant impairment. The behaviors of the computer-generated graphics can be altered to allow use by those with very limited range of motion and/or motor control. These behaviors can be adjusted to provide a continuing challenge as the user’s skills improve. This prototype is described in terms of functional capabilities that include a silhouette extraction from a video image, and generation of graphical objects that interact with the silhouette. The work is extended with a discussion of a more sophisticated region of interest detection algorithm that can select specific parts of the body.     

Perception-based lossy haptic compression considerations for velocity-based interactions

The ability of technology to transmit multi-media is very dependent on compression techniques. In particular lossy compression has been used in image compression (jpeg) audio compression (mp3) and video compression (mpg) to allow the transmission of audio and video over broadband network connections. Recently the sense of touch or haptics is becoming more important with its addition in computer games or in cruder applications such as vibrations in a cell phone. As haptic technology improves the ability to transmit compressed force sensations becomes more critical. Most lossy audio and visual compression techniques rely on the lack of sensitivity in humans to pick up detailed information in certain scenarios. Similarly limitations in the sensitivity of human touch could be exploited to create haptic models with much less detail and thus requiring smaller bandwidth. The focus of this paper is on the force thresholds of the human haptic system that can be used in a psychophysically motivated lossy haptic (force) compression technique. Most of the research in this field has measured the just noticeable difference (JND) of the human haptic system with a human user in static interaction with a stationary rigid object. In this paper our focus involves cases where the human user or the object are in relative motion. An example of such an application would be the haptic rendering of the user’s hand in contact with of a high-viscous material or interacting with a highly deformable object. Thus an approach is presented to measure the force threshold based on the velocity of the user’s hand motion. Two experiments are conducted to detect the absolute force threshold (AFT) of the human haptic system using methodologies from the field of psychophysics. The AFTs are detected for three different ranges of velocity of the user’s hand motion. This study implies that when a user’s hand is in motion fewer haptic details are required to be stored calculated or transmitted. Finally the implications of this study on a more complete future study will be discussed.

Beyond the visuals: tactile augmentation and sensory enhancement in an arthroscopy simulator

This paper considers tactile augmentation, the addition of a physical object within a virtual environment (VE) to provide haptic feedback. The resulting mixed reality environment is limited in terms of the ease with which changes can be made to the haptic properties of objects within it. Therefore sensory enhancements or illusions that make use of visual cues to alter the perceived hardness of a physical object allowing variation in haptic properties are considered. Experimental work demonstrates that a single physical surface can be made to ‘feel’ both softer and harder than it is in reality by the accompanying visual information presented. The strong impact visual cues have on the overall perception of object hardness, indicates haptic accuracy may not be essential for a realistic virtual experience. The experimental results are related specifically to the development of a VE for surgical training; however, the conclusions drawn are broadly applicable to the simulation of touch and the understanding of haptic perception within VEs.     

Stroke-based modeling and haptic skill display for Chinese calligraphy simulation system

The goal of this paper is to study haptic skill representation and display in a Chinese calligraphy training system. The challenge is to model haptic skill during the writing of different strokes in Chinese characters and to achieve haptic rendering with high fidelity and stability. The planning of the writing process is organized at three levels: task, representation and device level to describe the haptic handwriting skill. State transition graph (STG) is proposed to describe switches between tasks during the handwriting. Chinese characters are modeled using 39 typical strokes, which are further grouped into basic and compound strokes. The compound stroke is considered to be sequential combination of the basic strokes. Straight and curve strokes are modeled using line segment and the Bezier curve, respectively. Information from STG is used for real-time collision detection and haptic rendering. Ambiguity of the collision detection at stroke-corner points is prevented using active stroke combined with local nearest point computation. A modified virtual fixture method is developed for haptic rendering. The approach is tested on a prototype training system using Phantom desktop. Initial experiments suggest that the proposed modeling and rendering method is effective.     

Influences of haptic communication on a shared manual task

With the advent of new haptic feedback devices, researchers are giving serious consideration to the incorporation of haptic communication in collaborative virtual environments. For instance, haptic interactions based tools can be used for medical and related education whereby students can train in minimal invasive surgery using virtual reality before approaching human subjects. To design virtual environments that support haptic communication, a deeper understanding of humans′ haptic interactions is required. In this paper, human′s haptic collaboration is investigated. A collaborative virtual environment was designed to support performing a shared manual task. To evaluate this system, 60 medical students participated to an experimental study. Participants were asked to perform in dyads a needle insertion task after a training period. Results show that compared to conventional training methods, a visual-haptic training improves user′s collaborative performance. In addition, we found that haptic interaction influences the partners′ verbal communication when sharing haptic information. This indicates that the haptic communication training changes the nature of the users′ mental representations. Finally, we found that haptic interactions increased the sense of copresence in the virtual environment: haptic communication facilitates users′ collaboration in a shared manual task within a shared virtual environment. Design implications for including haptic communication in virtual environments are outlined.