New colonoscopy simulator with improved haptic fidelity

Colonoscopy is a safe and effective procedure to diagnose and treat the large bowel with the help of the flexible endoscope. This paper presents a new colonoscopy training simulator to help trainees practice and acquire the necessary skills and experiences with no risk to the patients and possibly less cost. The simulator includes a specialized haptic interface to transfer force feedback through a long and flexible tube, and graphics algorithms to display the virtual colon realistically while managing the large number of polygons. A new 2-d.o.f. haptic device with folding guides is developed to transmit large decoupled forces of the colonoscopy simulation to the user. The physicians apply a jiggling motion to the colonoscopy tube to advance the scope. This jiggling is an important skill of colonoscopy and is incorporated for the first time by using the new sensor mechanism. A colonoscope handle that shares the look, feel and functions with an actual colonoscope is developed with the necessary electronics inside. The simulator contains controllers to compensate for the inertia and friction effects, and is evaluated by physicians. New graphics algorithms including polygon reduction, navigation and collision detection are developed to compute the deformation and the corresponding reflective force in real-time.     

Design of a Haptic Interface for a Gastrointestinal Endoscopy Simulation

This paper presents a new design and analysis of a haptic interface for a gastrointestinal endoscopy simulation. The gastrointestinal endoscopy is a procedure in which the digestive tract and organs of a patient are diagnosed and treated using a long and flexible endoscope. The developed haptic interface incorporates two degrees of freedom (DOF), each of which is necessary to describe the movements of an endoscope during the actual endoscopy procedures. The haptic interface has a translational motion mechanism to implement the insertion movement of the endoscope, and a rotational motion mechanism to implement the rotational movement of the endoscope. The endoscope included in the haptic interface is supported by a folding guide to prevent the endoscope from buckling. Force feedback in each direction is provided by wire-driven mechanisms. The developed haptic interface has a workspace, sensitivity, and maximum attainable force and torque enough to simulate the endoscopy procedures such as colonoscopy, upper GI (gastrointestinal) endoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). The developed haptic interface is applied to implementation of a colonoscopy simulation. Performance including force bandwidth is evaluated through experiments and simulation.     

Development of an inheritance assist system for experiencing operation skills by using a haptic function of PHANToM

At present, several commercially available surgical simulators have been used in advanced medical facilities, and most of them provide a self-learning environment for trainees who are young doctors and/or residents. Therefore, we assume that inexperienced doctors have to inherit the veteran skills of experienced doctors because the lack of doctors and nurses is one of the most serious medical issues. This article proposes a training function on a surgical simulator system which enables trainees to sense the feelings of experienced surgeons while operating. The inheritance assistance function we are constructing might be of help in compensating for the lack of experience of young doctors and be more effective than the present system. The inheritance assistance function makes it possible to record the operational data of experienced doctors, and to reproduce this data in a training scene for young trainees. Then young doctors could practice their own skills by intuitively referencing the recorded advisory data in a surgical simulator. In order to accomplish the purpose of this study, we have constructed a fundamental system. A simple virtual object based on a dynamic model reacts in terms of visualization and force against the manipulation of virtual forceps via a haptic device, PHANToM. Several laboratory students of the experimental subject were challenged to obtain training with the system developed. This article describes the system and discusses the results and future work.     

A Precomputed Approach for Real-Time Haptic Interaction with Fluids

The key to enhancing perception of the virtual world is improving mechanisms for interacting with that world. Through providing a sense of touch, haptic rendering is one such mechanism. Many methods efficiently display force between rigid objects, but to achieve a truly realistic virtual environment, haptic interaction with fluids is also essential. In the field of computational fluid dynamics, researchers have developed methods to numerically estimate the resistance due to fluids by solving complex partial differential equations, called the Navier-Stokes equations. However, their estimation techniques, although numerically accurate, are prohibitively time-consuming. This becomes a serious problem for haptic rendering, which requires a high frame rate. To address this issue, we developed a method for rapidly estimating and displaying forces acting on a rigid virtual object due to water. In this article, we provide an overview of our method together with its implementation and two applications: a lure-fishing simulator and a virtual canoe simulator     

Computer-based virtual reality simulator for phacoemulsification cataract surgery training

Recent research in virtual reality indicates that computer-based simulators are an effective technology to use for surgeons learning to improve their surgical skills in a controlled environment. This article presents the development of a virtual reality simulator for phacoemulsification cataract surgery training, which is the most common surgical technique currently being used to remove cataracts from the patient’s eyes. The procedure requires emulsifying the cloudy natural lens of the eye and restoring vision by implanting an artificial lens through a small incision. The four main procedures of cataract surgery, namely corneal incision, capsulorhexis, phacoemulsification, and intraocular lens implantation, are incorporated in the simulator for virtual surgical training by implementing several surgical techniques. The surgical activity that are applied on the anatomy of the human eye, such as incision, grasping, tearing, emulsification, rotation, and implantation, are simulated in the system by using different types of mesh modifications. A virtual reality surgical simulator is developed, and the main procedures of phacoemulsification cataract surgery are successfully simulated in the system. The simulation results of the training system show that the developed simulator is capable of generating a virtual surgical environment with faithful force feedback for medical residents and trainees to conduct their training lessons via the computer using a pair of force-feedback haptic devices. In addition, the successful simulation of the mesh modifications on the human eyeball with visual realism and faithful force feedback throughout the surgical operation shows that the developed simulator is able to serve as a virtual surgical platform for surgeons to train their surgical skills.     

Enhancing Realism of Wet Surfaces in Temporal Bone Surgical Simulation

We present techniques to improve visual realism in an interactive surgical simulation application: a mastoidectomy simulator that offers a training environment for medical residents as a complement to using a cadaver. As well as displaying the mastoid bone through volume rendering, the simulation allows users to experience haptic feedback and appropriate sound cues while controlling a virtual bone drill and suction/irrigation device. The techniques employed to improve realism consist of a fluid simulator and a shading model. The former allows for deformable boundaries based on volumetric bone data, while the latter gives a wet look to the rendered bone to emulate more closely the appearance of the bone in a surgical environment. The fluid rendering includes bleeding effects, meniscus rendering, and refraction. We incorporate a planar computational fluid dynamics simulation into our three-dimensional rendering to effect realistic blood diffusion. Maintaining real-time performance while drilling away bone in the simulation is critical for engagement with the system.     

Influences of haptic communication on a shared manual task

With the advent of new haptic feedback devices, researchers are giving serious consideration to the incorporation of haptic communication in collaborative virtual environments. For instance, haptic interactions based tools can be used for medical and related education whereby students can train in minimal invasive surgery using virtual reality before approaching human subjects. To design virtual environments that support haptic communication, a deeper understanding of humans′ haptic interactions is required. In this paper, human′s haptic collaboration is investigated. A collaborative virtual environment was designed to support performing a shared manual task. To evaluate this system, 60 medical students participated to an experimental study. Participants were asked to perform in dyads a needle insertion task after a training period. Results show that compared to conventional training methods, a visual-haptic training improves user′s collaborative performance. In addition, we found that haptic interaction influences the partners′ verbal communication when sharing haptic information. This indicates that the haptic communication training changes the nature of the users′ mental representations. Finally, we found that haptic interactions increased the sense of copresence in the virtual environment: haptic communication facilitates users′ collaboration in a shared manual task within a shared virtual environment. Design implications for including haptic communication in virtual environments are outlined.     

Improving realism of a surgery simulator: linear anisotropic elasticity,complex interactions and force extrapolation

In this article, we describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. The goal of this simulator is to provide a realistic training test bed to perform laparoscopic procedures. Therefore, its main functionality is to simulate the action of virtual laparoscopic surgical instruments for deforming and cutting tridimensional anatomical models. Throughout this paper, we present the general features of this simulator including the implementation of several biomechanical models and the integration of two force‐feedback devices in the simulation platform. More precisely, we describe three new important developments that improve the overall realism of our simulator. First, we have developed biomechanical models, based on linear elasticity and finite element theory, that include the notion of anisotropic deformation. Indeed, we have generalized the linear elastic behaviour of anatomical models to ‘transversally isotropic’ materials, i.e. materials having a different behaviour in a given direction. We have also added to the volumetric model an external elastic membrane representing the ‘liver capsule’, a rather stiff skin surrounding the liver, which creates a kind of ‘surface anisotropy’. Second, we have developed new contact models between surgical instruments and soft tissue models. For instance, after detecting a contact with an instrument, we define specific boundary constraints on deformable models to represent various forms of interactions with a surgical tool, such as sliding, gripping, cutting or burning. In addition, we compute the reaction forces that should be felt by the user manipulating the force‐feedback devices. The last improvement is related to the problem of haptic rendering. Currently, we are able to achieve a simulation frequency of 25 Hz (visual real time) with anatomical models of complex geometry and behaviour. But to achieve a good haptic feedback requires a frequency update of applied forces typically above 300 Hz (haptic real time). Thus, we propose a force extrapolation algorithm in order to reach haptic real time. Copyright © 2002 John Wiley & Sons, Ltd.     

Stable haptic interaction using a damping model to implement a realistic tooth-cutting simulation for dental training

It is difficult to implement a stable and realistic haptic simulation for cutting rigid objects that is based on a damping model because of an inevitable conflict between stability and high output force. This paper presents passivity techniques to show that an excessive damping coefficient causes the output stiffness to exceed the maximum output stiffness of the haptic device, leading to instability. By analysing the damping model of a haptic dental-training simulator, we construct a relationship among the damping coefficient, position resolution, sampling frequency, human operation, and the maximum achievable device stiffness that will still maintain device stability. A method is also provided to restrict the output stiffness of the haptic device to ensure stability while enabling the realistic haptic simulation of cutting rigid objects (teeth) that is based in a damping model. Our analysis and conclusions are verified by a damping model that is constructed for a dental-training haptic display. Three types of haptic devices are used in our analysis and experiments.     

Stroke-based modeling and haptic skill display for Chinese calligraphy simulation system

The goal of this paper is to study haptic skill representation and display in a Chinese calligraphy training system. The challenge is to model haptic skill during the writing of different strokes in Chinese characters and to achieve haptic rendering with high fidelity and stability. The planning of the writing process is organized at three levels: task, representation and device level to describe the haptic handwriting skill. State transition graph (STG) is proposed to describe switches between tasks during the handwriting. Chinese characters are modeled using 39 typical strokes, which are further grouped into basic and compound strokes. The compound stroke is considered to be sequential combination of the basic strokes. Straight and curve strokes are modeled using line segment and the Bezier curve, respectively. Information from STG is used for real-time collision detection and haptic rendering. Ambiguity of the collision detection at stroke-corner points is prevented using active stroke combined with local nearest point computation. A modified virtual fixture method is developed for haptic rendering. The approach is tested on a prototype training system using Phantom desktop. Initial experiments suggest that the proposed modeling and rendering method is effective.     

带有力感觉的显微外科手术血管的仿真研究

随着虚拟现实技术的进步,带有力感觉的仿真研究迅速发展,特别是在医学上得到了广泛应用。利用虚拟现实技术取代效率低下的常规培训,可以大大减少所需的培训时间和昂贵的动物实验费用。针对复杂的显微外科手术的血管缝合,作者建立了可用于大夫培训、手术规划的虚拟现实系统。其中的交互设备采用了具有力反馈功能的PHANTOM—Desktop,使用与之配套的GHOSTSDK软件开发包,建立虚拟环境。考虑到柔性体仿真的实时性和真实性要求,采用了质量一弹簧模型的建模方法。在完成了对虚拟环境中柔性平面仿真的基础上,成功实现了带有力感觉的显微外科血管穿刺的虚拟仿真。     

Improving virtual reality navigation tasks using a haptic vest and upper body tracking

Haptic feedback is an important component of immersive virtual reality (VR) applications that is often suggested to complement visual information through the sense of touch. This paper investigates the use of a haptic vest in navigation tasks. The haptic vest produces a repulsive vibrotactile feedback from nearby static virtual obstacles that augments the user spatial awareness. The tasks require the user to perform complex movements in a 3D cluttered virtual environment, like avoiding obstacles while walking backwards and pulling a virtual object. The experimental setup consists of a room-scale environment. Our approach is the first study where a haptic vest is tracked in real time using a motion capture device so that proximity-based haptic feedback can be conveyed according to the actual movement of the upper body of the user.User study experiments have been conducted with and without haptic feedback in virtual environments involving both normal and limited visibility conditions. A quantitative evaluation was carried out by measuring task completion time and error (collision) rate. Multiple haptic rendering techniques have also been tested. Results show that under limited visibility conditions proximity-based haptic feedback generated by a wearable haptic vest can significantly reduce the number of collisions with obstacles in the virtual environment.     

The design and testing of a force feedback dental simulator

The Iowa Dental Surgical Simulator is a haptic simulator to train dental students in the haptic skills of dentistry. The initial design emphasizes the detection of carious lesions. This work describes the software and implementation of the prototype system, the design tradeoffs' and the technical issues associated with haptic and graphics subsystems. The work also describes the current system performance, including a formal evaluation by practicing dentists and performance measures. A discussion of the limitations of the current system is followed by an analysis of opportunities to improve the quality of the simulator. The results should be of interest to designers of medical haptic simulation systems and other simulation designers.     

Haptically integrated simulation of a finite element model of thoracolumbar spine combining offline biomechanical response analysis of intervertebral discs

In this paper, we first describe the construction of a finite element model of the human spine that may be used to assist the investigation of clinical problems by predicting its biomechanical behaviour. A beam finite element (FE) spine model for haptic interaction is built based on a solid FE spine model, which is created in an offline finite element analysis (FEA) software. The mechanical properties of the beam FE spine model are tuned so that its deformation behaviour is very similar to that of the offline solid spine model. Furthermore, the online beam FE spine model is greatly simplified as compared to the offline solid FEA model and hence more appropriate for real-time simulations. Haptic feedback is provided in the real-time simulation of the beam FE spine model, in order to enhance the human–computer interaction. Based on the results of spine deformation obtained from the haptic online FE simulator, the offline FEA spine model again is used to reproduce the same deformation and hence to provide more detailed deformation and vertebrae’s stress/strain information, which the haptic beam FE model is not capable to provide. Then, we present a tetrahedral mass–spring system to model intervertebral discs, which are interposed between vertebrae, and the offline simulation can be run to achieve deformation responses of these intervertebral discs. In our research, combining the haptic beam FE model and the intervertebral disc model can be useful for studying biokinematics of the spine as well as assessing medical conditions in the spine or the biomechanical behaviour of new designs of artificial intervertebral discs.     

System for interactive scientific driving simulation with haptic information

An interactive wheeled vehicle simulator is described, consisting of a software application for simulating vehicle dynamics and presenting the results in virtual 3D environment, and a haptic interface. The latter consists of a hydraulically actuated active seat and an electrically activated active steering wheel. The system makes it possible to simulate wheeled vehicles of various configurations on arbitrary terrains. The design of the system is presented and the specific considerations are discussed. The system was experimentally verified by measurements on a real off-road vehicle. The kinematic values measured on the real vehicle and those, measured on the haptic interface were compared. The results are discussed and are generally found to correspond well, making the system usable for studying vehicle performance and training its operators.     

Application of hidden Markov model to acquisition of manipulation skills from haptic rendered virtual environment

This paper explores the feasibility of reconstructing human manipulation skills in complex constrained motion by tracing and learning the manipulation performed by the operator. The peg-in-hole insertion problem is used as a case study, which represents a typical constrained motion force sensitive manufacturing task with the attendant issues of jamming, tight clearance and the need for quick assembly times. In the developed system, position and contact force and torque as well as orientation data generated in the haptic rendered virtual environment combined with a priori knowledge about the task are used to identify and learn the skills in the newly demonstrated task. The recorded training data is classified into contact states, which are identified with hidden Markov model (HMM) as human skills. The HMM parameters are obtained from the training data. By evaluating the controller's performance in each contact state from haptic rendered virtual environment, the robot develops the best trajectories to imitate the human behaviour. In this paper the significance of this research project is highlighted and the developed approach and the progress made so far on this project are reported.     

Simultaneous remote haptic collaboration for assembling tasks

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen’s University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user’s object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.     

On the development of a haptic system for rapid product development

This paper presents a system development that extends haptic modeling to a number of key aspects in product development. Since haptic modeling has been developed based on physical laws, it is anticipated that a natural link between the virtual world and practical applications can be established based on haptic interaction. In the proposed system, a haptic device is used as the central mechanism for reverse engineering, shape modeling, real time mechanical property analysis, machining tool path planning and coordinate measuring machine (CMM) tolerance inspection path planning. With all these features in a single haptic system, it is possible to construct a three dimensional part by either haptic shape modeling or reverse engineering, then performing real-time mechanical property analysis in which the stiffness of a part can be felt and intuitively evaluated by the user, or generating collision free cutter tool path and CMM tolerance inspection path. Due to the force feed back in all of the above activities, the product development process is more intuitive, efficient and user-friendly. A prototype system has been developed to demonstrate the proposed capabilities.     

The feasibility of a mixed reality surgical training environment

The Sheffield knee arthroscopy training system (SKATS) was originally a visual-based virtual environment without haptic feedback, but has been further developed as a mixed reality-training environment through the use of tactile augmentation (or passive haptics). The design of the new system is outlined and then tested. In the first experiment described, the effect of tactile augmentation on performance is considered by comparing novice performance using the original and mixed reality system. In the second experiment the mixed reality system is assessed in terms of construct validity by comparing the performance of users with differing levels of surgical expertise. The results are discussed in terms of the validity of a mixed reality environment for training knee arthroscopy.     

高帧频面阵CCD实时显示系统设计

为了能够在特殊环境下拍摄到最优的图像,需要实时地调整相机参数,这要求实时显示拍摄的图像。使用FPGA配合通用Camera Link采集卡,设计了一套通用性很强的高帧频面阵CCD实时显示系统,使用Verilog HDL语言编写CCD驱动程序,能够通过USB实时调整相机的工作参数,使相机工作在最佳状态,使用KAI340D CCD在205.6fps的帧频下测试,系统工作良好,满足了实验需求。