Haptic palpation for medical simulation in virtual environments

Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to determine their size, shape, consistency and location. Many medical procedures utilize palpation as a supplementary interaction technique and it can be therefore considered as an essential basic method. However, palpation is mostly neglected in medical training simulators, with the exception of very specialized simulators that solely focus on palpation, e.g., for manual cancer detection. In this article we propose a novel approach to enable haptic palpation interaction for virtual reality-based medical simulators. The main contribution is an extensive user study conducted with a large group of medical experts. To provide a plausible simulation framework for this user study, we contribute a novel and detailed interaction algorithm for palpation with tissue dragging, which utilizes a multi-object force algorithm to support multiple layers of anatomy and a pulse force algorithm for simulation of an arterial pulse. Furthermore, we propose a modification for an off–the–shelf haptic device by adding a lightweight palpation pad to support a more realistic finger grip configuration for palpation tasks. The user study itself has been conducted on a medical training simulator prototype with a specific procedure from regional anesthesia, which strongly depends on palpation. The prototype utilizes a co-rotational finite-element approach for soft tissue simulation and provides bimanual interaction by combining the aforementioned techniques with needle insertion for the other hand. The results of the user study suggest reasonable face validity of the simulator prototype and in particular validate medical plausibility of the proposed palpation interaction algorithm.     

A virtual environment simulator for mechanical system dynamics with online interactive control

Virtual prototyping is an effective tool in the development of mechanical product. Physically accurate simulation of multi-body mechanical system enables designers to investigate, explore, and experience the performance and behavior of an evolving product and thus reduce the number of physical prototypes needed. For the sake of better support for designers to manipulate the simulation, we have developed a dynamic simulation package of mechanical system, which provides an interactive control during the simulation process. The package incorporates physical behaviors and dynamic interactions by dynamics based modeling approach in multi-body mechanical system. In the package, user’s actions (e.g., loading a model, picking and dragging objects, steering objects during simulation process, etc.) are based on an ATN task management, and a multi-modal interface is provided which supports 2D desktop devices and 3D VR devices. The main contribution of the simulator is providing supports that allow users’ interactive manipulation in the simulation loop. During the simulation process, users can modify the constraints between components, apply force/torque to interested components and change the parameters of forces/torques. The simulator automatically updates the dynamic model of the mechanical system in real time, and then continuously simulates the behavior of the model under the current condition in the loop. An example of dynamic simulation of a vehicle is implemented and the simulation result is compared to that of ADAMS to verify the correctness and accuracy of the simulator. With the real-time interaction, solution and visualization of simulation model, the package affords better support for designers to participate in the simulation interactively and effectively.     

Contact and Deformation Modeling for Interactive Environments

Contact and deformation modeling for interactive environments has seen many applications, from surgical simulation and training, to virtual prototyping, to teleoperation, etc., where both visual feedback and haptic feedback are needed. High-quality feedback demands a high level of physical realism as well as a high update rate in rendering, which are often conflicting requirements. In this paper, we present a unique approach to modeling force and deformation between a rigid body and an elastic object under complex contacts, which achieves a good compromise of reasonable physical realism and real-time update rate (at least 1 kHz). We simulate contact forces based on a nonlinear physical model. We further introduce a novel approximation of material deformation suitable for interactive environments based on applying Bernoulli-Euler bending beam theory to the simulation of elastic shape deformation. Our approach is able to simulate the contact forces exerted upon the rigid body (that can be virtually held by a user via a haptic device) not only when it forms one or more than one contact with the elastic object, but also when it moves compliantly on the surface of the elastic object, taking friction into account. Our approach is also able to simulate the global and local shape deformation of the elastic object due to contact. All the simulations can be performed in a combined update rate of over 1 kHz, which we demonstrate in several examples.     

用于实时柔性触觉再现的平行菱形链连接模型

精度高且实时性好的柔性触觉变形模型是实现触觉再现系统的关键。提出了一种新的基于物理意义的平行菱形链连接触觉变形模型,系统中各个链结构单元相对位移的叠加对外等效为物体表面的变形,与之相连的弹簧弹性力的合力等效为物体表面的接触力。使用Delta 6-DOF手控器,建立了触觉再现实验系统,对柔性体的接触变形和实时虚拟触觉反馈进行仿真, 实验结果表明所提出的模型不仅计算简单,而且能够保证触觉接触力和形变计算具有较高精度,满足虚拟现实系统对精细作业和实时性的要求。     

Neural dynamics-based Poisson propagation for deformable modelling

This paper presents a new methodology from the standpoint of energy propagation for real-time and nonlinear modelling of deformable objects. It formulates the deformation process of a soft object as a process of energy propagation, in which the mechanical load applied to the object to cause deformation is viewed as the equivalent potential energy based on the law of conservation of energy and is further propagated among masses of the object based on the nonlinear Poisson propagation. Poisson propagation of mechanical load in conjunction with non-rigid mechanics of motion is developed to govern the dynamics of soft object deformation. Further, these two governing processes are modelled with cellular neural networks to achieve real-time computational performance. A prototype simulation system with a haptic device is implemented for real-time simulation of deformable objects with haptic feedback. Simulations, experiments as well as comparisons demonstrate that the proposed methodology exhibits nonlinear force–displacement relationship, capable of modelling large-range deformation. It can also accommodate homogeneous, anisotropic and heterogeneous materials by simply changing the constitutive coefficient value of mass points.

     

基于点采样的三维动画研究*

建立了一种对高弹性、高塑性物体进行动画模拟的新方式。本算法利用点采样的方式表现表面,使用采样分裂和聚合以适应动画中采样密度的变化。将弹性力学中的理论应用到三维动画中,使用MLS法计算应力、张力、弹性体力,使用可变张力来模拟塑性变化,利用向量位移进行了动画模拟,并使用了多种曲面表示提高表面质量。动画可表现融化、流动、分裂、融合等弹性、塑性变化,可用于实时交互也可用于高质量的离线渲染。     

一种基于物理意义的快速力反馈形变模型及实时力觉响应算法

虚拟物体在受力作用时的形变建模是虚拟环境中力/触觉人机交互的关键.文中提出了一种新的基于物理意义的形变建模方法,不仅计算速度快,满足力反馈的实时性要求,而且能够同时保证接触力和形变的计算具有较高的精度,适用于具有较大变形量的柔性物体的力反馈计算,满足精细作业对虚拟现实系统的要求.     

A realistic elastic rod model for real-time simulation of minimally invasive vascular interventions

Simulating intrinsic deformation behaviors of guidewire and catheters for interventional radiology (IR) procedures, such as minimally invasive vascular interventions is a challenging task. Especially real-time simulations for interactive training systems require not only the accuracy of guidewire manipulations, but also the efficiency of computations. The insertion of guidewires and catheters is an essential task for IR procedures and the success of these procedures depends on the accurate navigation of guidewires in complex 3D blood vessel structures to a clinical target, whilst avoiding complications or mistakes of damaging vital tissues and blood vessel walls. In this paper, a novel elastic model for modeling guidewires is presented and evaluated. Our interactive guidewire simulator models the medical instrument as thin flexible elastic rods with arbitrary cross sections, treating the centerline as dynamic and the deformation as quasi-static. Constraints are used to enforce inextensibility of guidewires, providing an efficient computation for bending and twisting modes of the physically-based simulation model. We demonstrate the effectiveness of the new model with a number of simulation examples.     

虚拟柔性体实时形变仿真模型研究

为了在虚拟现实柔性体力触觉交互研究中得到稳定、连续、真实的力触感,提出一种基于球面调和函数表达的虚拟柔性体实时形变仿真模型,利用球面调和函数的正交归一、旋转不变、多尺度等特性实现物体的快速准确表达.在变形体的密度、杨氏模量、泊松比等参数已知的情况下,基于径向基函数神经网络模型预测柔性体受力形变后的SH模型.仿真结果表明,该方法不仅可以准确表达柔性体的实时形变,而且使得基于SH表达的柔性体形变的视觉刷新描述与柔性体反馈力的触觉刷新描述同步,从而满足虚拟手术仿真训练等虚拟柔性体力触觉交互研究要求.     

A fast parametric deformation mechanism for virtual reality applications

Virtual reality technologies have been adopted in a wide variety of applications for its interactive ability and realistic senses. Despite early implementations regard VR only as a medium for lively animation; a practical VR work must deliver precise deformation on virtual objects based on real-time interactions. The exact ability is especially important for users who utilize VR to do collaborative design, for it will greatly reduce the amount of on-line computations on operating substance-based interactions, and consequently facilitates the collaboration. Therefore, this research will employ neural networks to memorize the deformation behavior of solid objects, and then perform instant and accurate deformations in the virtual environment. The proposed method also allows design variations for parametric features, and uses feature parameters as variable switches to adjust the deformation mechanism. There are three steps in the method: (1) For a sample object, generate force-induced deformations using the finite-element method; (2) memorize the surface displacements with artificial neural networks; and (3) convert the parametric deformation matrices into Behavioral Modules for the virtual reality engine. In the implementations, ANSYS is used to generate model deformations, and MATLAB is used to perform neural training. Finally, a virtual environment is built using Virtools where customized Building Blocks are created to present interactive deformation behavior. Experiments were carried out on an Intel XEON workstation with nVIDIA Quadro4 750GL display device. Sample workparts are tested to examine the ability of the method. The results show that both training accuracy and real-time capability are more than satisfactory.     

Physically accurate haptic rendering with dynamic effects

We focus on the realism/transparency aspect of haptic rendering. We introduce a novel approach that enables physically correct and accurate simulation of contact wrench W/sub c/ for general rigid objects in real time, taking into account not only friction and gravity but also dynamic effects. Our method for contact force and moment simulation builds on the real-time identification of geometrically valid contact states despite digital errors. Our approach applies to general rigid bodies including both polyhedral and nonpolyhedral objects. For nonpolyhedral, curved objects, we build our contact state representation and contact force/moment model directly on the smooth and accurate representation of the object surfaces. Our approach's key idea is to solve for the contact force and moment analytically based on not only the contact configuration, but also the real-time identification of the exact type of the corresponding contact state, the type of instantaneous motion of the held object prior to reaching the contact configuration.     

Objects interaction using superquadrics for telemanipulation system simulation

A high-fidelity “hybrid” telerobotic simulator suitable for telemanipulation rehearsal, supervisory control, operator training, and human factors performance evaluation has been developed. The simulator is capable of operation in the conventional rate-control, master/slave control, and a data-driven preprogrammed mode of operation. It has teach/playback capability, which allows an operator to generate joint commands for real-time teleoperation. For high-level task execution, the operator selects a specific task from a set of menu options and the simulator automatically generates the required joint commands. In this article, an object detection strategy that has been implemented in the simulator for more realistic telemanipulation is presented. The object detection technique is based on the use of superellipsoids, which have a convenient inside-outside function for interference testing. The technique, which treats both moving and stationary objects in a consistent manner, has proved to be easy to implement and optimize for real-time applications. The object detection feature is also used to provide visual feedback as a low-cost force reflection strategy.     

An autowave based methodology for deformable object simulation

This paper presents a new methodology for deformable object simulation by drawing an analogy between autowaves and elastic deformation. The potential energy stored in an elastic body as a result of a deformation caused by an external force is propagated among mass points by non-linear autowaves. The novelty of the methodology is that autowave techniques are established to describe the potential energy distribution of a deformation for extrapolating internal elastic forces, and non-linear material properties are modelled with non-linear autowaves other than geometric non-linearity. A haptic virtual reality system has been developed for deformation simulation with force feedback. The proposed methodology not only deals with large-range deformations, but also accommodates isotropic, anisotropic and inhomogeneous materials by simply modifying diffusion coefficients.     

Haptically integrated simulation of a finite element model of thoracolumbar spine combining offline biomechanical response analysis of intervertebral discs

In this paper, we first describe the construction of a finite element model of the human spine that may be used to assist the investigation of clinical problems by predicting its biomechanical behaviour. A beam finite element (FE) spine model for haptic interaction is built based on a solid FE spine model, which is created in an offline finite element analysis (FEA) software. The mechanical properties of the beam FE spine model are tuned so that its deformation behaviour is very similar to that of the offline solid spine model. Furthermore, the online beam FE spine model is greatly simplified as compared to the offline solid FEA model and hence more appropriate for real-time simulations. Haptic feedback is provided in the real-time simulation of the beam FE spine model, in order to enhance the human–computer interaction. Based on the results of spine deformation obtained from the haptic online FE simulator, the offline FEA spine model again is used to reproduce the same deformation and hence to provide more detailed deformation and vertebrae’s stress/strain information, which the haptic beam FE model is not capable to provide. Then, we present a tetrahedral mass–spring system to model intervertebral discs, which are interposed between vertebrae, and the offline simulation can be run to achieve deformation responses of these intervertebral discs. In our research, combining the haptic beam FE model and the intervertebral disc model can be useful for studying biokinematics of the spine as well as assessing medical conditions in the spine or the biomechanical behaviour of new designs of artificial intervertebral discs.     

Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator

Although people usually contact a surface with some area rather than a point, most haptic devices allow a user to interact with a virtual object at one point at a time and likewise most haptic rendering algorithms deal with such situations only. In a palpation procedure, medical doctors push and rub the organ's surface, and are provided the sensation of distributed pressure and contact force (reflecting force) for discerning doubtable areas of the organ. In this paper, we suggest real-time area-based haptic rendering to describe distributed pressure and contact force simultaneously, and present a haptic interface system to generate surface properties in accordance with the haptic rendering algorithm. We represent the haptic model using the shape-retaining chain link (S-chain) framework for a fast and stable computation of the contact force and distributed pressure from a volumetric virtual object. In addition, we developed a compact pin-array-type tactile display unit and attached it to the PHANToM^TM haptic device to complement each other. For the evaluation, experiments were conducted with non-homogenous volumetric cubic objects consisting of approximately 500 000 volume elements. The experimental results show that compared to the point contact, the area contact provides the user with more precise perception of the shape and softness of the object's composition, and that our proposed system satisfies the real-time and realism constraints to be useful for a virtual reality application.     

Real-time elastic deformations of soft tissues for surgerysimulation

We describe a novel method for surgery simulation including a volumetric model built from medical images and an elastic modeling of the deformations. The physical model is based on elasticity theory which suitably links the shape of deformable bodies and the forces associated with the deformation. A real time computation of the deformation is possible thanks to a preprocessing of elementary deformations derived from a finite element method. This method has been implemented in a system including a force feedback device and a collision detection algorithm. The simulator works in real time with a high resolution liver model     

一种基于非线性弹簧模型的虚拟手交互新方法

基于虚拟手的交互技术在人机交互和人机工程学测试等应用中发挥着重要的作用。为了实现直观自然、实时准确、接近真实世界中的虚拟手与虚拟物体的交互,并计算出反馈作用力,首先提出了用非线性弹簧模型计算抓取作用力,使虚拟手和虚拟环境之间实现了基于物理的交互;然后将计算结果以视觉渲染的形式反馈给用户,并对仿真的速率做了定量分析,以便使仿真速率可以达到屏幕刷新频率和力反馈刷新频率的要求。实验结果表明,虚拟手不仅可以直观自然地抓取3维虚拟物体,而且和3维物体之间能够进行实时交互,同时可计算出反馈作用力。     

现代训练仿真器技术的发展

以人构成闭环的仿真器，其逼真度是指人体的感受而言，对象数学模型的准确程度，不能直接反映人体及其器官的感受程度，因此，仿真器技术开发的重点，应是保证实现人体感受的逼真度，这样，仿真器必然是针对不同仿真对象合理选择的多种技术的综合应用。该文对仿真器涉及到的视景显示技术，运动仿真技术等当前发展的情况进行了综述，同时指出其适用范围，也提到某些不适合技术的应用实例。     

虚拟手术中模拟软组织切割的混合弹性模型

虚拟手术仿真系统应该同时满足真实性和实时性的要求.弹簧质点模型可以满足实时性的要求,但它无法模拟人体的粘弹性特征;而传统的有限元模型虽然可以模拟人体的粘弹性特征,但巨大的时间消耗使得这个方法几乎无法给出实时的反馈.混合弹性模型是一种基于有限元方法的优化模型,由于采用了简化措施,使得它既克服了弹簧质点模型的缺陷;同时也满足了虚拟手术的实时性要求.实验证明混合弹性模型在虚拟手术的真实性和实时性之间达到了一个较好的折中效果,它可以实时的提供视觉反馈,同时也使在虚拟手术系统中引入力反馈器成为可能.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.