Haptic interaction and volume modeling techniques for realistic dental simulation

We present haptic simulation and volume modeling techniques for a virtual dental training system. The system allows dental students to learn dental procedures and master their skills with realistic tactual feelings. It supports various dental procedures, such as dental probing, to diagnose carious lesions, drilling operation for cavity preparation, and filling the prepared cavities with amalgam. The system requires fast and stable haptic rendering and volume modeling techniques working on the virtual tooth. Collision detection and force computation are implemented on an offset surface in volumetric representation to simulate reasonable physical interactions between dental tools with a certain volume and the teeth model. To avoid discrete haptic feeling due to the gap between the fast haptic process (1 KHz) and much slower visual update frequency (30 Hz) during drilling and filling the cavities, we employed an intermediate implicit surface to be animated between the original and target surfaces. The volumetric teeth model is converted into a geometric model by an adaptive polygonization method to maintain sharp features in every visual frame. Volumetric material properties are represented by stiffness and color values to simulate the resistance and texture information depending on anatomical tissues. Finally, we made a dental workbench to register sensory modalities like visual, auditory and haptic sensation.     

Force modeling for tooth preparation in a dental training system

Feedback force is very important for novices to simulate tooth preparation by using the haptic interaction system (dental training system) in a virtual environment. In the process of haptic simulation, the fidelity of generated forces by a haptic device decides whether the simulation is successful. A force model computes feedback force, and we present an analytical force model to compute the force between a tooth and a dental pin during tooth preparation. The force between a tooth and a dental pin is modeled in two parts: (1) force to resist human’s operation and (2) friction to resist the rotation of the dental engine. The force to resist the human’s operation is divided into three parts in the coordinates that are constructed on the bottom center of the dental pin. In addition, we also consider the effects of dental-pin type, tooth stiffness, and contact geometry in the force model. To determine the parameters of the force model, we construct a measuring system by using machine vision and a force/torque sensor to track the human’s operations and measure the forces between the dental pins and teeth. Based on the measuring results, we construct the relation between the force and the human’s operation. The force model is implemented in the prototype of a dental training system that uses the Phantom as the haptic interface. Dentists performing virtual operations have confirmed the fidelity of feedback force.     

Six degree-of-freedom haptic rendering using spatialized normal cone search

This paper describes a haptic rendering algorithm for arbitrary polygonal models using a six degree-of-freedom haptic interface. The algorithm supports activities such as virtual prototyping of complex polygonal models and adding haptic interaction to virtual environments. The underlying collision system computes local extrema in distance between the model controlled by the haptic device and the rest of the scene. The haptic rendering computes forces and torques on the moving model based on these local extrema. The system is demonstrated on models with tens of thousands of triangles and developed in an accessibility application for finding collision-free paths.     

Sensation-preserving haptic rendering

Most human-computer interactive systems focus primarily on the graphical rendering of visual information and, to a lesser extent, on the display of auditory information. Haptic interfaces have the potential to increase the quality of human-computer interaction by accommodating the sense of touch. They provide an attractive augmentation to visual display and enhance the level of understanding of complex data sets. A haptic rendering system generates contact or restoring forces to prevent penetration into the virtual objects and create a sense of touch. The system computes contact forces by first detecting if a collision or penetration has occurred. Then, the system determines the (projected) contact points on the model surface. Finally, it computes restoring forces based on the amount of penetration. Researchers have recently investigated the problem of rendering the contact forces and torques between 3D virtual objects. This problem is known as six-degrees-of-freedom (6-DOF) haptic rendering, as the computed output includes both 3-DOF forces and 3-DOF torques. This article presents an overview of our work in this area. We suggest different approximation methods based on the principle of preserving the dominant perceptual factors in haptic exploration.     

Stroke-based modeling and haptic skill display for Chinese calligraphy simulation system

The goal of this paper is to study haptic skill representation and display in a Chinese calligraphy training system. The challenge is to model haptic skill during the writing of different strokes in Chinese characters and to achieve haptic rendering with high fidelity and stability. The planning of the writing process is organized at three levels: task, representation and device level to describe the haptic handwriting skill. State transition graph (STG) is proposed to describe switches between tasks during the handwriting. Chinese characters are modeled using 39 typical strokes, which are further grouped into basic and compound strokes. The compound stroke is considered to be sequential combination of the basic strokes. Straight and curve strokes are modeled using line segment and the Bezier curve, respectively. Information from STG is used for real-time collision detection and haptic rendering. Ambiguity of the collision detection at stroke-corner points is prevented using active stroke combined with local nearest point computation. A modified virtual fixture method is developed for haptic rendering. The approach is tested on a prototype training system using Phantom desktop. Initial experiments suggest that the proposed modeling and rendering method is effective.     

A modular haptic rendering algorithm for stable and transparent 6-DOF manipulation

This paper presents a modular algorithm for six-degree-of-freedom (6-DOF) haptic rendering. The algorithm is aimed to provide transparent manipulation of rigid models with a high polygon count. On the one hand, enabling a stable display is simplified by exploiting the concept of virtual coupling and employing passive implicit integration methods for the simulation of the virtual tool. On the other hand, transparency is enhanced by maximizing the update rate of the simulation of the virtual tool, and thereby the coupling impedance, and allowing for stable simulation with small mass values. The combination of a linearized contact model that frees the simulation from the computational bottleneck of collision detection, with penalty-based collision response well suited for fixed time-stepping, guarantees that the motion of the virtual tool is simulated at the same high rate as the synthesis of feedback force and torque. Moreover, sensation-preserving multiresolution collision detection ensures a fast update of the linearized contact model in complex contact scenarios, and a novel contact clustering technique alleviates possible instability problems induced by penalty-based collision response.     

速度驱动的复杂场景多层级力觉交互算法

触/力觉交互技术应用环境的复杂化,对复杂触/力觉场景的研究提出了新的挑战.文中以虚拟现实牙科手术训练系统为背景,研究基于速度驱动的复杂场景多层级力觉交互算法.文中采用CATIA对下牙列大量数据进行网格简化,建立了下牙列场景的层次细节(Levels Of Detail,LOD)模型;基于人类触觉感知的精度随交互速度的变化规律,设计了速度驱动的LOD模型触发机制;提出了虚拟工具化身SCP(Surface Contact Point)层级映射算法,保证了力觉交互设备的稳定性,并发现了映射约束线段的长度是保证力觉逼真性及实时性协调的关键变量;最后,设计了针对一般儿何体和复杂曲面的实验,量化评价文中算法的逼真性及实时性,证明了将人手运动速度作为场景模型复杂度切换驱动条件的有效性.     

交互式乐器演奏的六自由度力觉渲染方法

目的在进行虚拟乐器交互演奏时,需要模拟触力觉-视觉-听觉多通道同步反馈,其中触力觉反馈的难点在于模拟人手操作乐器的六自由度(6-Do F)力觉交互过程。方法提出一种基于混合模型和单边约束优化的六自由度力觉合成方法,实现了虚拟人手和琴弦的多点多区域接触力觉模拟。虚拟人手采用层次化球树模型表达,古琴采用混合模型表达,其中琴体和琴弦分别采用层次化球树模型和直线解析模型。提出了基于混合模型的离散碰撞检测算法,实时检测虚拟手和琴弦是否产生碰撞;基于发生碰撞的几何元素对建立单边不可穿透约束方程,通过Active Set方法求解约束优化后方程,获得6维位姿变量保证图形显示场景中的虚拟手不会和琴弦产生穿透。为模拟琴弦变形,提出变直径的圆柱体模型来模拟琴弦在不同振动幅度下的动力学响应;提出交互状态敏感的力计算模型以刻画人手在弹奏不同状态琴弦(静态、振动态)的力觉感受差异。结果基于力觉交互设备Phantom Premium 3.0建立了实验平台,实验结果表明,本文算法可以模拟单点、多点等不同接触状态,并能模拟6维力和力矩,操作者可以感受到琴弦振动时的细腻力感觉,力觉交互过程稳定,算法计算效率在1 k Hz以上。结论算法可模拟针对琴弦一类的超薄形状物体的多点接触力觉交互过程,算法计算效率高,包含碰撞检测、约束优化、琴弦变形仿真等计算回路的更新频率也能达到要求,该混合模型能为后续复杂形状物体的碰撞响应研究提供思路。     

Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator

Although people usually contact a surface with some area rather than a point, most haptic devices allow a user to interact with a virtual object at one point at a time and likewise most haptic rendering algorithms deal with such situations only. In a palpation procedure, medical doctors push and rub the organ's surface, and are provided the sensation of distributed pressure and contact force (reflecting force) for discerning doubtable areas of the organ. In this paper, we suggest real-time area-based haptic rendering to describe distributed pressure and contact force simultaneously, and present a haptic interface system to generate surface properties in accordance with the haptic rendering algorithm. We represent the haptic model using the shape-retaining chain link (S-chain) framework for a fast and stable computation of the contact force and distributed pressure from a volumetric virtual object. In addition, we developed a compact pin-array-type tactile display unit and attached it to the PHANToM^TM haptic device to complement each other. For the evaluation, experiments were conducted with non-homogenous volumetric cubic objects consisting of approximately 500 000 volume elements. The experimental results show that compared to the point contact, the area contact provides the user with more precise perception of the shape and softness of the object's composition, and that our proposed system satisfies the real-time and realism constraints to be useful for a virtual reality application.     

高度复杂拟凸体的实时触觉绘制

触觉绘制目前已成为虚拟现实的研究热点之一,其核心是触觉代理与目标物体的实时碰撞检测与距离计算,对高度复杂模型的触觉绘制仍然是虚拟现实领域的难题,原因是对模型复杂度敏感的算法均不能满足触觉绘制苛刻的时间要求.提出触觉包围盒概念,并针对拟凸体提出一种基于触觉包围盒的实时触觉绘制算法.触觉包围盒记录了离散的采样光线与目标物体的相交信息,进行触觉计算时,只需要执行1次线段与包围盒求交运算和5次双线性插值运算即可高速求出碰撞信息,且所得到的碰撞信息可直接用于反馈力计算.实验结果表明,该算法不仅快速有效,而且具有与触觉绘制模型复杂度不相关的优点.     

用于实时柔性触觉再现的平行菱形链连接模型

精度高且实时性好的柔性触觉变形模型是实现触觉再现系统的关键。提出了一种新的基于物理意义的平行菱形链连接触觉变形模型,系统中各个链结构单元相对位移的叠加对外等效为物体表面的变形,与之相连的弹簧弹性力的合力等效为物体表面的接触力。使用Delta 6-DOF手控器,建立了触觉再现实验系统,对柔性体的接触变形和实时虚拟触觉反馈进行仿真, 实验结果表明所提出的模型不仅计算简单,而且能够保证触觉接触力和形变计算具有较高精度,满足虚拟现实系统对精细作业和实时性的要求。     

Stable adaptive algorithm for Six Degrees-of-Freedom haptic rendering in a dynamic environment

Recently, physically-based simulations with haptics interaction attracted many researchers. In this paper, we propose an adaptive Six Degrees-of-Freedom (6-DOF) haptic rendering algorithm based on virtual coupling, which can automatically adjust virtual coupling parameters according to mass values of the simulated virtual tools. The algorithm can overcome the virtual tool displacement problem caused by the large mass values of the virtual tool and can provide stable force/torque display. The force/torque magnitude is saturated to the maximum force/torque values of the haptic device automatically. The implemented algorithm is tested on the simple and complex standard benchmarks. The experimental results confirm that the proposed adaptive 6-DOF haptic rendering algorithm displays good stability and accuracy for haptic rendering of dynamic virtual objects with mass values.     

Interactive mixed reality white cane simulation for the training of the blind and the visually impaired

This paper presents a mixed reality tool developed for the training of the visually impaired based on haptic and auditory feedback. The proposed approach focuses on the development of a highly interactive and extensible Haptic Mixed Reality training system that allows visually impaired to navigate into real size Virtual Reality environments. The system is based on the use of the CyberGrasp™ haptic device. An efficient collision detection algorithm based on superquadrics is also integrated into the system so as to allow real time collision detection in complex environments. A set of evaluation tests is designed in order to identify the importance of haptic, auditory and multimodal feedback and to compare the MR cane against the existing Virtual Reality cane simulation system.     

Haptic collision handling for simulation of transnasal surgery

Simulation of endoscopic navigation in the narrow nasal cavity poses important challenges to the computation of adequate and near‐realistic collision response and haptic feedback because extensive multidirectional contact and massive tissue deformations are inevitable. We present a virtual coupling algorithm that provides stable collision response as well as intuitive and smooth haptic interaction in all phases of the simulation. In each iteration, continuous collision detection between the point shell representing the surface of the virtual patient anatomy and the endoscope, represented by a cylinder, is performed. This allows for rolling back the instrument movement to the point in time the first collision occurred. Subsequently, a relaxation process locally optimizes the position and orientation of the instrument. A novel method of applying contact forces to colliding tissues and thus triggering appropriate deformations improves the fluency of navigation. This paper describes the algorithm and presents experimental results. © Her Majesty the Queen in Right of Canada 2012. Reproduced with the permission of the Minister of Medical Devices, National Research Council Canada.     

基于OSG的虚拟场景中包围盒碰撞检测的研究

碰撞检测是虚拟场景的核心技术,其效果的好坏直接影响整个虚拟场景的真实感。基于OSG（OpenSceneGraph）三维场景渲染引擎和Multigen Creator三维可视化仿真建模软件,采用包围盒碰撞检测算法,实现了多个静态物体与动态物体的碰撞检测,通过使用射线检测算法可以很好地解决静态物体与地面以及动态物体与地面的碰撞检测。为了满足实时性的要求,提出了多个静态物体与动态物体的碰撞检测优化算法的数学模型。结果表明,算法具有很好的有效性和快速性,能够满足要求。     

一种基于物理意义的快速力反馈形变模型及实时力觉响应算法

虚拟物体在受力作用时的形变建模是虚拟环境中力/触觉人机交互的关键.文中提出了一种新的基于物理意义的形变建模方法,不仅计算速度快,满足力反馈的实时性要求,而且能够同时保证接触力和形变的计算具有较高的精度,适用于具有较大变形量的柔性物体的力反馈计算,满足精细作业对虚拟现实系统的要求.     

面向磁悬浮视触觉交互的多速率系统框架

非接触式磁悬浮视触觉交互克服了机械式交互的固有摩擦,具有广阔应用前景,但存在交互过程中虚拟工具穿透物体、图形渲染与触觉渲染速率不一致等问题。针对上述问题,提出面向磁悬浮视触觉交互的多速率系统框架,通过扩展三自由度（3-DOF）单射线触觉渲染方法,利用多射线对虚拟工具进行建模,避免工具穿透,实现六自由度（6-DOF）触觉渲染;通过多速率并行,实现速率不一致模块间相互协同;通过构建映射滤波算法,实现视觉定位数据到虚拟工具位姿的稳定映射。实验结果表明,该系统能有效避免交互过程中的穿透现象,并提供稳定、真实的视触觉反馈。     

虚拟装配中基于导纳控制的力觉渲染技术

传统的力触觉渲染多采用阻抗控制,不能很好地满足虚拟装配的应用要求,相比之下导纳控制模式更适用这一领域.为此提出一种基于导纳控制的双线程力觉渲染构架,并给出相应的力觉渲染算法.首先建立用于导纳控制的动力学模型,并讨论了碰撞和约束这2个状态下的力觉渲染;为了使用力觉交互接口进行虚拟装配中的小间隙装配,提出物理约束与几何约束结合的力觉渲染方法;最后针对物理计算和力反馈循环2个线程刷新频率不匹配的问题,利用二次拉格朗日多项式进行数值插值,实现了力觉交互接口的平稳输出.通过力反馈设备与自主开发的虚拟装配原型系统VAPP的连接与应用,验证了所提出的算法满足虚拟装配系统中力觉交互的应用要求.     

力／触觉再现设备的研究现状与应用

力/触觉再现技术作为一种新兴的人机交互模式在虚拟现实系统得到较多应用。力/触觉再现设备是力/触觉再现系统实现过程中必不可少的环节,受到广泛的重视和研究。讨论了力/触觉再现设备的种类和特点,分类介绍了国内外力/触觉再现设备的研究现状,并作了分析比较。最后讨论了力/触觉再现设备在不同领域的应用,并探讨了其未来的研究发展趋势。     

虚拟体空间中的触觉雕刻

目前，在虚拟环境中大多数的信息获取是通过视觉、听觉等非接触感觉获得的。然而缺乏触觉反馈的信息减少了很大一部分的信息源。在看和听之外，能够触摸、感觉和操纵物体，在很大程度上提高了虚拟环境的真实性。该文研究了触觉绘制的基本模型，提出了采用虚平面作为中介实现体数据的实时触觉绘制。并在此基础上探讨了体的局部变形及结合触觉反馈模型，实现了具有触觉反馈的虚拟雕刻交互系统。该系统可应用于融化、燃烧、印记、构造和着色实时交互操作。