虚拟体空间中的触觉雕刻

目前，在虚拟环境中大多数的信息获取是通过视觉、听觉等非接触感觉获得的。然而缺乏触觉反馈的信息减少了很大一部分的信息源。在看和听之外，能够触摸、感觉和操纵物体，在很大程度上提高了虚拟环境的真实性。该文研究了触觉绘制的基本模型，提出了采用虚平面作为中介实现体数据的实时触觉绘制。并在此基础上探讨了体的局部变形及结合触觉反馈模型，实现了具有触觉反馈的虚拟雕刻交互系统。该系统可应用于融化、燃烧、印记、构造和着色实时交互操作。     

A Precomputed Approach for Real-Time Haptic Interaction with Fluids

The key to enhancing perception of the virtual world is improving mechanisms for interacting with that world. Through providing a sense of touch, haptic rendering is one such mechanism. Many methods efficiently display force between rigid objects, but to achieve a truly realistic virtual environment, haptic interaction with fluids is also essential. In the field of computational fluid dynamics, researchers have developed methods to numerically estimate the resistance due to fluids by solving complex partial differential equations, called the Navier-Stokes equations. However, their estimation techniques, although numerically accurate, are prohibitively time-consuming. This becomes a serious problem for haptic rendering, which requires a high frame rate. To address this issue, we developed a method for rapidly estimating and displaying forces acting on a rigid virtual object due to water. In this article, we provide an overview of our method together with its implementation and two applications: a lure-fishing simulator and a virtual canoe simulator     

Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator

Although people usually contact a surface with some area rather than a point, most haptic devices allow a user to interact with a virtual object at one point at a time and likewise most haptic rendering algorithms deal with such situations only. In a palpation procedure, medical doctors push and rub the organ's surface, and are provided the sensation of distributed pressure and contact force (reflecting force) for discerning doubtable areas of the organ. In this paper, we suggest real-time area-based haptic rendering to describe distributed pressure and contact force simultaneously, and present a haptic interface system to generate surface properties in accordance with the haptic rendering algorithm. We represent the haptic model using the shape-retaining chain link (S-chain) framework for a fast and stable computation of the contact force and distributed pressure from a volumetric virtual object. In addition, we developed a compact pin-array-type tactile display unit and attached it to the PHANToM^TM haptic device to complement each other. For the evaluation, experiments were conducted with non-homogenous volumetric cubic objects consisting of approximately 500 000 volume elements. The experimental results show that compared to the point contact, the area contact provides the user with more precise perception of the shape and softness of the object's composition, and that our proposed system satisfies the real-time and realism constraints to be useful for a virtual reality application.     

From measured physical parameters to the haptic feeling of fabric

Abstract real-time cloth simulation involves the solution of many computational challenges, particularly in the context of haptic applications, where high frame rates are necessary for obtaining a satisfactory tactile experience. In this paper, we present a real-time cloth simulation system that offers a compromise between a realistic physically-based simulation of fabrics and a haptic application with high requirements in terms of computation speed. We place emphasis on architecture and algorithmic choices for obtaining the best compromise in the context of haptic applications. A first implementation using a haptic device demonstrates the features of the proposed system and leads to the development of new approaches for haptic rendering using the proposed approach.     

Enhancing Realism of Wet Surfaces in Temporal Bone Surgical Simulation

We present techniques to improve visual realism in an interactive surgical simulation application: a mastoidectomy simulator that offers a training environment for medical residents as a complement to using a cadaver. As well as displaying the mastoid bone through volume rendering, the simulation allows users to experience haptic feedback and appropriate sound cues while controlling a virtual bone drill and suction/irrigation device. The techniques employed to improve realism consist of a fluid simulator and a shading model. The former allows for deformable boundaries based on volumetric bone data, while the latter gives a wet look to the rendered bone to emulate more closely the appearance of the bone in a surgical environment. The fluid rendering includes bleeding effects, meniscus rendering, and refraction. We incorporate a planar computational fluid dynamics simulation into our three-dimensional rendering to effect realistic blood diffusion. Maintaining real-time performance while drilling away bone in the simulation is critical for engagement with the system.     

面向磁悬浮视触觉交互的多速率系统框架

非接触式磁悬浮视触觉交互克服了机械式交互的固有摩擦,具有广阔应用前景,但存在交互过程中虚拟工具穿透物体、图形渲染与触觉渲染速率不一致等问题。针对上述问题,提出面向磁悬浮视触觉交互的多速率系统框架,通过扩展三自由度（3-DOF）单射线触觉渲染方法,利用多射线对虚拟工具进行建模,避免工具穿透,实现六自由度（6-DOF）触觉渲染;通过多速率并行,实现速率不一致模块间相互协同;通过构建映射滤波算法,实现视觉定位数据到虚拟工具位姿的稳定映射。实验结果表明,该系统能有效避免交互过程中的穿透现象,并提供稳定、真实的视触觉反馈。     

Sensation-preserving haptic rendering

Most human-computer interactive systems focus primarily on the graphical rendering of visual information and, to a lesser extent, on the display of auditory information. Haptic interfaces have the potential to increase the quality of human-computer interaction by accommodating the sense of touch. They provide an attractive augmentation to visual display and enhance the level of understanding of complex data sets. A haptic rendering system generates contact or restoring forces to prevent penetration into the virtual objects and create a sense of touch. The system computes contact forces by first detecting if a collision or penetration has occurred. Then, the system determines the (projected) contact points on the model surface. Finally, it computes restoring forces based on the amount of penetration. Researchers have recently investigated the problem of rendering the contact forces and torques between 3D virtual objects. This problem is known as six-degrees-of-freedom (6-DOF) haptic rendering, as the computed output includes both 3-DOF forces and 3-DOF torques. This article presents an overview of our work in this area. We suggest different approximation methods based on the principle of preserving the dominant perceptual factors in haptic exploration.     

Layered rhombus-chain-connected model for real-time haptic rendering

The modeling and simulation of deformable objects is a challenging topic in the field of haptic rendering between human and virtual environment. In this paper, a novel and efficient layered rhombus-chain-connected haptic deformation model based on physics is proposed for an excellent haptic rendering. During the modeling, the accumulation of relative displacements in each chain structure unit in each layer is equal to the deformation on the virtual object surface, and the resultant force of corresponding springs is equivalent to the external force. The layered rhombus-chain-connected model is convenient and fast to calculate, and can satisfy real-time requirement due to its simplicity. Experimental study in both homogenous and non-homogenous virtual human liver and lungs based on the proposed model are conducted, and the results demonstrate that our model provides stable and realistic haptic feeling in real time. Meanwhile, the display result is vivid.     

Six degree-of-freedom haptic rendering using spatialized normal cone search

This paper describes a haptic rendering algorithm for arbitrary polygonal models using a six degree-of-freedom haptic interface. The algorithm supports activities such as virtual prototyping of complex polygonal models and adding haptic interaction to virtual environments. The underlying collision system computes local extrema in distance between the model controlled by the haptic device and the rest of the scene. The haptic rendering computes forces and torques on the moving model based on these local extrema. The system is demonstrated on models with tens of thousands of triangles and developed in an accessibility application for finding collision-free paths.     

Haptic interaction and volume modeling techniques for realistic dental simulation

We present haptic simulation and volume modeling techniques for a virtual dental training system. The system allows dental students to learn dental procedures and master their skills with realistic tactual feelings. It supports various dental procedures, such as dental probing, to diagnose carious lesions, drilling operation for cavity preparation, and filling the prepared cavities with amalgam. The system requires fast and stable haptic rendering and volume modeling techniques working on the virtual tooth. Collision detection and force computation are implemented on an offset surface in volumetric representation to simulate reasonable physical interactions between dental tools with a certain volume and the teeth model. To avoid discrete haptic feeling due to the gap between the fast haptic process (1 KHz) and much slower visual update frequency (30 Hz) during drilling and filling the cavities, we employed an intermediate implicit surface to be animated between the original and target surfaces. The volumetric teeth model is converted into a geometric model by an adaptive polygonization method to maintain sharp features in every visual frame. Volumetric material properties are represented by stiffness and color values to simulate the resistance and texture information depending on anatomical tissues. Finally, we made a dental workbench to register sensory modalities like visual, auditory and haptic sensation.     

高度复杂拟凸体的实时触觉绘制

触觉绘制目前已成为虚拟现实的研究热点之一,其核心是触觉代理与目标物体的实时碰撞检测与距离计算,对高度复杂模型的触觉绘制仍然是虚拟现实领域的难题,原因是对模型复杂度敏感的算法均不能满足触觉绘制苛刻的时间要求.提出触觉包围盒概念,并针对拟凸体提出一种基于触觉包围盒的实时触觉绘制算法.触觉包围盒记录了离散的采样光线与目标物体的相交信息,进行触觉计算时,只需要执行1次线段与包围盒求交运算和5次双线性插值运算即可高速求出碰撞信息,且所得到的碰撞信息可直接用于反馈力计算.实验结果表明,该算法不仅快速有效,而且具有与触觉绘制模型复杂度不相关的优点.     

Stable adaptive algorithm for Six Degrees-of-Freedom haptic rendering in a dynamic environment

Recently, physically-based simulations with haptics interaction attracted many researchers. In this paper, we propose an adaptive Six Degrees-of-Freedom (6-DOF) haptic rendering algorithm based on virtual coupling, which can automatically adjust virtual coupling parameters according to mass values of the simulated virtual tools. The algorithm can overcome the virtual tool displacement problem caused by the large mass values of the virtual tool and can provide stable force/torque display. The force/torque magnitude is saturated to the maximum force/torque values of the haptic device automatically. The implemented algorithm is tested on the simple and complex standard benchmarks. The experimental results confirm that the proposed adaptive 6-DOF haptic rendering algorithm displays good stability and accuracy for haptic rendering of dynamic virtual objects with mass values.     

Haptics-equiped interactive PCI simulation for patient-specific surgery training and rehearsing针对特定病人的可交互经皮冠状动脉介入手术模拟

Despite the long history of medical simulations, suffering from the patient-specific heterogeneous heart physiological structure and complex intravascular procedures, it is still challenging for patient-specific percutaneous coronary intervention (PCI) surgery simulation. In this paper, we advocate a haptics-equiped interactive prototype system towards PCI surgeons training and patient-specific surgery rehearsing, which can afford trainees the opportunity to approximately experience the entire PCI procedures and customized emergency cases that might occur in common clinical settings. The full simulation covers tissue deformation, catheter and wire simulation, X-ray simulation, haptic feedback, and 3D realistic rendering, which in all give rise to the integrated physical, visual, haptic, and procedural realism. Our system can accommodate various comprehensive operations involved in PCI-related procedures, including feeding wires, releasing stents, injecting contrast medium, simulating X-ray, bleeding, etc. Moreover, our system framework is fully built upon CUDA, and thus can achieve real-time performance even on a common desktop. The high-fidelity, real-time efficiency and stableness of our system show great potentials for its practical applications in clinical training fields.     

MIMO Output Estimation With Reduced Multirate Sampling for Real-Time Haptic Rendering

This paper presents an output-estimation method with reduced multirate sampling for real-time multi-input-multi-output (MIMO) haptic rendering. Haptic systems employ physics-based deformation models such as finite-element models and mass-spring models. These physics-based deformation models for high fidelity have to deal with complex geometries, material properties, and realistic behavior of virtual objects. This incurs heavy computational burden and time delays so that the reflective force often cannot be computed at 1 kHz which is a safe frequency for stability of the haptic systems. Lower update rates of the haptic loop and the computational time delay also deteriorate the realism of the haptic system. This problem is resolved by the proposed MIMO output-estimation method. The haptic system is designed to have two sampling times, T and JT, for the haptic loop and the graphic loop, respectively. Dynamics of the physics-based deformation is captured in a discrete and deterministic input-output model. The MIMO output estimation method is developed drawing on a least-squares algorithm and an output-error estimation model. The P-matrix resetting algorithm is also designed to deal with the changing input-output relationship of the deformation model. The parameters of the discrete input-output model are adjusted online. Intersample outputs are computed from the estimated input-output model at a high rate, and traces the correct output computed from the deformation model. This method enables graphics rendering at a lower update rate, and haptic rendering at a higher update rate. Convergence of the proposed method is proved, and performance is demonstrated through simulation with both a linear tensor-mass and a linear mass-spring models.     

Haptic rendering for dental training system

Immersion and interaction are two key features of virtual reality systems, which are especially important for medical applications. Based on the requirement of motor skill training in dental surgery, haptic rendering method based on triangle model is investigated in this paper. Multi-rate haptic rendering architecture is proposed to solve the contradiction between fidelity and efficiency requirements. Realtime collision detection algorithm based on spatial partition and time coherence is utilized to enable fast contact determination. Proxy-based collision response algorithm is proposed to compute surface contact point. Cutting force model based on piecewise contact transition model is proposed for dental drilling simulation during tooth preparation. Velocity-driven levels of detail haptic rendering algorithm is proposed to maintain high update rate for complex scenes with a large number of triangles. Hapticvisual collocated dental training prototype is established using half-mirror solution. Typical dental operations have been realized including dental caries exploration, detection of boundary within dental cross-section plane, and dental drilling during tooth preparation. The haptic rendering method is a fundamental technology to improve immersion and interaction of virtual reality training systems, which is useful not only in dental training, but also in other surgical training systems. Supported by National Natural Science Foundation of China (Grant Nos. 60605027, 50575011), National High-Tech Research & Development Program of China (Grant No. 2007AA01Z310)     

多用户虚拟校园漫游系统的设计与实现

以北方工业大学的数字化和虚拟化为目标,结合虚拟现实技术,利用Lyinux商业引擎,建立多用户虚拟校园漫游系统,实现交互漫游、全景导航、实时聊天等功能。重点研究提高虚拟场景渲染速度和实现多用户之间的同步。该系统在交互性、渲染速度和真实感等方面取得很好的效果。     

虚拟手术中的模型实时绘制

实时的视觉反馈在虚拟手术中可以加强操作者的沉浸感。该文分析了体绘制和面绘制的优缺点,在系统实时性的要求下,实现了基于立体纹理映射的三维体模型的面绘制。算法在容许实时交互的同时,为操作者提供了真实感较强的视觉反馈。由于文中立体纹理生成及纹理坐标计算的特殊性,对虚拟手术中的拉压变形和切割等操作也能很好地处理。     

一种基于非线性弹簧模型的虚拟手交互新方法

基于虚拟手的交互技术在人机交互和人机工程学测试等应用中发挥着重要的作用。为了实现直观自然、实时准确、接近真实世界中的虚拟手与虚拟物体的交互,并计算出反馈作用力,首先提出了用非线性弹簧模型计算抓取作用力,使虚拟手和虚拟环境之间实现了基于物理的交互;然后将计算结果以视觉渲染的形式反馈给用户,并对仿真的速率做了定量分析,以便使仿真速率可以达到屏幕刷新频率和力反馈刷新频率的要求。实验结果表明,虚拟手不仅可以直观自然地抓取3维虚拟物体,而且和3维物体之间能够进行实时交互,同时可计算出反馈作用力。     

A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment

This study presents a 3D virtual reality (VR) keyboard system with realistic haptic feedback. The system uses two five-fingered data gloves to track finger positions and postures, uses micro-speakers to create simulated vibrations, and uses a head-mounted display (HMD) for 3D display. When users press a virtual key in the VR environment, the system can provide realistic simulated key click haptic feedback to users. The results of this study show that the advantages of the haptic VR keyboard are that users can use it when wearing HMDs (users do not need to remove HMDs to use the VR keyboard), the haptic VR keyboard can pop-up display at any location in the VR environments (users do not need to go to a specific location to use an actual physical keyboard), and the haptic VR keyboard can be used to provide realistic key click haptic feedback (which other studies have shown enhances user performance). The results also show that the haptic VR keyboard system can be used to create complex vibrations that simulate measured vibrations from a real keyboard and enhance keyboard interaction in a fully immersive VR environment.     

Biological cell injection visual and haptic interface

This paper presents a new three-dimensional (3-D) biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in liquid and observed through a microscope, 2-D visual feedback makes accurate manipulation in the 3-D world difficult. To improve the manipulation work, we proposed an intelligent human–machine interface. The 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to perform stable microinjection tasks, the operator needs force feedback and haptic assistance during penetration of the cell envelop — the chorion. Thus, realistic haptic rendering techniques have been implemented to validate stable insertion of a micropipette in a living cell. The proposed human–machine user's interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates, virtual haptic rendering to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by adequately controlling position, velocity and force parameters. Experiments showed that the virtualized reality interface acts as a tool for total guidance and assistance during microinjection tasks.