Hybrid client-server architecture and control techniques for collaborative product development using haptic interfaces

In this paper, a collaborative product development and prototyping framework is proposed by using distributed haptic interfaces along with deformable objects modeling. Collaborative Virtual Environment (CVE) is a promising technique for industrial product development and virtual prototyping. Network control problems such as network traffic and network delay in communication have greatly limited collaborative virtual environment applications. The problems become more difficult when high-update-rate haptic interfaces and computation intensive deformable objects modeling are integrated into CVEs for intuitive manipulation and enhanced realism. A hybrid network architecture is proposed to balance the computational burden of haptic rendering and deformable object simulation. Adaptive artificial time compensation is used to reduce the time discrepancy between the server and the client. Interpolation and extrapolation approaches are used to synchronize graphic and haptic data transmitted over the network. The proposed techniques can be used for collaborative product development, virtual assembly, remote product simulation and other collaborative virtual environments where both haptic interfaces and deformable object models are involved.     

Haptic‐constraint modeling based on interactive metaballs

Adding interactive haptic‐constraint sensations is important in interactive computer gaming and 3D shape design. Usually constraints are set on vertices of the object to drive the deformation. How to simulate dynamic force constraints in interactive design is still a challenging task. In this paper, we propose a novel haptic‐constraint modeling method based on interactive metaballs, during which the haptic‐constraint tools are attracted to the target location and then control the touch‐enabled deformation within the constrained areas. The interactive force feedbacks facilitate designers to accurately deform the target regions and fine carve the details as their intention on the objects. Our work studies how to apply touch sensation in such constrained deformations using interactive metaballs, thus users can truly feel and control the soft‐touch objects during the deforming interactions. Experimental results show that the dynamic sense of touch during the haptic manipulation is intuitively simulated to users, via the interacting interface we have developed. Copyright © 2010 John Wiley & Sons, Ltd.     

A Boundary Element Method for Simulation of Deformable Objects

In this paper,a boundary element method is first applied to real-tim animation of deformable objects and to simplify data preparation.Next,the visibleexternal surface of the object in deforming process is represented by B-spline surface,whose control points are embedded in dynamic equations of BEM.Fi-nally,the above method is applied to anatomical simulation.A pituitary model in human brain,which is reconstructed from a set of anatomical sections, is selected to be the deformable object under action of virtual tool such as scapel or probe.It produces fair graphic realism and high speed performance.The results show that BEM not only has less computational expense than FEM,but also is convenient to combine with the 3D reconstruction and surface modeling as it enables the reduction of the dimensionality of the problem by one.     

A Survey of Physically Based Simulation of Cuts in Deformable Bodies

Virtual cutting of deformable bodies has been an important and active research topic in physically based modelling and simulation for more than a decade. A particular challenge in virtual cutting is the robust and efficient incorporation of cuts into an accurate computational model that is used for the simulation of the deformable body. This report presents a coherent summary of the state of the art in virtual cutting of deformable bodies, focusing on the distinct geometrical and topological representations of the deformable body, as well as the specific numerical discretizations of the governing equations of motion. In particular, we discuss virtual cutting based on tetrahedral, hexahedral and polyhedral meshes, in combination with standard, polyhedral, composite and extended finite element discretizations. A separate section is devoted to meshfree methods. Furthermore, we discuss cutting‐related research problems such as collision detection and haptic rendering in the context of interactive cutting scenarios. The report is complemented with an application study to assess the performance of virtual cutting simulators.     

Layered rhombus-chain-connected model for real-time haptic rendering

The modeling and simulation of deformable objects is a challenging topic in the field of haptic rendering between human and virtual environment. In this paper, a novel and efficient layered rhombus-chain-connected haptic deformation model based on physics is proposed for an excellent haptic rendering. During the modeling, the accumulation of relative displacements in each chain structure unit in each layer is equal to the deformation on the virtual object surface, and the resultant force of corresponding springs is equivalent to the external force. The layered rhombus-chain-connected model is convenient and fast to calculate, and can satisfy real-time requirement due to its simplicity. Experimental study in both homogenous and non-homogenous virtual human liver and lungs based on the proposed model are conducted, and the results demonstrate that our model provides stable and realistic haptic feeling in real time. Meanwhile, the display result is vivid.     

Nonlinear viscoelastic contact and deformation of freeform virtual surfaces

This article is concerned with the haptic deformation display of discrete viscoelastic surfaces by means of a human fingertip. The virtual surface of a deformable quadrilateral mesh is interactively deformed by a Kelvin–Voigt soft fingertip model attached to the end-effector of a haptic interface device. In achieving this task, a nonlinear constitutive model approximating experimental data from literature is developed for determining the contact point deformations. By employing a new kernel weighting function, the deformations are distributed dependently on the discrete surface topology based on a nonlinear spring–damper net around the contact location. For illustration and evaluation of the proposed approach, a parallel robotic device with a constraint-based controller is adopted. The grip of the device is moved by the user to feel a sense of touch as the soft fingertip deforms the mesh surface of an ex vivo porcine liver tissue. Experimental data indicates stable realistic interactions thorough mechanical coupling between the soft fingertip and the deforming liver tissue. Dynamic response data of liver show rate-dependent hysteretic deformations and match closely with experimental indentation data from literature. A thorough analysis of mesh node count on the sample rate and the rendering quality is also presented.     

Interactive Rendering of Deforming NURBS Surfaces

Non-uniform rational B-splines (NURBS) has been widely accepted as a standard tool for geometry representation and design. Its rich geometric properties allow it to represent both analytic shapes and free-form curves and surfaces precisely. Moreover, a set of tools is available for shape modification or more implicitly, object deformation. Existing NURBS rendering methods include de Boor algorithm, Oslo algorithm, Shantz’s adaptive forward differencing algorithm and Silbermann’s high speed implementation of NURBS. However, these methods consider only speeding up the rendering process of individual frames. Recently, Kumar et al. proposed an incremental method for rendering NURBS surfaces, but it is still limited to static surfaces. In real-time applications such as virtual reality, interactive display is needed. If a virtual environment contains a lot of deforming objects, these methods cannot provide a good solution. In this paper, we propose an efficient method for interactive rendering of deformable objects by maintaining a polygon model of each deforming NURBS surface and adaptively refining the resolution of the polygon model. We also look at how this method may be applied to multi-resolution modelling.     

虚拟体空间中的触觉雕刻

目前，在虚拟环境中大多数的信息获取是通过视觉、听觉等非接触感觉获得的。然而缺乏触觉反馈的信息减少了很大一部分的信息源。在看和听之外，能够触摸、感觉和操纵物体，在很大程度上提高了虚拟环境的真实性。该文研究了触觉绘制的基本模型，提出了采用虚平面作为中介实现体数据的实时触觉绘制。并在此基础上探讨了体的局部变形及结合触觉反馈模型，实现了具有触觉反馈的虚拟雕刻交互系统。该系统可应用于融化、燃烧、印记、构造和着色实时交互操作。     

Realistic haptic rendering of interacting deformable objects in virtual environments

A new computer haptics algorithm to be used in general interactive manipulations of deformable virtual objects is presented. In multimodal interactive simulations, haptic feedback computation often comes from contact forces. Subsequently, the fidelity of haptic rendering depends significantly on contact space modeling. Contact and friction laws between deformable models are often simplified in up to date methods. They do not allow a "realistic" rendering of the subtleties of contact space physical phenomena (such as slip and stick effects due to friction or mechanical coupling between contacts). In this paper, we use Signorini's contact law and Coulomb's friction law as a computer haptics basis. Real-time performance is made possible thanks to a linearization of the behavior in the contact space, formulated as the so-called Delassus operator, and iteratively solved by a Gauss-Seidel type algorithm. Dynamic deformation uses corotational global formulation to obtain the Delassus operator in which the mass and stiffness ratio are dissociated from the simulation time step. This last point is crucial to keep stable haptic feedback. This global approach has been packaged, implemented, and tested. Stable and realistic 6D haptic feedback is demonstrated through a clipping task experiment.     

Quantitative assessment of the effectiveness of using display techniques with a haptic device for manipulating 3D objects in virtual environments

Quantitative assessment is made of using two display techniques, providing two different levels of depth perception, in conjunction with a haptic device for manipulating 3D objects in virtual environments. The two display techniques are 2D display, and interactive 3D stereoscopic virtual holography display on a zSpace tablet. Experiments were conducted, by several users of different ages and computer training. The experiments involved selected pointing and manipulation tasks. The speed of performing the tasks using the two display techniques were recorded. Statistical analysis of the data is presented. As expected, the use of interactive 3D stereoscopic display resulted in faster performance of the tasks. The improvement in performance was particularly noticeable for the cases wherein the subjects needed to manipulate the haptic arm to reach objects/targets at different depths, and also when the objects/targets were occluded partially by the obstacles.     

Haptics-based dynamic implicit solid modeling

We systematically present a novel, interactive solid modeling framework, haptics-based dynamic implicit solid modeling, which is founded upon volumetric implicit functions and powerful physics-based modeling. In particular, we augment our modeling framework with a haptic mechanism in order to take advantage of additional realism associated with a 3D haptic interface. Our dynamic implicit solids are semialgebraic sets of volumetric implicit functions and are governed by the principles of dynamics, hence responding to sculpting forces in a natural and predictable manner. In order to directly manipulate existing volumetric data sets as well as point clouds, we develop a hierarchical fitting algorithm to reconstruct and represent discrete data sets using our continuous implicit functions, which permit users to further design and edit those existing 3D models in real-time using a large variety of haptic and geometric toolkits, and visualize their interactive deformation at arbitrary resolution. The additional geometric and physical constraints afford more sophisticated control of the dynamic implicit solids. The versatility of our dynamic implicit modeling enables the user to easily modify both the geometry and the topology of modeled objects, while the inherent physical properties can offer an intuitive haptic interface for direct manipulation with force feedback.     

Manipulation of CAD surface models with haptics based on shape control functions

With traditional two-dimensional based interfaces, many CAD surface models are difficult to design and edit due to their 3D nature. This paper discusses a technique for the deformation of CAD surface models with haptic interaction based on shape control functions. With the technique, designers can use a haptic interface to directly touch a native B-rep CAD model, and deform it in real-time by pushing, pulling and dragging its surfaces in a natural 3D environment. The deformation is governed by shape control functions. By using the shape functions, designers can specify the area of deformation, and also have greater controls on the shape of deformation. This technique is numerically efficient, and can deform complex surface models involving several thousand control points in real-time. The haptic-based deforming approach gives designers greater flexibility for the manipulation of complex CAD surfaces.     

Real-time finite element modeling for surgery simulation: an application to virtual suturing

Real-time finite element (FE) analysis can be used to represent complex deformable geometries in virtual environments. The need for accurate surgical simulation has spurred the development of many of the new real-time FE methodologies that enable haptic support and real-time deformation. These techniques are computationally intensive and it has proved to be a challenge to achieve the high modeling resolutions required to accurately represent complex anatomies. The authors present a new real-time methodology based on linear FE analysis that is appropriate for a wide range of surgical simulation applications. A methodology is proposed that is characterized by high model resolution, low preprocessing time, unrestricted multipoint surface contact, and adjustable boundary conditions. These features make the method ideal for modeling suturing, which is an element common to almost every surgical procedure. This paper describes constraints in the context of a Suturing Simulator currently being developed by the authors.     

Dynamic Modeling and Experimental Validation for Interactive Endodontic Simulation

To facilitate training of endodontic operations, we have developed an interactive virtual environment to simulate endodontic shaping operations. This paper presents methodologies for dynamic modeling, visual/haptic display and model validation of endodontic shaping. We first investigate the forces generated in the course of shaping operations and discuss the challenging issues in their modeling. Based on the special properties and constraints associated with both pulpal tissue and endodontic files, we propose a dynamic model to simulate endodontic shaping, which is a smoothed particle based model derived for the pulpal tissue coupled with a finite element model for the endodontic files. The virtual environment has been implemented with both graphic and haptic interfaces. Furthermore, the effectiveness of the proposed model has been validated by experimental results through a novel robotic endodontic measurement system.     

Haptic sculpting of multi-resolution B-spline surfaces with shaped tools

In this paper, we first propose an implicit surface to B-spline surface haptic interface, which provides both force and torque feedback. We then present a new haptic sculpting system for B-spline surfaces with shaped tools of implicit surface. In the physical world, people touch or sculpt with their fingers or tools, instead of just manipulating points. Shaped virtual sculpting tools help users to relate the virtual modeling process to physical-world experience. Various novel haptic sculpting operations are developed to make the sculpting of B-spline surfaces more intuitive. Wavelet-based multi-resolution tools are provided to let modelers adjust the resolution of sculpture surfaces and thus the scale of deformation can be easily controlled. Moreover, sweep editing and 3D texture have been implemented by taking advantage of both the wavelet technique and haptic sculpting tools.     

An efficient dynamic point algorithm for line‐based collision detection in real time virtual environments involving haptics

In real time computer graphics, “interactivity” is limited to a display rate of 30 frames per second. However, in multimodal virtual environments involving haptic interactions, a much higher update rate of about 1 kHz is necessary to ensure continuous interactions and smooth transitions. The simplest and most efficient interaction paradigm in such environments is to represent the haptic cursor as a point. However, in many situations, such as those in the development of real time medical simulations involving the interactions of long slender surgical tools with soft deformable organs, such a paradigm is nonrealistic and at least a line‐based interaction is desirable. While such paradigms exist, the main impediment to their widespread use is the associated computational complexity. In this paper, we introduce, for the first time, an efficient algorithm for computing the interaction of a line‐shaped haptic cursor and polygonal surface models which has a near constant complexity. The algorithm relies on space‐time coherence, topological information, and the properties of lines in 3D space to maintain proximity information between a line segment and triangle meshes. For interaction with convex objects, the line is represented by its end points and a dynamic point, which is the closest point on the line to any potentially colliding triangle. To deal with multiple contacts and non‐convexities, the line is decomposed into segments and a dynamic point is used for each segment. The algorithm may be used to compute collision detection and response with rigid as well as deformable objects with no performance penalty. Realistic examples are presented to demonstrate the effectiveness of our approach. Copyright © 2008 John Wiley & Sons, Ltd.     

Image-based collision detection for deformable cloth models

Modeling the natural interaction of cloth and garments with objects in a 3D environment is currently one of the most computationally demanding tasks. These highly deformable materials are subject to a very large number of contact points in the proximity of other moving objects. Furthermore, cloth objects often fold, roll, and drape within themselves, generating a large number of self-collision areas. The interactive requirements of 3D games and physically driven virtual environments make the cloth collisions and self-collision computations more challenging. By exploiting mathematically well-defined smoothness conditions over smaller patches of deformable surfaces and resorting to image-based collision detection tests, we developed an efficient collision detection method that achieves interactive rates while tracking self-interactions in highly deformable surfaces consisting of a large number of elements. The method makes use of a novel technique for dynamically generating a hierarchy of cloth bounding boxes in order to perform object-level culling and image-based intersection tests using conventional graphics hardware support. An efficient backward voxel-based AABB hierarchy method is proposed to handle deformable surfaces which are highly compressed.     

Design and validation of a complete haptic system for manipulative tasks

The present work deals with the design, implementation and assessment of a new haptic system specifically conceived for manipulative tasks in virtual environments. Such a system was designed by taking into account specific issues related to fine manipulation, such as multipoint haptics, coherence, transparency and physical representation. The haptic system described herein is integrated with a virtual environment engine for the simulation of multifinger manipulation. A preliminary evaluation of the system was conducted by comparing human performance in the manipulation of virtual objects with respect to real objects, according to the data available in the literature. The experiments confirm how the most relevant relationships among physiological and physical parameters involved in manipulation are also preserved during virtual manipulation. However, an in-depth analysis of the results shows that simulation parameters affect the level of force control during virtual manipulation and the quality of the perceived force feedback.     

On the Synthesis of Haptic Textures

Advanced, synthetic haptic virtual environments require textured virtual surfaces. We found that texturing smooth surfaces often reduces the system passivity margin of a haptic simulation. As a result, a smooth virtual surface that can be rendered in a passive manner may loose this property once textured. We propose that any texture algorithm is associated with a characteristic number that expresses the relative change in loop gain. We further found that a passive virtual interaction can have severe unwanted artifacts if the synthesized force field is not conservative. The energy characteristics of seven algorithms are analyzed. Finally, a new texture synthesis algorithm, which operates by modulating a friction force during scanning, is shown to have several advantages over previous ones.     

网上3D试衣系统技术研究

虚拟试衣系统是一种应用于服装电子商务的实时交互平台。为了满足虚拟试衣系统真实性、实时性、方便性等设计要求,该文采用服装操控技术为虚拟服装建立三维模型,并把其中的交互建模标记绑定到服装模型上,以此取代手工指定的标记,从而把原来需要人机交互的三维服装建模过程转化成一个后台计算过程。保留服装操控技术中的表面拖曳交互机制,作为用户对建模完成之后的虚拟服装的微调手段,并把原来需要人机交互指定的拖曳不动点信息固化到服装模型数据文件,以此提高虚拟试衣间实际应用中的方便性。在网上3D试衣系统的设计中,提出了布片无缝拼接算法以及二面角调整算法,并在实验中取得了较好的效果。