A haptic-rendering technique based on hybrid surface representation

A novel haptic rendering technique using a hybrid surface representation addresses conventional limitations in haptic displays. A haptic interface lets the user touch, explore, paint, and manipulate virtual 3D models in a natural way using a haptic display device. A haptic rendering algorithm must generate a force field to simulate the presence of these virtual objects and their surface properties (such as friction and texture), or to guide the user along a specific trajectory. We can roughly classify haptic rendering algorithms according to the surface representation they use: geometric haptic algorithms for surface data, and volumetric haptic algorithms based on volumetric data including implicit surface representation. Our algorithm is based on a hybrid surface representation - a combination of geometric (B-rep) and implicit (V-rep) surface representations for a given 3D object, which takes advantage of both surface representations.     

VR-CAD integration: Multimodal immersive interaction and advanced haptic paradigms for implicit edition of CAD models

This paper presents an approach for the integration of Virtual Reality (VR) and Computer-Aided Design (CAD). Our general goal is to develop a VR-CAD framework making possible intuitive and direct 3D edition on CAD objects within Virtual Environments (VE). Such a framework can be applied to collaborative part design activities and to immersive project reviews. The cornerstone of our approach is a model that manages implicit editing of CAD objects. This model uses a naming technique of B-Rep components and a set of logical rules to provide straight access to the operators of Construction History Graphs (CHG). Another set of logical rules and the replay capacities of CHG make it possible to modify in real-time the parameters of these operators according to the user’s 3D interactions. A demonstrator of our model has been developed on the OpenCASCADE geometric kernel, but we explain how it can be applied to more standard CAD systems such as CATIA. We combined our VR-CAD framework with multimodal immersive interaction (using 6 DoF tracking, speech and gesture recognition systems) to gain direct and intuitive deformation of the objects’ shapes within a VE, thus avoiding explicit interactions with the CHG within a classical WIMP interface. In addition, we present several haptic paradigms specially conceptualized and evaluated to provide an accurate perception of B-Rep components and to help the user during his/her 3D interactions. Finally, we conclude on some issues for future researches in the field of VR-CAD integration.     

Function-based approach to mixed haptic effects rendering

Commonly, surface and solid haptic effects are defined in such a way that they hardly can be rendered together. We propose a method for defining mixed haptic effects including surface, solid, and force fields. These haptic effects can be applied to virtual scenes containing various objects, including polygon meshes, point clouds, impostors, and layered textures, voxel models as well as function-based shapes. Accordingly, we propose a way how to identify location of the haptic tool in such virtual scenes as well as consistently and seamlessly determine haptic effects when the haptic tool moves in the scenes with objects having different sizes, locations, and mutual penetrations. To provide for an efficient and flexible rendering of haptic effects, we propose to concurrently use explicit, implicit and parametric functions, and algorithmic procedures.     

Haptics-based dynamic implicit solid modeling

We systematically present a novel, interactive solid modeling framework, haptics-based dynamic implicit solid modeling, which is founded upon volumetric implicit functions and powerful physics-based modeling. In particular, we augment our modeling framework with a haptic mechanism in order to take advantage of additional realism associated with a 3D haptic interface. Our dynamic implicit solids are semialgebraic sets of volumetric implicit functions and are governed by the principles of dynamics, hence responding to sculpting forces in a natural and predictable manner. In order to directly manipulate existing volumetric data sets as well as point clouds, we develop a hierarchical fitting algorithm to reconstruct and represent discrete data sets using our continuous implicit functions, which permit users to further design and edit those existing 3D models in real-time using a large variety of haptic and geometric toolkits, and visualize their interactive deformation at arbitrary resolution. The additional geometric and physical constraints afford more sophisticated control of the dynamic implicit solids. The versatility of our dynamic implicit modeling enables the user to easily modify both the geometry and the topology of modeled objects, while the inherent physical properties can offer an intuitive haptic interface for direct manipulation with force feedback.     

SnapPaste: an interactive technique for easy mesh composition

Editing and manipulation of existing 3D geometric objects are a means to extend their repertoire and promote their availability. Traditionally, tools to compose or manipulate objects defined by 3D meshes are in the realm of artists and experts. In this paper, we introduce a simple and effective user interface for easy composition of 3D mesh-parts for non-professionals. Our technique borrows from the cut-and-paste paradigm where a user can cut parts out of existing objects and paste them onto others to create new designs. To assist the user attach objects to each other in a quick and simple manner, many applications in computer graphics support the notion of “snapping”. Similarly, our tool allows the user to loosely drag one mesh part onto another with an overlap, and lets the system snap them together in a graceful manner. Snapping is accomplished using our Soft-ICP algorithm which replaces the global transformation in the ICP algorithm with a set of point-wise locally supported transformations. The technique enhances registration with a set of rigid to elastic transformations that account for simultaneous global positioning and local blending of the objects. For completeness of our framework, we present an additional simple mesh-cutting tool, adapting the graph-cut algorithm to meshes.     

Modeling and Exploring Co‐variations in the Geometry and Configuration of Man‐made 3D Shape Families

We introduce co‐variation analysis as a tool for modeling the way part geometries and configurations co‐vary across a family of man‐made 3D shapes. While man‐made 3D objects exhibit large geometric and structural variations, the geometry, structure, and configuration of their individual components usually do not vary independently from each other but in a correlated fashion. The size of the body of an airplane, for example, constrains the range of deformations its wings can undergo to ensure that the entire object remains a functionally‐valid airplane. These co‐variation constraints, which are often non‐linear, can be either physical, and thus they can be explicitly enumerated, or implicit to the design and style of the shape family. In this article, we propose a data‐driven approach, which takes pre‐segmented 3D shapes with known component‐wise correspondences and learns how various geometric and structural properties of their components co‐vary across the set. We demonstrate, using a variety of 3D shape families, the utility of the proposed co‐variation analysis in various applications including 3D shape repositories exploration and shape editing where the propagation of deformations is guided by the co‐variation analysis. We also show that the framework can be used for context‐guided orientation of objects in 3D scenes.     

Object Repositioning Based on the Perspective in a Single Image

We propose an image editing system for repositioning objects in a single image based on the perspective of the scene. In our system, an input image is transformed into a layer structure that is composed of object layers and a background layer, and then the scene depth is computed from the ground region that is specified by the user using a simple boundary line. The object size and order of overlapping are automatically determined during the reposition based on the scene depth. In addition, our system enables the user to move shadows along with objects naturally by extracting the shadow mattes using only a few user‐specified scribbles. Finally, we demonstrate the versatility of our system through applications to depth‐of‐field effects, fog synthesis and 3D walkthrough in an image.     

Three-Dimensional Modeling and Effects on Still Images

Designers and creative artists use computer graphics and image processing effects on stall photographs in application areas such as advertising entertainment broadcasting and the arts Most of the effects available in research arid commercial work are two-dimensional in nature, for example image processing filters [blur, edge enhancement) and creative effects (tilings, reflections) There is almost no usage of information taken from the 3-D world in which the objects appearing an the image are located. In this paper we present a novel method for creating 3-D effects on photographs or in general on any image created by rendering a 3-D world The artist interacts with the image using a set of intuitive direct manipulation interface objects These objects let the user define a 3-D model, display at, and manipulate it in a 3-D space which is correlated with that of the input image. The generated model can be an arbitrarily complex 3-D polyhedron Any texture, including texture taken from the input photograph, can be mapped into any of its faces arid used for special effects We discuss and show examples for effects such as copy and paste, motion blur, model editing and deformations lighting effects, and shadows.     

B‐Mesh: A Modeling System for Base Meshes of 3D Articulated Shapes

This paper presents a novel modeling system, called B‐Mesh, for generating base meshes of 3D articulated shapes. The user only needs to draw a one‐dimensional skeleton and to specify key balls at the skeletal nodes. The system then automatically generates a quad dominant initial mesh. Further subdivision and evolution are performed to refine the initial mesh and generate a quad mesh which has good edge flow along the skeleton directions. The user can also modify and manipulate the shape by editing the skeleton and the key balls and can easily compose new shapes by cutting and pasting existing models in our system. The mesh models generated in our system greatly benefit the sculpting operators for sculpting modeling and skeleton‐based animation.     

Object interaction and task programming by demonstration in visuo-haptic augmented reality

A visuo-haptic augmented reality system is presented for object manipulation and task learning from human demonstration. The proposed system consists of a desktop augmented reality setup where users operate a haptic device for object interaction. Users of the haptic device are not co-located with the environment where real objects are present. A three degrees of freedom haptic device, providing force feedback, is adopted for object interaction by pushing, selection, translation and rotation. The system also supports physics-based animation of rigid bodies. Virtual objects are simulated in a physically plausible manner and seem to coexist with real objects in the augmented reality space. Algorithms for calibration, object recognition, registration and haptic rendering have been developed. Automatic model-based object recognition and registration are performed from 3D range data acquired by a moving laser scanner mounted on a robot arm. Several experiments have been performed to evaluate the augmented reality system in both single-user and collaborative tasks. Moreover, the potential of the system for programming robot manipulation tasks by demonstration is investigated. Experiments show that a precedence graph, encoding the sequential structure of the task, can be successfully extracted from multiple user demonstrations and that the learned task can be executed by a robot system.     

The effects of cast shadows and stereopsis on performing computer-generated spatial tasks

More powerful computer hardware and software has enabled the development of numerous depth-cue techniques to depict the three-dimensional (3D) characteristics of computer-generated imagery. Particularly, the role of stereoscopic viewing in promoting the depth perception of objects in 3D space has been widely studied. However, there has been little study of the use of cast shadows in this context, and few investigations of how computer-generated depth cues are perceptually integrated. To investigate these issues, subjects: 1) position a sphere to complete a directional vector/vector segment extending in space and 2) resize a sphere to match the perceived size of a second sphere displaced in space. Task accuracies and response times are recorded under the following conditions: 1) stereoscopic, and monoscopic, scene viewing; 2) objects casting zero, or one shadow; 3) flat, and zig-zag, background surface shapes; and 4) solid and checkerboard background surface textures. The results suggest that, of the cues considered, stereo viewing has a consistently beneficial effect on positioning and resizing spheres in depth. However, objects casting shadows improve positioning accuracy to a level afforded by stereo viewing. Finally, more complex scene backgrounds impair positioning and resizing task performances. Implications for the design of effective 3D user interfaces are discussed.     

Real-time knot-tying simulation

The real-time simulation of rope, and knot tying in particular, raises difficult issues in contact detection and management. Some practical knots can only be achieved by complicated crossings of the rope, yielding multiple simultaneous contacts, especially when the rope is pulled tight. This paper describes a graphical simulator that allows a user to grasp and smoothly manipulate a virtual rope and to tie arbitrary knots, including knots around other objects, in real time. A first component of the simulator computes the global configuration of the rope based on user interactions. Another component of the simulator precisely detects self-collisions in the rope as well as collisions with other objects. Finally, a third component manages collisions to prevent penetration, while making the rope slide with some friction along itself and other objects, so that knots can be pulled tight in a realistic manner. An additional module uses recent results from knot theory to identify, also in real time, which topological knots have been tied. This work was motivated by surgical suturing, but simulation in other domains, such as sailing and rock climbing, could also benefit from it.     

Quantitative assessment of the effectiveness of using display techniques with a haptic device for manipulating 3D objects in virtual environments

Quantitative assessment is made of using two display techniques, providing two different levels of depth perception, in conjunction with a haptic device for manipulating 3D objects in virtual environments. The two display techniques are 2D display, and interactive 3D stereoscopic virtual holography display on a zSpace tablet. Experiments were conducted, by several users of different ages and computer training. The experiments involved selected pointing and manipulation tasks. The speed of performing the tasks using the two display techniques were recorded. Statistical analysis of the data is presented. As expected, the use of interactive 3D stereoscopic display resulted in faster performance of the tasks. The improvement in performance was particularly noticeable for the cases wherein the subjects needed to manipulate the haptic arm to reach objects/targets at different depths, and also when the objects/targets were occluded partially by the obstacles.     

Vector Fields: an Interactive Tool for Animation, Modeling and Simulation with Physically Based 3D Particle Systems and Soft Objects

Vectorfields have traditionally been used in computer graphics as a means of visualizing models over time. This paper presents a system which extends the use of vector fields as an interactive tool for physically based three dimensional particle systems and soft objects. The techniques implemented in the system provide the user with new flexibility in animation, modeling and simulation. This paper describes bounded interactive vectorfields and how they can be used to manipulate particle systems and a class of soft objects. Applications to animation, modeling and simulation are also presented.     

A Survey of Haptic Rendering Techniques

Computer Graphics technologies have developed considerably over the past decades. Realistic virtual environments can be produced incorporating complex geometry for graphical objects and utilising hardware acceleration for per pixel effects. To enhance these environments, in terms of the immersive experience perceived by users, the human's sense of touch, or haptic system, can be exploited. To this end haptic feedback devices capable of exerting forces on the user are incorporated. The process of determining a reaction force for a given position of the haptic device is known as haptic rendering. For over a decade users have been able to interact with a virtual environment with a haptic device. This paper focuses on the haptic rendering algorithms which have been developed to compute forces as users manipulate the haptic device in the virtual environment.     

Head-Tracked Stereo Viewing with Two-Handed 3 D Interaction for Animated Character Construction

In this paper, we demonstrate how a new interactive 3 D desktop metaphor based on two-handed 3 D direct manipulation registered with head-tracked stereo viewing can be applied to the task of constructing animated characters. In our configuration, a six degree-of-freedom head-tracker and CrystalEyes shutter glasses are used to produce stereo images that dynamically follow the user head motion. 3 D virtual objects can be made to appear at a fixed location in physical space which the user may view from different angles by moving his head. To construct 3 D animated characters, the user interacts with the simulated environment using both hands simultaneously: the left hand, controlling a Spaceball, is used for 3 D navigation and object movement, while the right hand, holding a 3 D mouse, is used to manipulate through a virtual tool metaphor the objects appearing in front of the screen. In this way, both incremental and absolute interactive input techniques are provided by the system. Hand-eye coordination is made possible by registering virtual space exactly to physical space, allowing a variety of complex 3 D tasks necessary for constructing 3 D animated characters to be performed more easily and more rapidly than is possible using traditional interactive techniques. The system has been tested using both Polhemus Fastrak and Logitech ultrasonic input devices for tracking the head and 3 D mouse.     

Free Appearance-Editing with Improved Poisson Image Cloning

In this paper, we present a new edit tool for the user to conveniently preserve or freely edit the object appearance during seamless image composition. We observe that though Poisson image editing is effective for seamless image composition. Its color bleeding (the color of the target image is propagated into the source image) is not always desired in applications, and it provides no way to allow the user to edit the appearance of the source image. To make it more flexible and practical, we introduce new energy terms to control the appearance change, and integrate them into the Poisson image editing framework. The new energy function could still be realized using efficient sparse linear solvers, and the user can interactively refine the constraints. With the new tool, the user can enjoy not only seamless image composition, but also the flexibility to preserve or manipulate the appearance of the source image at the same time. This provides more potential for creating new images. Experimental results demonstrate the effectiveness of our new edit tool, with similar time cost to the original Poisson image editing.     

Dynamic Maps for Exploring and Browsing Shapes

Large datasets of 3D objects require an intuitive way to browse and quickly explore shapes from the collection. We present a dynamic map of shapes where similar shapes are placed next to each other. Similarity between 3D models exists in a high dimensional space which cannot be accurately expressed in a two dimensional map. We solve this discrepancy by providing a local map with pan capabilities and a user interface that resembles an online experience of navigating through geographical maps. As the user navigates through the map, new shapes appear which correspond to the specific navigation tendencies and interests of the user, while maintaining a continuous browsing experience. In contrast with state of the art methods which typically reduce the search space by selecting constraints or employing relevance feedback, our method enables exploration of large sets without constraining the search space, allowing the user greater creativity and serendipity. A user study evaluation showed a strong preference of users for our method over a standard relevance feedback method.     

An Interactive Designing System with Virtual Sculpting and Virtual Woodcut Printing

In this paper, we propose an interactive designing method and a system based on it to create 3D objects and 2D images. This system consists of two subsystems for virtual sculpting to create a 3D shape and virtual printing to produce a picture with a printing block. In the virtual sculpting subsystem, a user can form solid objects with curved surfaces as if sculpting them. The user operates virtual chisels, and can remove or attach arbitrary shapes of ellipsoids or cubes from or to the workpiece. A 3D object generated by virtual sculpting looks like a real wooden sculpture. If using a board as a workpiece, a user can generate a virtual printing block. In the virtual printing subsystem, a user can synthesize a woodcut printing image from the virtual printing block mentioned above, a virtual paper sheet, and a printing brush. The user can synthesize a realistic woodcut print with a procedure similar to the actual woodcut printing.     

Haptic state surface interactions

Haptic state-surface interaction techniques, which encode modes into force states, make interactions with 3D objects more fluid and natural. Our techniques comprise only a subset of those necessary to support a fully functional haptic application with capabilities similar to a drawing program such as Adobe Illustrator. However, these techniques can potentially simplify some common operations, such as editing points on a curve.