The reconstruction of dynamic 3D structure of biological objects using stereo microscope images

In this paper, we address the analysis of 3D shape and shape change in non-rigid biological objects imaged via a stereo light microscope. We propose an integrated approach for the reconstruction of 3D structure and the motion analysis for images in which only a few informative features are available. The key components of this framework are: 1) image registration using a correlation-based approach, 2) region-of-interest extraction using motion-based segmentation, and 3) stereo and motion analysis using a cooperative spatial and temporal matching process. We describe these three stages of processing and illustrate the efficacy of the proposed approach using real images of a live frog's ventricle. The reconstructed dynamic 3D structure of the ventricle is demonstrated in our experimental results, and it agrees qualitatively with the observed images of the ventricle.     

A word extraction algorithm for machine-printed documents using a 3D neighborhood graph model

Automatic character recognition and image understanding of a given paper document are the main objectives of the computer vision field. For these problems, a basic step is to isolate characters and group words from these isolated characters. In this paper, we propose a new method for extracting characters from a mixed text/graphic machine-printed document and an algorithm for distinguishing words from the isolated characters. For extracting characters, we exploit several features (size, elongation, and density) of characters and propose a characteristic value for classification using the run-length frequency of the image component. In the context of word grouping, previous works have largely been concerned with words which are placed on a horizontal or vertical line. Our word grouping algorithm can group words which are on inclined lines, intersecting lines, and even curved lines. To do this, we introduce the 3D neighborhood graph model which is very useful and efficient for character classification and word grouping. In the 3D neighborhood graph model, each connected component of a text image segment is mapped onto 3D space according to the area of the bounding box and positional information from the document. We conducted tests with more than 20 English documents and more than ten oriental documents scanned from books, brochures, and magazines. Experimental results show that more than 95% of words are successfully extracted from general documents, even in very complicated oriental documents. Received August 3, 2001 / Accepted August 8, 2001     

A JPEG-like texture compression with adaptive quantization for 3D graphics application

Published online: 2 October 2001     

Parameterization for reconstruction of 3D freeform objects from laser-scanned data based on a PDE method

Published online: 19 July 2001     

基于几何元素的城市三维地理信息系统空间数据模型

空间数据模型是城市三维地理信息系统建立的核心问题。基于不同的城市空间对象划分方法,提出了城市三维地理信息系统的十八个空间数据模型:把空间现象抽象为六类对象:点对象、线对象、规则面对象、规则体对象、不规则对象和DTM,基于六种几何元素(点、直线段、多边形、函数构造面、TIN面片、规则体素)的不同组合,提出了城市三维地理信息系统的九个空间数据模型;把空间现象抽象为五类对象:0维空间对象(点对象)、一维空间对象(线对象)、二维空间对象(面对象)、三维空间对象(体对象)和DTM,基于六种几何元素的不同组合提出了八个空间数据模型;把空间现象抽象为0维对象(点对象)、一维对象(线对象)、二维对象(面对象)、三维对象(体对象)、DTM,基于点、直线段、TIN面片、多边形、规则面、不规则面六类几何元素提出了一个空间数据模型。通过对一个数据模型的实验验证和各模型的理论分析,表明了模型的特点及可行性。     

The perceptive workbench: Computer-vision-based gesture tracking,object tracking,and 3D reconstruction for augmented desks

Abstract. The Perceptive Workbench endeavors to create a spontaneous and unimpeded interface between the physical and virtual worlds. Its vision-based methods for interaction constitute an alternative to wired input devices and tethered tracking. Objects are recognized and tracked when placed on the display surface. By using multiple infrared light sources, the object's 3-D shape can be captured and inserted into the virtual interface. This ability permits spontaneity, since either preloaded objects or those objects selected at run-time by the user can become physical icons. Integrated into the same vision-based interface is the ability to identify 3-D hand position, pointing direction, and sweeping arm gestures. Such gestures can enhance selection, manipulation, and navigation tasks. The Perceptive Workbench has been used for a variety of applications, including augmented reality gaming and terrain navigation. This paper focuses on the techniques used in implementing the Perceptive Workbench and the system's performance.     

Multiple expert classification: a new methodology for parallel decision fusion

Abstract. A new parallel hybrid decision fusion methodology is proposed. It is demonstrated that existing parallel multiple expert decision combination approaches can be divided into two broad categories based on the implicit decision emphasis implemented. The first category consists of methods implementing computationally intensive decision frameworks incorporating a priori information about the target task domain and the reliability of the participating experts, while the second category encompasses approaches implementing group consensus without assigning any importance to the reliability of the experts and ignoring other contextual information. The methodology proposed in this paper is a hybridisation of these two approaches and has shown significant performance enhancements in terms of higher overall recognition rates along with lower substitution rates. Detailed analysis using two different databases supports this claim. Received January 19, 1999 / Revised March 20, 2000     

Simulation of postoperative 3D facial morphology using a physics-based head model

We propose a prototype of a facial surgery simulation system for surgical planning and the prediction of facial deformation. We use a physics-based human head model. Our head model has a 3D hierarchical structure that consists of soft tissue and the skull, constructed from the exact 3D CT patient data. Anatomic points measured on X-ray images from both frontal and side views are used to fire the model to the patient's head. The purposes of this research is to analyze the relationship between changes of mandibular position and facial morphology after orthognathic surgery, and to simulate the exact postoperative 3D facial shape. In the experiment, we used our model to predict the facial shape after surgery for patients with mandibular prognathism. Comparing the simulation results and the actual facial images after the surgery shows that the proposed method is practical.     

A stereovision system for a planetary rover: calibration, correlation, registration, and fusion

This paper describes a complete stereovision system, which was originally developed for planetary applications, but can be used for other applications such as object modeling. A new effective on-site calibration technique has been developed, which can make use of the information from the surrounding environment as well as the information from the calibration apparatus. A correlation-based stereo algorithm is used, which can produce sufficient dense range maps with an algorithmic structure for fast implementations. A technique based on iterative closest-point matching has been developed for registration of successive depth maps and computation of the displacements between successive positions. A statistical method based on the distance distribution is integrated into this registration technique, which allows us to deal with such important problems as outliers, occlusion, appearance, and disappearance. Finally, the registered maps are expressed in the same coordinate system and are fused, erroneous data are eliminated through consistency checking, and a global digital elevation map is built incrementally.     

Automatic reconstruction of 3D human arm motion from a monocular image sequence

A model-based approach to reconstruction of 3D human arm motion from a monocular image sequence taken under orthographic projection is presented. The reconstruction is divided into two stages. First, a 2D shape model is used to track the arm silhouettes and second-order curves are used to model the arm based on an iteratively reweighted least square method. As a result, 2D stick figures are extracted. In the second stage, the stick figures are backprojected into the scene. 3D postures are reconstructed using the constraints of a 3D kinematic model of the human arm. The motion of the arm is then derived as a transition between the arm postures. Applications of these results are foreseen in the analysis of human motion patterns. Received: 26 January 1996 / Accepted: 17 July 1997     

Tracking the human arm using constraint fusion and multiple-cue localization

Abstract. The use of hand gestures provides an attractive means of interacting naturally with a computer-generated display. Using one or more video cameras, the hand movements can potentially be interpreted as meaningful gestures. One key problem in building such an interface without a restricted setup is the ability to localize and track the human arm robustly in video sequences. This paper proposes a multiple-cue localization scheme combined with a tracking framework to reliably track the dynamics of the human arm in unconstrained environments. The localization scheme integrates the multiple cues of motion, shape, and color for locating a set of key image features. Using constraint fusion, these features are tracked by a modified extended Kalman filter that exploits the articulated structure of the human arm. Moreover, an interaction scheme between tracking and localization is used for improving the estimation process while reducing the computational requirements. The performance of the localization/tracking framework is validated with the help of extensive experiments and simulations. These experiments include tracking with calibrated stereo camera and uncalibrated broadcast video. Received: 19 January 2001 / Accepted: 27 December 2001 Correspondence to: R. Sharma     

Acquiring, stitching and blending diffuse appearance attributes on 3D models

Published online: 23 April 2002     

Layered animation of captured data

normal volume of a triangle to convert individual triangles to a volumetric representation. A layered model is constructed to animate the reconstructed high-resolution surface. The model consists of 3 layers: a skeleton for animation from key-frame or motion capture; a low-resolution control model for real-time mesh deformation; and a high-resolution model to represent the captured surface detail. Initially the skeleton model is manually placed inside the low-resolution control model and high-resolution scanned data. Automatic techniques are introduced to map both the control model and captured data into a single layered model. The high-resolution captured data is mapped onto the low-resolution control model using the normal volume. The resulting model enables efficient, seamless animation by manipulation of the skeleton while maintaining the captured high-resolution surface detail. The animation of high-resolution captured data based on a low-resolution generic model of the object opens up the possibility of rapid capture and animation of new objects based on libraries of generic models. Published online: 2 October 2001     

Machine vision system for curved surface inspection

This application-oriented paper discusses a non-contact 3D range data measurement system to improve the performance of the existing 2D herring roe grading system. The existing system uses a single CCD camera with unstructured halogen lighting to acquire and analyze the shape of the 2D shape of the herring roe for size and deformity grading. Our system will act as an additional system module, which can be integrated into the existing 2D grading system, providing the additional third dimension to detect deformities in the herring roe, which were not detected in the 2D analysis. Furthermore, the additional surface depth data will increase the accuracy of the weight information used in the existing grading system. In the proposed system, multiple laser light stripes are projected into the herring roe and the single B/W CCD camera records the image of the scene. The distortion in the projected line pattern is due to the surface curvature and orientation. Utilizing the linear relation between the projected line distortion and surface depth, the range data was recovered from a single camera image. The measurement technique is described and the depth information is obtained through four steps: (1) image capture, (2) stripe extraction, (3) stripe coding, (4) triangulation, and system calibration. Then, this depth information can be converted into the curvature and orientation of the shape for deformity inspection, and also used for the weight estimation. Preliminary results are included to show the feasibility and performance of our measurement technique. The accuracy and reliability of the computerized herring roe grading system can be greatly improved by integrating this system into existing system in the future.     

Using vanishing points for camera calibration and coarse 3D reconstruction from a single image

In this paper, we show how to calibrate a camera and to recover the geometry and the photometry (textures) of objects from a single image. The aim of this work is to make it possible walkthrough and augment reality in a 3D model reconstructed from a single image. The calibration step does not need any calibration target and makes only four assumptions: (1) the single image contains at least two vanishing points, (2) the length (in 3D space) of one line segment (for determining the translation vector) in the image is known, (3) the principle point is the center of the image, and (4) the aspect ratio is fixed by the user. Each vanishing point is determined from a set of parallel lines. These vanishing points help determine a 3D world coordinate system R _o. After having computed the focal length, the rotation matrix and the translation vector are evaluated in turn for describing the rigid motion between R _o and the camera coordinate system R _c. Next, the reconstruction step consists in placing, rotating, scaling, and translating a rectangular 3D box that must fit at best with the potential objects within the scene as seen through the single image. With each face of a rectangular box, a texture that may contain holes due to invisible parts of certain objects is assigned. We show how the textures are extracted and how these holes are located and filled. Our method has been applied to various real images (pictures scanned from books, photographs) and synthetic images.     

Using 3D Sound to Improve the Effectiveness of the Advanced Driver Assistance Systems

Future generation cars will be characterized by a wide range of Information Technology (IT) services providing safety and infotainment. This makes the car an information intensive environment where the visual channel is overloaded, putting the safety of drivers and passengers in jeopardy. We propose the use of a 3D auditory display to provide information from the Advanced Driver Assistance Systems. This reduces the eye-off-road time, exploiting the human capability to associate sounds with positions in space. Preliminary lab tests reveal the suitability of this approach. The system still has to be carefully tuned and personalized to achieve usability and reliability, but we think that it provides a complementary channel that is specially useful in low visibility conditions.     

A system for a new two-dimensional code: Secure 2D code

A new 2D code called Secure 2D code is designed in this paper, both encoder and decoder are also proposed. Secure 2D code can store any kind of data and provides high security. With regard to security, the input data is divided into two parts: general and secret. The general data is transformed into a 2D code pattern, then secret data is hidden in the 2D code pattern. To raise the reading speed and allow various reading environments, some features are added around the 2D code pattern boundary. As to the reliability, RS code is adopted to treat damaged patterns. Received: 9 September 1997 / Accepted: 2 March 1998