Area-Based Matching for Simultaneous Registration of Multiple 3-D Profile Maps

We propose an area-based method for the simultaneous registration of multiple 3-D data sets without known correspondences when a rough initial registration is known. The method is applicable to data sets which can be represented as single valued parametric surfaces. We introduce such a representation called a 3-D profile map for the data acquired by light striping. During registration, the relative orientations of the profile maps measured from different viewpoints are iteratively refined by matching the parametric surfaces in the overlapping areas in the sense that the mean of the squares of weighted distances between the surfaces is minimized. Incompatible matches are rejected by an adaptive weighting function and the minimization task is solved simultaneously for all the maps. Hierarchical techniques are utilized in order to better localize a global minimum and to speed up the processing. The computational operations are formulated using the notation of an image algebra and realized in a parallel manner thus reducing further the computation time. Test results are shown with real data.     

Wavelet-Based De-noising of Positron Emission Tomography Scans

A method to improve the signal-to-noise-ratio (SNR)of positron emission tomography (PET) scans is presented. A wavelet-based image decomposition technique decomposes an image into two parts, one which primarily contains the desired restored image and the other primarily the remaining unwanted portion of the image. Because the method is based on a texture extraction model that identifies the desired image in the space of bounded variation, these restorations are approximations of piecewise constant images, and are referred to as the cartoon part of the image. Here an approximation using a wavelet decomposition is used which allows solutions to be computed very efficiently. To process 3-D volume data a slice by slice approach in all three directions is adopted. Using a redundant discrete wavelet transform, 3-D restorations can be efficiently computed on standard desktop computers. The method is illustrated for PET images which have been reconstructed from simulated data using the expectation maximization algorithm. When post-processed by the presented wavelet decomposition they show a significant increase in SNR. It is concluded that the new wavelet based method can be used as an alternative to the well established de-noising of PET scans by smoothing with a Gaussian point spread function. In particular, if the volume data are reconstructed using the EM algorithm with a larger number of iterations than the number of iterations that would be used without post-processing, the 3-D images are sharper and show more detail. A MATLAB^® based graphical user interface is provided that allows easy exploration of the impact of parameter choices.     

Three dimensional reconstruction of coronary arteries from two views

Geometric representation and measurements of localized lumen stenosis of coronary arteries are important considerations in the diagnosis of cardiovascular diseases. This discrete narrowing of the arteries typically impairs blood flow in regions of the heart, and can be present along the entire length of the artery. Three-dimensional (3-D) reconstruction of coronary arterial tree allows clinician to visualize vascular geometry. Three-dimensional representation of tree topology facilitates calculation of hemodynamic measurements to study myocardial infarction and stenosis. The 3-D arterial tree, computed from two views, can provide more information about the tree geometry than individual views. In this paper, a 3-step algorithm for 3-D reconstruction of arterial tree using two standard views is presented. The first step is a multi-resolution segmentation of the coronary vessels followed by medial-axis detection along the entire arterial tree for both views. In the second step, arterial trees from the two views are registered using medial-axis representation at the coarsest resolution level to obtain an initial 3-D reconstruction. This initial reconstruction at the coarsest level is then modified using 3-D geometrical a priori information. In the third step, the modified reconstruction is projected on the next higher-resolution segmented medial-axis representation and an updated reconstruction is obtained at the higher resolution. The process is iterated until the final 3-D reconstruction is obtained at the finest resolution level. Linear programming based constrained optimization method is used for registering two views at the coarse resolution. This is followed by a Tree-Search method for registering detailed branches at higher resolutions. The automated 3-D reconstruction method was evaluated on computer-simulated as well as human angiogram data. Results show that the automated 3-D reconstruction method provided good registration of computer-simulated data. On human angiogram data, the computed 3-D reconstruction matched well with manual registration.     

Joined Spectral Trees for Scalable SPIHT-Based Multispectral Image Compression

In this paper, the compression of multispectral images is addressed. Such 3-D data are characterized by a high correlation across the spectral components. The efficiency of the state-of-the-art wavelet-based coder 3-D SPIHT is considered. Although the 3-D SPIHT algorithm provides the obvious way to process a multispectral image as a volumetric block and, consequently, maintain the attractive properties exhibited in 2-D (excellent performance, low complexity, and embeddedness of the bit-stream), its 3-D trees structure is shown to be not adequately suited for 3-D wavelet transformed (DWT) multispectral images. The fact that each parent has eight children in the 3-D structure considerably increases the list of insignificant sets (LIS) and the list of insignificant pixels (LIP) since the partitioning of any set produces eight subsets which will be processed similarly during the sorting pass. Thus, a significant portion from the overall bit-budget is wastedly spent to sort insignificant information. Through an investigation based on results analysis, we demonstrate that a straightforward 2-D SPIHT technique, when suitably adjusted to maintain the rate scalability and carried out in the 3-D DWT domain, overcomes this weakness. In addition, a new SPIHT-based scalable multispectral image compression algorithm is used in the initial iterations to exploit the redundancies within each group of two consecutive spectral bands. Numerical experiments on a number of multispectral images have shown that the proposed scheme provides significant improvements over related works.     

Three-way decision reduction in neighborhood systems

Rough set reduction has been used as an important preprocessing tool for pattern recognition, machine learning and data mining. As the classical Pawlak rough sets can just be used to evaluate categorical features, a neighborhood rough set model is introduced to deal with numerical data sets. Three-way decision theory proposed by Yao comes from Pawlak rough sets and probability rough sets for trading off different types of classification error in order to obtain a minimum cost ternary classifier. In this paper, we discuss reduction questions based on three-way decisions and neighborhood rough sets. First, the three-way decision reducts of positive region preservation, boundary region preservation and negative region preservation are introduced into the neighborhood rough set model. Second, three condition entropy measures are constructed based on three-way decision regions by considering variants of neighborhood classes. The monotonic principles of entropy measures are proved, from which we can obtain the heuristic reduction algorithms in neighborhood systems. Finally, the experimental results show that the three-way decision reduction approaches are effective feature selection techniques for addressing numerical data sets.     

Development and application of a three-dimensional artificial visual system

A three-dimensional artificial visual system has been developed to aid in the analysis of 3-D fluorescence images of smooth muscle cells. The system consists of three sets of 3-D spatial filters that decompose the image to enable a simple recombination algorithm to locate the discrete bodies of protein concentration in a cell, classify the concentration bodies as globular or oval, and determine the 3-D orientation of the oval bodies. A graphic model of the protein concentration is created from the data provided by the artificial visual system. Patterns of organization in the distribution of the protein bodies are investigated using an interactive graphics system.     

The accuracy of the computation of optical flow and of the recoveryof motion parameters

The accuracy and the dependence on parameters of a general scheme for the analysis of time-varying image sequences are discussed. The approach is able to produce vector fields from which it is possible to recover 3-D motion parameters such as time-to-collision and angular velocity. The numerical stability of the computed optical flow and the dependence of the recovery of 3-D motion parameters on spatial and temporal filtering is investigated. By considering optical flows computed on subsampled images or along single scanlines, it is also possible to recover 3-D motion parameters from reduced optical flows. An adequate estimate of time-to-collision can be obtained from sequences of images with spatial resolution reduced to 128×128 pixels or from sequences of single scanlines passing near the focus of expansion. The use of Kalman filtering increases the accuracy and the robustness of the estimation of motion parameters. The proposed approach seems to be able to provide not only a theoretical background but also practical tools that are adequate for the analysis of time-varying image sequences     

Comprehensive Cardiovascular Image Analysis Using MR and CT at Siemens Corporate Research

At Siemens Corporate Research we have created a set of tools for the analysis of MR and CT cardiovascular images in the applications Argus, Vessel View, and Proteus. Argus is designed to assess cardiovascular function by reporting measures of morphology and tissue health using a 2-D approach. Vessel View, a 3-D application, is capable of quantifying vascular integrity and provides tools for segmenting vessels. Lastly, Proteus has functionality for registering 3-D cardiac data sets (e.g., MR and CT). Taken together, these applications allow for a comprehensive analysis of MR and CT cardiovascular studies. Throughout this paper we will illustrate the capabilities of our tools via their application to an actual clinical case. Our contribution lies in combining several computer vision technologies and applying them to practical, real world problems.     

Learning fuzzy rules from fuzzy samples based on rough set technique

Although the traditional rough set theory has been a powerful mathematical tool for modeling incompleteness and vagueness, its performance in dealing with initial fuzzy data is usually poor. This paper makes an attempt to improve its performance by extending the traditional rough set approach to the fuzzy environment. The extension is twofold. One is knowledge representation and the other is knowledge reduction. First, we provide new definitions of fuzzy lower and upper approximations by considering the similarity between the two objects. Second, we extend a number of underlying concepts of knowledge reduction (such as the reduct and core) to the fuzzy environment and use these extensions to propose a heuristic algorithm to learn fuzzy rules from initial fuzzy data. Finally, we provide some numerical experiments to demonstrate the feasibility of the proposed algorithm. One of the main contributions of this paper is that the fundamental relationship between the reducts and core of rough sets is still pertinent after the proposed extension.     

3-D mapping of natural environments with trees by means of mobile perception

In this paper, a method of generating a three-dimensional (3-D) geometric model for large-scale natural environments with trees is presented. The environment mapping method, which uses range images as measurement data, consists of three main phases. First, geometric feature objects are extracted from each of the range images. Second, the relative coordinate transformations (i.e., registrations) between the sensor viewpoint locations, where the range data are measured, are computed. Third, an integrated map is formed by transforming the submap data into a common frame of reference. Tree trunks visible in the range images are modeled with cylinder segments and utilized as reference features for registration computation. The final integrated 3-D model consists of the cylinder segments representing the visible sections of the tree trunks, as well as of the ground elevation data. The constructed environment map can be utilized as, for example, a virtual task environment for outdoor robotic machines such as new-generation forest working machines or service robots.     

Web-based three-dimensional geo-referenced visualization

This paper addresses several approaches to implementing web-based, three-dimensional (3-D), geo-referenced visualization. The discussion focuses on the relationship between multi-dimensional data sets and applications, as well as the thick/thin client and heavy/light server structure. Two models of data sets are addressed in this paper. One is the use of traditional 3-D data format such as 3-D Studio Max, Open Inventor 2.0, Vis5D and OBJ. The other is modelled by a web-based language such as VRML. Also, traditional languages such as C and C++, as well as web-based programming tools such as Java, Java3D and ActiveX, can be used for developing applications. The strengths and weaknesses of each approach are elaborated. Four practical solutions for using VRML and Java, Java and Java3D, VRML and ActiveX and Java wrapper classes (Java and C/C++), to develop applications are presented for web-based, real-time interactive and explorative visualization.     

Reconstructing Open Surfaces from Image Data

In this paper a method for fitting open surfaces to data obtained from images is presented using a level set representation of the surface. This is done by tracking a curve, representing the boundary, on the implicitly defined surface. This curve is given as the intersection of the level set describing the surface and an auxiliary level set. These two level sets are propagated using the same motion vector field. Special care has to be taken in order for the surfaces not to intersect at other places than at the desired boundary. Methods for accomplishing this are presented and a fast scheme for finding initial values is proposed. This method gives a piecewise linear approximation of the initial surface boundary using a partition of the convex hull of the recovered 3D data. With the approach described in this paper, open surfaces can be fitted to e.g. point clouds obtained using structure from motion techniques. This paper solves an important practical problem since in many cases the surfaces in the scene are open or can only be viewed from certain directions. Experiments on several data sets support the method.     

Iterative point matching for registration of free-form curves and surfaces

A heuristic method has been developed for registering two sets of 3-D curves obtained by using an edge-based stereo system, or two dense 3-D maps obtained by using a correlation-based stereo system. Geometric matching in general is a difficult unsolved problem in computer vision. Fortunately, in many practical applications, some a priori knowledge exists which considerably simplifies the problem. In visual navigation, for example, the motion between successive positions is usually approximately known. From this initial estimate, our algorithm computes observer motion with very good precision, which is required for environment modeling (e.g., building a Digital Elevation Map). Objects are represented by a set of 3-D points, which are considered as the samples of a surface. No constraint is imposed on the form of the objects. The proposed algorithm is based on iteratively matching points in one set to the closest points in the other. A statistical method based on the distance distribution is used to deal with outliers, occlusion, appearance and disappearance, which allows us to do subset-subset matching. A least-squares technique is used to estimate 3-D motion from the point correspondences, which reduces the average distance between points in the two sets. Both synthetic and real data have been used to test the algorithm, and the results show that it is efficient and robust, and yields an accurate motion estimate.     

Detecting outliers in categorical data through rough clustering

Outlier detection is an important data mining task with many contemporary applications. Clustering based methods for outlier detection try to identify the data objects that deviate from the normal data. However, the uncertainty regarding the cluster membership of an outlier object has to be handled appropriately during the clustering process. Additionally, carrying out the clustering process on data described using categorical attributes is challenging, due to the difficulty in defining requisite methods and measures dealing with such data. Addressing these issues, a novel algorithm for clustering categorical data aimed at outlier detection is proposed here by modifying the standard \(k\)-modes algorithm. The uncertainty regarding the clustering process is addressed by considering a soft computing approach based on rough sets. Accordingly, the modified clustering algorithm incorporates the lower and upper approximation properties of rough sets. The efficacy of the proposed rough \(k\)-modes clustering algorithm for outlier detection is demonstrated using various benchmark categorical data sets.     

Template Assembly for Detailed Urban Reconstruction

We propose a new framework to reconstruct building details by automatically assembling 3D templates on coarse textured building models. In a preprocessing step, we generate an initial coarse model to approximate a point cloud computed using Structure from Motion and Multi View Stereo, and we model a set of 3D templates of facade details. Next, we optimize the initial coarse model to enforce consistency between geometry and appearance (texture images). Then, building details are reconstructed by assembling templates on the textured faces of the coarse model. The 3D templates are automatically chosen and located by our optimization‐based template assembly algorithm that balances image matching and structural regularity. In the results, we demonstrate how our framework can enrich the details of coarse models using various data sets.     

Moment feature-based three-dimensional tracking using two high-speed vision systems

When considering real-world applications of robot control with visual servoing, both three-dimensional (3-D) information and a high feedback rate are required. We have developed a 3-D target-tracking system with a 1-ms feedback rate using two high-speed vision systems called Column Parallel Vision (CPV) systems. To obtain 3-D information, such as position, orientation and shape parameters of the target object, a feature-based algorithm has been introduced using moment feature values extracted from vision systems for a spheroidal object model. Also, we propose a new 3-D self-windowing method to extract the target in 3-D space using epipolar geometry, which is an extension of the conventional self-windowing method in 2-D images.     

Identifying Noncooperative Subjects at a Distance Using Face Images and Inferred Three-Dimensional Face Models

We present an approach to identify noncooperative individuals at a distance from a sequence of images, using 3-D face models. Most biometric features (such as fingerprints, hand shape, iris, or retinal scans) require cooperative subjects in close proximity to the biometric system. We process images acquired with an ultrahigh-resolution video camera, infer the location of the subjects' head, use this information to crop the region of interest, build a 3-D face model, and use this 3-D model to perform biometric identification. To build the 3-D model, we use an image sequence, as natural head and body motion provides enough viewpoint variation to perform stereomotion for 3-D face reconstruction. We have conducted experiments on a 2-D and 3-D databases collected in our laboratory. First, we found that metric 3-D face models can be used for recognition by using simple scaling method even though there is no exact scale in the 3-D reconstruction. Second, experiments using a commercial 3-D matching engine suggest the feasibility of the proposed approach for recognition against 3-D galleries at a distance (3, 6, and 9 m). Moreover, we show initial 3-D face modeling results on various factors including head motion, outdoor lighting conditions, and glasses. The evaluation results suggest that video data alone, at a distance of 3 to 9 meters, can provide a 3-D face shape that supports successful face recognition. The performance of 3-D–3-D recognition with the currently generated models does not quite match that of 2-D–2-D. We attribute this to the quality of the inferred models, and this suggests a clear path for future research.      

Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera

Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In related work, a dense 3-D model is generated by using a stereo method. However, such approaches cannot use several hundreds images together for dense depth estimation because it is difficult to accurately calibrate a large number of cameras. In this paper, we propose a dense 3-D reconstruction method that first estimates extrinsic camera parameters of a hand-held video camera, and then reconstructs a dense 3-D model of a scene. In the first process, extrinsic camera parameters are estimated by tracking a small number of predefined markers of known 3-D positions and natural features automatically. Then, several hundreds dense depth maps obtained by multi-baseline stereo are combined together in a voxel space.So, we can acquire a dense 3-D model of the outdoor scene accurately by using several hundreds input images captured by a hand-held video camera.     

Three-dimensional reconstruction of the left ventricle from two angiographic views: an evidence combination approach

Clinical interventional hemodynamic studies quantify the ventricular function from two-dimensional (2-D) X-ray projection images without having enough information of the actual three-dimensional (3-D) shape of this cardiac cavity. This paper reports a left ventricle 3-D reconstruction method from two orthogonal angiographic projections. This investigation is motivated by the lack of information about the actual 3-D shape of the cardiac cavity. The proposed algorithm works in 3-D space and considers the oblique projection geometry associated with the biplane image acquisition equipment. The reconstruction process starts by performing an approximate reconstruction based on the Cylindrical Closure Operation and the Dempster-Shafer theory. This approximate reconstruction is appropriately deformed in order to match the given projections. The deformation procedure is carried out by an iterative process that, by means of the Dempster-Shafer and the fuzzy integral theory, combines the information provided by the projection error and the connectivity between voxels. The performance of the proposed reconstruction method is evaluated by considering first the reconstruction of two 3-D binary databases from two orthogonal synthetized projections, obtaining errors as low as 6.48%. The method is then tested on real data, where two orthogonal preprocessed angiographic images are used for reconstruction. The performance of the technique, in this case, is assessed by means of the projection error, whose average for both views is 7.5%. The reconstruction method is also tested by performing the 3-D reconstruction of a ventriculographic sequence throughout an entire cardiac cycle.