Estimation of contour motion and deformation for nonrigid object tracking

We present an algorithm for nonrigid contour tracking in heavily cluttered background scenes. Based on the properties of nonrigid contour movements, a sequential framework for estimating contour motion and deformation is proposed. We solve the nonrigid contour tracking problem by decomposing it into three subproblems: motion estimation, deformation estimation, and shape regulation. First, we employ a particle filter to estimate the global motion parameters of the affine transform between successive frames. Then we generate a probabilistic deformation map to deform the contour. To improve robustness, multiple cues are used for deformation probability estimation. Finally, we use a shape prior model to constrain the deformed contour. This enables us to retrieve the occluded parts of the contours and accurately track them while allowing shape changes specific to the given object types. Our experiments show that the proposed algorithm significantly improves the tracker performance.     

Skeleton body pose tracking from efficient three-dimensional motion estimation and volumetric reconstruction

We address the problem of body pose tracking in a scenario of multiple camera setup with the aim of recovering body motion robustly and accurately. The tracking is performed on three-dimensional (3D) space using 3D data, including colored volume and 3D optical flow, which are reconstructed at each time step. We introduce strategies to compute multiple camera-based 3D optical flow and have attained efficient and robust 3D motion estimation. Body pose estimation starts with a prediction using 3D optical flow and then is changed to a lower-dimensional global optimization problem. Our method utilizes a voxel subject-specific body model, exploits multiple 3D image cues, and incorporates physical constraints into a stochastic particle-based search initialized from the deterministic prediction and stochastic sampling. It leads to a robust 3D pose tracker. Experiments on publicly available sequences show the robustness and accuracy of our approach.     

Robust structure from motion estimation using inertial data.

The utility of using inertial data for the structure-from-motion (SfM) problem is addressed. We show how inertial data can be used for improved noise resistance, reduction of inherent ambiguities, and handling of mixed-domain sequences. We also show that the number of feature points needed for accurate and robust SfM estimation can be significantly reduced when inertial data are employed. Cramér-Rao lower bounds are computed to quantify the improvements in estimating motion parameters. A robust extended-Kalman-filter-based SfM algorithm using inertial data is then developed to fully exploit the inertial information. This algorithm has been tested by using synthetic and real image sequences, and the results show the efficacy of using inertial data for the SfM problem.     

Curvature-based nonrigid motion analysis from three-dimensional correspondences

We describe a curvature-based approach for estimating nonrigid motion of moving surfaces. We deal with conformal motion, which can be characterized by stretching of the surface. At each point, this stretching is equal in all directions but different for different points. The stretching function can be defined as additional (with global translation and rotation) motion parameter. We present a new algorithm for local stretching recovery from Gaussian curvature, based on polynomial (linear and quadratic) approximations of the stretching function. It requires point correspondences between time frames but not the complete knowledge of nonrigid transformation. Experiments on simulated and real data are performed to illustrate performance and accuracy of derived algorithms. Noise sensitivity of the algorithm is also evaluated using Gaussian noise on simulated data.©1993 John Wiley & Sons Inc     

Image estimation from scattered field data

The purpose of this paper is to reappraise the linearizing methods frequently used to solve inverse scattering problems. We describe inversion algorithms based on the Born and the Rytov approximations and the nature of the distortions obtained in the reconstructions when using them. We present extensions of these methods, namely, the distorted-wave Born and the distorted-wave Rytov approximations, which incorporate prior knowledge about part of the scattering structure. A method for inverting scattered field data using these distorted-wave approximations is described, which retains the computational simplicity of the Born and the Rytov techniques. Some examples of their use with simulated and real data are given. A further extension of our distorted-wave formalism, which leads to improvements of the reconstructed image, is suggested. This entails a spectral estimation procedure also based on the incorporation of prior knowledge about the scatterer. This spectral estimation procedure can be useful for interpolation of scattered field data as well as resolution enhancement.     

Phase-based ultrasonic deformation estimation

Deformation estimation is the foundation of emerging techniques for imaging the mechanical properties of soft tissues. We present theoretical analysis and experimental results from an investigation of phase-based ultrasonic deformation estimators. Numerous phase-based algorithm variants were tested quantitatively on simulated RF data from uniform scatterer fields, subject to a range of uniform strain deformations. Particular attention is paid to a new algorithm, weighted phase separation, the performance of which is demonstrated in application to in vivo freehand strain imaging. Good results support the theory that underlies the new algorithm, and more generally highlight the factors that should be considered in the design of high performance deformation estimators for practical applications. For context, note that this represents progress with an algorithm class that is suitable for real-time applications, yet has already been shown quantitatively to offer greater accuracy over a wide range of scanning conditions than adaptive companding methods based on correlation coefficient or sum of absolute differences.     

Phase object data obtained from defocused laser speckle displacement

An optical technique that is based on defocused digital speckle photography is proposed for the evaluation of phase objects. Phase objects are different kinds of transparent or semi-transparent media that allow light to be transmitted. A phase object inserted in a laser speckle field introduces speckle displacement, from which information about the object may be extracted. It is shown that one may use speckle displacements to determine both the phase gradients and the positions of phase objects. As an illustration the positions and focal lengths of two weak lenses have been derived from defocused laser speckle displacement.     

Regional cardiac motion and strain estimation in three-dimensional echocardiography: a validation study in thick-walled univentricular phantoms

Automatic quantification of regional left ventricular deformation in volumetric ultrasound data remains challenging. Many methods have been proposed to extract myocardial motion, including techniques using block matching, phase-based correlation, differential optical flow methods, and image registration. Our lab previously presented an approach based on elastic registration of subsequent volumes using a B-spline representation of the underlying transformation field. Encouraging results were obtained for the assessment of global left ventricular function, but a thorough validation on a regional level was still lacking. For this purpose, univentricular thick-walled cardiac phantoms were deformed in an experimental setup to locally assess strain accuracy against sonomicrometry as a reference method and to assess whether regions containing stiff inclusions could be detected. Our method showed good correlations against sonomicrometry: r(2) was 0.96, 0.92, and 0.84 for the radial (ε(RR)), longitudinal (ε(LL)), and circumferential (ε(CC)) strain, respectively. Absolute strain errors and strain drift were low for ε(LL) (absolute mean error: 2.42%, drift: -1.05%) and ε(CC) (error: 1.79%, drift: -1.33%) and slightly higher for ε(RR) (error: 3.37%, drift: 3.05%). The discriminative power of our methodology was adequate to resolve full transmural inclusions down to 17 mm in diameter, although the inclusion-to-surrounding tissue stiffness ratio was required to be at least 5:2 (absolute difference of 39.42 kPa). When the inclusion-to-surrounding tissue stiffness ratio was lowered to approximately 2:1 (absolute difference of 22.63 kPa), only larger inclusions down to 27 mm in diameter could still be identified. Radial strain was found not to be reliable in identifying dysfunctional regions.     

Fractional profilometry correlator for three-dimensional object recognition

A novel method for three-dimensional shape recognition is proposed. It combines the Fourier-transform-profilometry technique with a two-dimensional fractional correlation algorithm. A grating is projected onto the object surface, resulting in a distorted grating pattern that carries information about the height and the shape of the object. Three-dimensional objects are recognized by a fractional correlator by use of the transformed complex amplitude. An optoelectronic hybrid implementation is also suggested.     

Distortion-tolerant three-dimensional object recognition with digital holography

We present a technique to implement three-dimensional (3-D) object recognition based on phase-shift digital holography. We use a nonlinear composite correlation filter to achieve distortion tolerance. We take advantage of the properties of holograms to make the composite filter by using one single hologram. Experiments are presented to illustrate the recognition of a 3-D object in the presence of out-of-plane rotation and longitudinal shift along the z axis.     

Binocular three-dimensional motion detection: contributions of lateral motion and stereomotion

When an object moves along a trajectory in three-dimensional (3-D) space, there are potentially two orthogonal components that could be used to detect its motion: stereomotion resulting from the difference or disparity between the images in the right and left eyes, and lateral motion from the sum or average of the image motions in the right and left eyes. Using a suprathreshold search task for a target moving amid 3-D distractors, we found a range of 3-D trajectories for which increasing the stereomotion component did not improve detection. However, with larger stereomotion components, performance improved. The addition of random-motion noise to only the lateral motion component adversely affected the detection of both lateral motion and stereomotion. These data suggest that the visual system uses the average of the monocular image motions for the detection of a range of 3-D trajectories. In addition, a mechanism sensitive to the changing disparity may also be used but only for a very restricted range of 3-D motions.     

Hazard rate estimation from incomplete and unclean warranty data

Warranty data are a rich source of information for feedback on product reliability. However, two-dimensional automobile warranties that include both time and mileage limits, pose two interesting and challenging problems in reliability studies. First, warranty data are restricted only to the reported failures within warranty coverage and such incompleteness can lead to inaccurate estimates of field failure rate or hazard rate. Second, factors such as inexact time/mileage data and vague reported failures in a warranty claim make warranty data unclean that can suppress inherent failure pattern. In this paper we discuss two parameter estimation methods that address the incompleteness issue. We use a simulation-based experiment to study these estimation methods when departure from normality and varying amount of truncation exists. Using a life cycle model of the vehicle, we also highlight and explore issues that lead to warranty data not being very clean. We then propose a five-step methodology to arrive at meaningful component level empirical hazard plots from incomplete and unclean warranty data.     

High-accuracy three-dimensional shape acquisition of a large-scale object from multiple uncalibrated camera views

Acquiring a high-accuracy three-dimensional (3D) shape of a large-scale object from multiple uncalibrated camera views remains a big challenge, since a considerable number of images is required to cover the entire surface; the use of multiple images could, however, result in accumulative errors from each processed image. Here error propagation rules in the 3D reconstruction process have been deduced on the basis of the traditional dual-view reconstruction method. We propose a method that can control the accumulative errors by reducing the times of coordinate transformation with common-view-based dual-view reconstruction. This method involves constructing an image network composed of many image groups, each of which contains a common view. A baseline threshold method is introduced to construct a high-quality image network, and the sums or reprojection residual of all the common points is proposed to assess the validity of the solutions of the orientation. Experiments carried out with both synthetic and real images demonstrate that the proposed method can handle the accumulative error problem with robust and highly accurate results.     

Image motion estimation by clustering

Image motion is estimated by matching feature “interest” points in different frames of video image sequences. The matching is based on local similarity of the displacement vectors. Clustering in the displacement vector space is used to determine the set of plausible match vectors. Subsequently, a similarity-based algorithm performs the actual matching. The feature points are computed using a multiple-filter image decomposition operator. The algorithm has been tested on synthetic as well as real video images. The novelty of the approach is that it handles multiple motions and performs motion segmentation.     

Real-time three-dimensional object recognition with multiple perspectives imaging

A novel system for recognizing three-dimensional (3D) objects by use of multiple perspectives imaging is proposed. A 3D object under incoherent illumination is projected into an array of two-dimensional (2D) elemental images by use of a microlens array. Each elemental 2D image corresponds to a different perspective of the 3D object. Multiple perspectives imaging based on integral photography has been used for 3D display. In this way, the whole set of 2D elemental images records 3D information about the input object. After an optical incoherent-to-coherent conversion, an optical processor is employed to perform the correlation between the input and the reference 3D objects. Use of micro-optics allows us to process the 3D information in real time and with a compact optical system. To the best of our knowledge this 3D processor is the first to apply the principle of integral photography to 3D image recognition. We present experimental results obtained with both a digital and an optical implementation of the system. We also show that the system can recognize a slightly out-of-plane rotated 3D object.     

Compression of digital holograms for three-dimensional object reconstruction and recognition

We present the results of applying lossless and lossy data compression to a three-dimensional object reconstruction and recognition technique based on phase-shift digital holography. We find that the best lossless (Lempel-Ziv, Lempel-Ziv-Welch, Huffman, Burrows-Wheeler) compression rates can be expected when the digital hologram is stored in an intermediate coding of separate data streams for real and imaginary components. The lossy techniques are based on subsampling, quantization, and discrete Fourier transformation. For various degrees of speckle reduction, we quantify the number of Fourier coefficients that can be removed from the hologram domain, and the lowest level of quantization achievable, without incurring significant loss in correlation performance or significant error in the reconstructed object domain.     

Flyer plate motion and its deformation during flight

Velocity and peripheral deformation of a flyer plate, driven by the detonation of contact explosive, have been studied during its flight both theoretically and experimentally. In theoretical formulations, density or velocity of the detonation products behind the flyer plate have been assumed to vary linearly; the results thus obtained compare well with the experimental results. Several experiments were performed in which the guard ring size and air gap between the explosive and the plate were varied. The arrangement which gave negligible peripheral deformation is presented.     

A scale model for fitting object shapes from fixed location data

In this paper we present a scale model for fitting an ideal shape to an object. The measurements of the object are taken corresponding to a fixed coordinate system at a set of well-defined locations on the surface of the object. We propose an algorithm to estimate the model parameters and hypothesis tests to make statistical inferences about several possible special cases of the general model. The model is tested with an example that analyzes data on the feet of people in Hong Kong.     

Algebraic methods in 3-d motion estimation from two-view point correspondences

We consider the problem of determining motion (3-D rotation and translation) of rigid objects from their images taken at two time instants. We assume that the locations of the perspective projection on the image plane of n points from the surface of the rigid body are known at two time instants. For n = 5, we show that there are at most ten possible motion values (in rotation and translation) and give many examples. For n ≥ 6, we show that the solution is generally unique. We derive a variety of necessary and sufficient conditions a solution must satisfy, show their equivalence, and use algebraic geometry to derive the bound on the number of solutions. A homotopy method is then used to compute all the solutions. Several examples are worked out and our computational experience is summarized.     

根据帧间运动特性的变尺寸宏块运动估计

提出了一种新的用于视频编码的变尺寸宏块运动估计方法。该方法通过粗尺度 16×16 的当前块和邻域块运动矢量之间的差值 S 判断场景运动的复杂度,再根据差值 S 和当前矢量,决定宏块尺寸。场景运动平稳的部分采用 16×16 的宏块,复杂的部分采用 8×8 或 4×4 的宏块。实验表明,低码率条件下,采用普通运动补偿算法,该变尺寸宏块运动估计方法的预测帧的平均 PSNR 值比常规的固定尺寸块运动估计约提高 1.1-2.7 dB,并显著降低了方块效应。