Dense Non-Rigid Motion Estimation in Sequences of Medical Images Using Differential Constraints

We describe a new method for computing the displacement vector field in time sequences of 2D or 3D images (4D data). The method is energy-minimizing on the space of correspondence functions; the energy is split into two terms, with one term matching differential singularities in the images, and the other constraining the regularity of the field. In order to reduce the computational time of the motion estimation, we use an adaptive image mesh, the resolution of which depends on the value of the gradient intensity. We solve numerically the minimization problem with the finite element method which gives a continuous approximation of the solution. We present experimental results on synthetic data and on medical images and we show how to use these results for analyzing cardiac deformations.     

Matching two perspective views

A computational approach to image matching is described. It uses multiple attributes associated with each image point to yield a generally overdetermined system of constraints, taking into account possible structural discontinuities and occlusions. In the algorithm implemented, intensity, edgeness, and cornerness attributes are used in conjunction with the constraints arising from intraregional smoothness, field continuity and discontinuity, and occlusions to compute dense displacement fields and occlusion maps along the pixel grids. The intensity, edgeness, and cornerness are invariant under rigid motion in the image plane. In order to cope with large disparities, a multiresolution multigrid structure is employed. Coarser level edgeness and cornerness measures are obtained by blurring the finer level measures. The algorithm has been tested on real-world scenes with depth discontinuities and occlusions. A special case of two-view matching is stereo matching, where the motion between two images is known. The algorithm can be easily specialized to perform stereo matching using the epipolar constraint     

Artificial neural networks for 3-D motion analysis. I. Rigid motion

Proposes an approach applying artificial neural net techniques to 3D rigid motion analysis based on sequential multiple time frames. The approach consists of two phases: (1) matching between every two consecutive frames and (2) estimating motion parameters based on the correspondences established. Phase 1 specifies the matching constraints to ensure a stable and coherent feature correspondence establishment between two sequential time frames and configures a 2D Hopfield neural net to enforce these constraints. Phase 2 constructs a 3-layer net to estimate parameters through supervised learning. The method performs motion analysis based on sequential multiple time frames. It represents an effective way to achieve optimal matching between two frames using neural net techniques. The energy function of the Hopfield net is designed to reflect the matching constraints and the minimization of this function leads to the optimal feature correspondence establishment. The approach introduces the learning concept to motion estimation. The structure of the net provides the flexibility in estimating motion parameters based on information from multiple frames.     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

Matching point features under small nonrigid motion

This paper describes a method for matching point features between images of objects that have undergone small nonrigid motion. Feature points are assumed to be available and, given a properly extracted set of feature points, a robust matching is established under the condition that the local nonrigid motion of each point is restricted to a circle of radius δ, where δ is not too large. This is in contrast to other techniques for point matching which assume either rigid motion or nonrigid motion of a known kind. The point matching problem is viewed in terms of weighted bipartite graph matching. In order to account for the possibility that the feature selector can be imprecise, we incorporate a greedy matching strategy with the weighted graph matching algorithm. Our algorithm is robust and insensitive to noise and missing features. The resulting matching can be used with image warping or other techniques for nonrigid motion analysis, image subtraction, etc. We present our experimental results on sequences of mammograms, images of a deformable clay object and satellite cloud images. In the first two cases we provide quantitative comparison with known ground truth.     

A validation of a posture matching approach for the determination of 3D cumulative back loads

The purpose of this project was to investigate the amount of error in calculating cumulative lumbar spine kinetics using a posture matching approach (3DMatch) compared to a 3D coordinate electromagnetic tracking approach (FASTRAK). Six subjects were required to perform five repeats each of two symmetrical and two asymmetrical lifts while being simultaneously recorded from 4 camera views at viewing angles of 0 degrees , 45 degrees , 60 degrees and 90 degrees to the sagittal plane while wearing eight FASTRAK sensors to define an 8 segment rigid link model (RLM) of the head, arms, and trunk. Four hundred and eighty lifts (6 subjects x20 lifts x4 camera views) were analyzed using the 3DMatch posture-matching program to calculate the following cumulative loads at the L4/L5 joint: compression, anterior shear, posterior shear, reaction shear and extension moment. The errors in cumulative load calculation were determined as the difference between the values calculated for the same lifts using a 3D RLM that used electromagnetic motion tracking sensors (FASTRAK) positioned at the segment center of masses as model inputs. No significant difference (p<0.05) in the relative error for any of the cumulative loading variables between the four camera views and the 3D RLM approach was found. Furthermore the relative errors for cumulative compression, joint anterior shear, reaction anterior shear and extension moment were all below 12%. These results suggest that posture matching by trained users can provide reasonable 3D data to calculate cumulative low back loads with a biomechanical model.     

Some experiments on estimating the 3-d motion parameters of a rigid body from two consecutive image frames

This paper describes the results of some experiments on estimating the 3-D motion parameters of a rigid body from two consecutive TV images, and discusses several factors which affect the accuracy of the results. These factors include the sizes of the motion parameters themselves, the accuracy of the raw data, and the number of point correspondences. In addition, we address two related topics: determining corner positions to subpixel accuracy and matching point patterns with different scales.     

Practical Structure and Motion from Stereo When Motion is Unconstrained

This paper describes a system which robustly estimates motion, and the 3D structure of a rigid environment, as a stereo vision platform moves through it. The system can cope with any camera motion, and any scene structure and is successful even in the presence of large jumps in camera position between the capture of successive image pairs, and when point matching is ambiguous. The system was developed to provide robust obstacle avoidance for a partially sighted person.The process described attempts to maximise use of the abundant information present in a stereo sequence. Key features include the use of multiple stereo match hypotheses, efficient motion computation from three images, and the use of this motion to ensure reliable matching, and to eliminate multiple stereo matches. Points are reconstructed in 3D space and tracked in a static coordinate frame with a Kalman Filter.This results in good 3D scene reconstructions. Structure which is impossible to match with certainty is absent, rather than being incorrectly reconstructed. As a result, the system is appropriate for obstacle detection. The results of processing some indoor and outdoor scenes, are given in the paper, and practical issues are highlighted throughout.     

Parallel distributed detection of feature trajectories in multiplediscontinuous motion image sequences

Concerns the 3D interpretation of image sequences showing multiple objects in motion. Each object exhibits smooth motion except at certain time instants when a motion discontinuity may occur. The objects are assumed to contain point features which are detected as the images are acquired. Estimating feature trajectories in the first two frames amounts to feature matching. As more images are acquired, existing trajectories are extended. Both initial detection and extension of trajectories are done by enforcing pertinent constraints from among the following: similarity of the image plane arrangement of neighboring features, smoothness of the 3D motion and smoothness of the image plane motion. The constraints are incorporated into energy functions which are minimized using 2D Hopfield networks. Wrong matches that result from convergence to local minima are eliminated using a 1D Hopfield-like network. Experimental results on several image sequences are shown.     

Efficient planar object tracking and parameter estimation usingcompactly represented cubic B-spline curves

In this paper, we consider the problem of matching 2D planar object curves from a database, and tracking moving object curves through an image sequence. The first part of the paper describes a curve data compression method using B-spline curve approximation. We present a new constrained active B-spline curve model based on the minimum mean square error (MMSE) criterion, and an iterative algorithm for selecting the “best” segment border points for each B-spline curve. The second part of the paper describes a method for simultaneous object tracking and affine parameter estimation using the approximate curves and profiles. We propose a novel B-spline point assignment algorithm which incorporates the significant corners for interpolating corresponding points on the two curves to be compared. A gradient-based algorithm is presented for simultaneously tracking object curves, and estimating the associated translation, rotation and scaling parameters. The performance of each proposed method is evaluated using still images and image sequences containing simple objects     

Multi-object reconstruction from dynamic scenes: An object-centered approach

In this paper, we present a new framework for three-dimensional (3D) reconstruction of multiple rigid objects from dynamic scenes. Conventional 3D reconstruction from multiple views is applicable to static scenes, in which the configuration of objects is fixed while the images are taken. In our framework, we aim to reconstruct the 3D models of multiple objects in a more general setting where the configuration of the objects varies among views. We solve this problem by object-centered decomposition of the dynamic scenes using unsupervised co-recognition approach. Unlike conventional motion segmentation algorithms that require small motion assumption between consecutive views, co-recognition method provides reliable accurate correspondences of a same object among unordered and wide-baseline views. In order to segment each object region, we benefit from the 3D sparse points obtained from the structure-from-motion. These points are reliable and serve as automatic seed points for a seeded-segmentation algorithm. Experiments on various real challenging image sequences demonstrate the effectiveness of our approach, especially in the presence of abrupt independent motions of objects.     

用二维点对应数据分割和估计多刚体运动

已知含有多个三维刚体的场景,在运动前后的二维点对应数据集合,其中可以包 含高斯噪声和出格点数据,发展了初始部分匹配的生成-生长技术并运用刚性约束.将上述 二维点对应数据集合.分割成多个分别对应于不同刚体运动的二维点对应数据子集,并能分 离出所有出格点数据.再利用单刚体运动估计算法就可估计出各个刚体运动参数.实验结果 表明了算法的有效性.     

基于模型的头部运动估计和面部图像合成

文中讨论一种基于模型的头部运动估计和面部图像合成方法。首先建立了一个基于人脸几何模型的可变形三维面部模型,此模型可根据不同人脸图像特征修正特定人脸模型。为了使特定人脸模型与特定人脸图像相匹配,需根据变形模型修正人脸模型。文中采用自动调整与人机交互相结合的方法实现特定人脸模型匹配。在调整完模型形状之后,应用３个方向的面部图像进行纹理映射生成不同视点方向的面部图像。应用合成面部图像与输入面部图像最佳匹     

Uncertainty propagation and the matching of junctions as featuregroupings

The interpretation of the 3D world from image sequences requires the identification and correspondences of key features in the scene. We describe a robust algorithm for matching groupings of features related to the objects in the scene. We consider the propagation of uncertainty from the feature detection stage through the grouping stage to provide a measure of uncertainty at the matching stage. We focus upon indoor scenes and match junctions, which are groupings of line segments that meet at a single point. A model of the uncertainty in junction detection is described, and the junction uncertainty under the epipolar constraint is determined. Junction correspondence is achieved through matching of each line segment associated with the junction. A match likelihood is then derived based upon the detection uncertainties and then combined with information on junction topology to create a similarity measure. A robust matching algorithm is proposed and used to match junctions between pairs of images. The presented experimental results on real images show that the matching algorithm produces sufficiently reliable results for applications such as structure from motion     

Motion analysis of articulated objects from monocular images

This paper presents a new method of motion analysis of articulated objects from feature point correspondences over monocular perspective images without imposing any constraints on motion. An articulated object is modeled as a kinematic chain consisting of joints and links, and its 3D joint positions are estimated within a scale factor using the connection relationship of two links over two or three images. Then, twists and exponential maps are employed to represent the motion of each link, including the general motion of the base link and the rotation of other links around their joints. Finally, constraints from image point correspondences, which are similar to that of the essential matrix in rigid motion, are developed to estimate the motion. In the algorithm, the characteristic of articulated motion, i.e., motion correlation among links, is applied to decrease the complexity of the problem and improve the robustness. A point pattern matching algorithm for articulated objects is also discussed in this paper. Simulations and experiments on real images show the correctness and efficiency of the algorithms.     

Motion visualization of human left ventricle with a time‐varying deformable model for cardiac diagnosis

We present a time‐varying deformable model to visualize and analyze the motion of the left ventricle from a time series of 3‐D images. The model is composed of a non‐rigid body that deforms around a reference shape obtained from the previous time step. At each time step, the position and orientation of the left ventricle are extracted from the feature points of images. This information gives the position and orientation of the coordinate system attached to the non‐rigid body. To compute a dense non‐rigid motion field over the entire endocardial wall of the left ventricle, we introduce a 3‐D blob finite element and Galerkin interpolants based on 3‐D Gaussian, and use a physically based finite element method and a modal analysis. Then, cinematic attributes are visualized in pseudo colors on the reconstructed surface in order to help medical doctors in their interpretation of the data. Using the presented model, we estimate clinically useful quantitative parameters such as regional wall motion and ejection fraction. Experimental results are shown in a time series of X‐ray angiographic images. Copyright ©2001 John Wiley & Sons, Ltd.     

Determining three-dimensional motion and structure from optical flow generated by several moving objects

A new approach for the interpretation of optical flow fields is presented. The flow field, which can be produced by a sensor moving through an environment with several independently moving, rigid objects, is allowed to be sparse, noisy, and partially incorrect. The approach is based on two main stages. In the first stage, the flow field is partitioned into connected segments of flow vectors, where each segment is consistent with a rigid motion of a roughly planar surface. In the second stage, segments are grouped under the hypothesis that they are induced by a single, rigidly moving object. Each hypothesis is tested by searching for three-dimensional (3-D) motion parameters which are compatible with all the segments in the corresponding group. Once the motion parameters are recovered, the relative environmental depth can be estimated as well. Experiments based on real and simulated data are presented.     

A Continuum Mechanical Approach to Geodesics in Shape Space

In this paper concepts from continuum mechanics are used to define geodesic paths in the space of shapes, where shapes are implicitly described as boundary contours of objects. The proposed shape metric is derived from a continuum mechanical notion of viscous dissipation. A geodesic path is defined as the family of shapes such that the total amount of viscous dissipation caused by an optimal material transport along the path is minimized. The approach can easily be generalized to shapes given as segment contours of multi-labeled images and to geodesic paths between partially occluded objects. The proposed computational framework for finding such a minimizer is based on the time discretization of a geodesic path as a sequence of pairwise matching problems, which is strictly invariant with respect to rigid body motions and ensures a 1–1 correspondence along the induced flow in shape space. When decreasing the time step size, the proposed model leads to the minimization of the actual geodesic length, where the Hessian of the pairwise matching energy reflects the chosen Riemannian metric on the underlying shape space. If the constraint of pairwise shape correspondence is replaced by the volume of the shape mismatch as a penalty functional, one obtains for decreasing time step size an optical flow term controlling the transport of the shape by the underlying motion field. The method is implemented via a level set representation of shapes, and a finite element approximation is employed as spatial discretization both for the pairwise matching deformations and for the level set representations. The numerical relaxation of the energy is performed via an efficient multi-scale procedure in space and time. Various examples for 2D and 3D shapes underline the effectiveness and robustness of the proposed approach.     

Estimating myocardial motion by 4D image warping

A method for spatio-temporally smooth and consistent estimation of cardiac motion from MR cine sequences is proposed. Myocardial motion is estimated within a four-dimensional (4D) registration framework, in which all three-dimensional (3D) images obtained at different cardiac phases are simultaneously registered. This facilitates spatio-temporally consistent estimation of motion as opposed to other registration-based algorithms which estimate the motion by sequentially registering one frame to another. To facilitate image matching, an attribute vector (AV) is constructed for each point in the image, and is intended to serve as a “morphological signature” of that point. The AV includes intensity, boundary, and geometric moment invariants (GMIs). Hierarchical registration of two image sequences is achieved by using the most distinctive points for initial registration of two sequences and gradually adding less-distinctive points to refine the registration. Experimental results on real data demonstrate good performance of the proposed method for cardiac image registration and motion estimation. The motion estimation is validated via comparisons with motion estimates obtained from MR images with myocardial tagging.     

3D Articulated Models and Multiview Tracking with Physical Forces

We present a method for automatically estimating the motion of an articulated object filmed by two or more fixed cameras. We focus our work on the case where the quality of the images is poor, and where only an approximation of a geometric model of the tracked object is available. Our technique uses physical forces applied to each rigid part of a kinematic 3D model of the object we are tracking. These forces guide the minimization of the differences between the pose of the 3D model and the pose of the real object in the video images. We use a fast recursive algorithm to solve the dynamical equations of motion of any 3D articulated model. We explain the key parts of our algorithms: how relevant information is extracted from the images, how the forces are created, and how the dynamical equations of motion are solved. A study of what kind of information should be extracted in the images and of when our algorithms fail is also presented. Finally we present some results about the tracking of a person. We also show the application of our method to the tracking of a hand in sequences of images, showing that the kind of information to extract from the images depends on their quality and of the configuration of the cameras.