Observability of 3D Motion

This paper examines the inherent difficulties in observing 3D rigid motion from image sequences. It does so without considering a particular estimator. Instead, it presents a statistical analysis of all the possible computational models which can be used for estimating 3D motion from an image sequence. These computational models are classified according to the mathematical constraints that they employ and the characteristics of the imaging sensor (restricted field of view and full field of view). Regarding the mathematical constraints, there exist two principles relating a sequence of images taken by a moving camera. One is the epipolar constraint, applied to motion fields, and the other the positive depth constraint, applied to normal flow fields. 3D motion estimation amounts to optimizing these constraints over the image. A statistical modeling of these constraints leads to functions which are studied with regard to their topographic structure, specifically as regards the errors in the 3D motion parameters at the places representing the minima of the functions. For conventional video cameras possessing a restricted field of view, the analysis shows that for algorithms in both classes which estimate all motion parameters simultaneously, the obtained solution has an error such that the projections of the translational and rotational errors on the image plane are perpendicular to each other. Furthermore, the estimated projection of the translation on the image lies on a line through the origin and the projection of the real translation. The situation is different for a camera with a full (360 degree) field of view (achieved by a panoramic sensor or by a system of conventional cameras). In this case, at the locations of the minima of the above two functions, either the translational or the rotational error becomes zero, while in the case of a restricted field of view both errors are non-zero. Although some ambiguities still remain in the full field of view case, the implication is that visual navigation tasks, such as visual servoing, involving 3D motion estimation are easier to solve by employing panoramic vision. Also, the analysis makes it possible to compare properties of algorithms that first estimate the translation and on the basis of the translational result estimate the rotation, algorithms that do the opposite, and algorithms that estimate all motion parameters simultaneously, thus providing a sound framework for the observability of 3D motion. Finally, the introduced framework points to new avenues for studying the stability of image-based servoing schemes.     

Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene

This paper is about multi-view modeling of a rigid scene. We merge the traditional approaches of reconstructing image-extractable features and of modeling via user-provided geometry. We use features to obtain a first guess for structure and motion, fit geometric primitives, correct the structure so that reconstructed features lie exactly on geometric primitives and optimize both structure and motion in a bundle adjustment manner while enforcing the underlying constraints. We specialize this general scheme to the point features and the plane geometric primitives. The underlying geometric relationships are described by multi-coplanarity constraints. We propose a minimal parameterization of the structure enforcing these constraints and use it to devise the corresponding maximum likelihood estimator. The recovered primitives are then textured from the input images. The result is an accurate and photorealistic model.Experimental results using simulated data confirm that the accuracy of the model using the constrained methods is of clearly superior quality compared to that of traditional methods and that our approach performs better than existing ones, for various scene configurations. In addition, we observe that the method still performs better in a number of configurations when the observed surfaces are not exactly planar. We also validate our method using real images.     

基于直线光流场的三维运动和结构重建

利用直线间运动对应关系,将像素点光流的概念和定义方法应用于直线,提出了直线光流的概念,建立了求解空间物体运动参数的线性方程组,利用三幅图像21条直线的光流场,可以求得物体运动的12个参数以及空间直线坐标．但是在实际应用当中,要找出这21条直线的光流场是很困难的,因此该文提出了运用解非线性方程组的方法,只需要6条直线的光流．就可以分步求出物体的12个运动参数,并根据求得的12个运动参数和一致的图像坐标系中的直线坐标,求得空间直线的坐标,从而实现了三维场景的重建．     

Uncalibrated Motion Capture Exploiting Articulated Structure Constraints

We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.     

双视三维重建的高精度运动参数估计方法

从两幅透视图像恢复被摄目标的三维结构是计算机视觉最基本的任务之一,其中,运动估计算法的性能决定了最终的三维重建精度。首先讨论了双视成像的基本数学模型,并介绍了几种现有运动参数估计方法的基本原理和不足。随后,基于投影误差最小判决函数,提出了用于双像运动估计的改进非线性迭代优化方法。数值仿真结果表明,在大平移小旋转角及小平移大旋转角2种运动条件下,采用文中提出的方法,运动估计精度均有所提高。此外,根据运动参数的估计值对真实目标进行三维重建实验,结果表明尺度重建误差小于2%且角度误差在3°以内。     

Correspondence-free Structure from Motion

We present a novel approach for the estimation of 3D-motion directly from two images using the Radon transform. The feasibility of any camera motion is computed by integrating over all feature pairs that satisfy the epipolar constraint. This integration is equivalent to taking the inner product of a similarity function on feature pairs with a Dirac function embedding the epipolar constraint. The maxima in this five dimensional motion space will correspond to compatible rigid motions. The main novelty is in the realization that the Radon transform is a filtering operator: If we assume that the similarity and Dirac functions are defined on spheres and the epipolar constraint is a group action of rotations on spheres, then the Radon transform is a correlation integral. We propose a new algorithm to compute this integral from the spherical Fourier transform of the similarity and Dirac functions. Generating the similarity function now becomes a preprocessing step which reduces the complexity of the Radon computation by a factor equal to the number of feature pairs processed. The strength of the algorithm is in avoiding a commitment to correspondences, thus being robust to erroneous feature detection, outliers, and multiple motions. The authors are grateful for support through the following grants: NSF-IIS-0083209, NSF-IIS-0121293, NSF-EIA-0324977, NSF-CNS-0423891, NSF-IIS-0431070, and ARO/MURI DAAD19-02-1-0383. The author is grateful for the generous support of the ARO MURI program (DAAD-19-02-1-0383) while at U. C. Berkeley.     

一种基于点对的深度和运动估计方法

针对基于模型的姿态估计问题提出了一种新颖的深度和运动估计方法．估计过程分为两步．首先根据由3个模型点构成的3个点对估计旋转矩阵．然后,根据3个点对中的一个计算平移和深度．本文还提出了一种简单而有效的优化算法用于提高估计结果的精度．该估计方法的特点是独立地估计旋转矩阵,而且只需少量的模型点就能获得好的估计结果．模拟试验和实际试验结果表明了该方法的有效性．     

基于运动估计的交互式医学图像序列分割

针对医学图像难以自动分割,而医学图像序列采用手工分割时工作量巨大、效率低的问题,提出了一种新的交互式图像序列分割方法.在计算机的辅助下,用手工精确地描画出第一幅图像中对象的边界轮廓.后续图像的分割曲线用运动估计的方法自动得到.每完成一幅图像的分割用户都可以检查分割效果,如果不满意则可用手工修正.这个过程重复进行,直到整个图像序列分割完毕.实验结果表明,该方法能精确、快速地实现医学图像序列的分割.     

摄像机沿光轴运动下的场景三维重建

讨论了采用针孔摄像机进行摄像机沿光轴运动下的场景三维重建的方法．基于摄像机轴向运动的特点和性质,利用该方法找到图像间的缩放因子,进而解决了轴向运动下的特征匹配;采用Sturm的摄像机自标定方法得到摄像机的内外参数;从而实现了摄像机沿光轴运动下的场景三维重建．     

基于透视和正投影图像的3D运动与结构恢复

论文首次研究了由一幅正投影图像和一幅透视投影图像的特征点对应来进行刚体3D运动重建与结构恢复的问题,给出了有效的线性算法。以往的由运动恢复结构的工作主要集中于一组透视图像或一组正投影(通常为仿射)图像,文中采用了透视模型和正投影模型的组合。数据模拟实验结果显示该方法是比较有效和稳定的。     

深度学习单目深度估计研究进展

单目深度估计是从单幅图像中获取场景深度信息的重要技术,在智能汽车和机器人定位等领域应用广泛,具有重要的研究价值。随着深度学习技术的发展,涌现出许多基于深度学习的单目深度估计研究,单目深度估计性能也取得了很大进展。本文按照单目深度估计模型采用的训练数据的类型,从3个方面综述了近年来基于深度学习的单目深度估计方法:基于单图像训练的模型、基于多图像训练的模型和基于辅助信息优化训练的单目深度估计模型。同时,本文在综述了单目深度估计研究常用数据集和性能指标基础上,对经典的单目深度估计模型进行了性能比较分析。以单幅图像作为训练数据的模型具有网络结构简单的特点,但泛化性能较差。采用多图像训练的深度估计网络有更强的泛化性,但网络的参数量大、网络收敛速度慢、训练耗时长。引入辅助信息的深度估计网络的深度估计精度得到了进一步提升,但辅助信息的引入会造成网络结构复杂、收敛速度慢等问题。单目深度估计研究还存在许多的难题和挑战。利用多图像输入中包含的潜在信息和特定领域的约束信息,来提高单目深度估计的性能,逐渐成为了单目深度估计研究的趋势。     

基于多视角视频的运动重建

为了更逼真地从视频图像序列中实现三维人体骨架动画形式的提取,以便进一步地对人体运动进行分析与研究,提出了一种基于多视角视频的运动重建的方法。该方法充分利用了标记点的信息,其核心步骤有标定摄像机,提取标记点,跟踪标记点和人体运动三维重建四个主要方面。其中,在跟踪标记点时,使用了基于多视觉的目标跟踪算法,该算法由结合了扩展卡尔曼滤波预测与标记点轨迹平滑性约束所构成的双目立体视觉跟踪与多目视觉数据融合两个方面。实验结果证明了所提方法的有效性与可行性。     

基于平行性约束的摄像机标定与3D重构

引入了梯形的一个仿射不变量,并利用这个不变量,建立了梯形的相似不变量与摄像机内参数之间的约束关系.基于这个约束关系,利用摄像机内参数的知识或梯形相似不变量的知识,可以线性确定摄像机的内参数、运动参数和梯形的相似不变量.由于梯形是由一对平行线段唯一确定的,平行线段在许多场景中经常出现,因而该方法有很广泛的适用性.实验结果表明了该算法的有效性.该工作提供了一个基于平行性约束的框架,以往的基于平行四边形、平行六面体的方法都可以纳入到这个框架中.     

Stereo Without Epipolar Lines: A Maximum-Flow Formulation

This paper describes a new algorithm for solving the stereo correspondence problem with a global 2-d optimization by transforming it into a maximum-flow problem in a graph. This transformation effectively removes explicit use of epipolar geometry, thus allowing direct use of multiple cameras with arbitrary geometries. The maximum-flow, solved both efficiently and globally, yields a minimum-cut that corresponds to a disparity surface for the whole image at once. This global and efficient approach to stereo analysis allows the reconstruction to proceed in an arbitrary volume of space and provides a more accurate and coherent depth map than the traditional stereo algorithms. In particular, smoothness is applied uniformly instead of only along epipolar lines, while the global optimality of the depth surface is guaranteed. Results show improved depth estimation as well as better handling of depth discontinuities. While the worst case running time is O(n^1.5 d^1.5 log(nd)), the observed average running time is O(n^1.2 d^1.3) for an image size of n pixels and depth resolution d.     

单目视频中人体三维运动的迭代优化估计

提出并实现了一种从单目视频流中重建人体三维运动的方法．该方法通过交互定制得到个性化的人体骨架模型和视频序列每一帧中人体各关节点的二维坐标后,分别针对单帧和连续多帧进行优化并迭代求解,得到每一帧的比例因子的最优解;最后反求各关节点的三维坐标,重建人体三维运动序列．对包含复杂和快速多变的人体运动的视频进行的实验表明,该方法简单有效,适用于包括体育、影视等在内的实际视频源．     

基于运动恢复结构方法的植株三维重建

为了探究基于运动恢复结构(Structure From Motion, SFM)方法的植株三维重建模型的效果,为植物三维重建工作提供研究案例,本文以紫叶鸭跖草(setcreasea pallida)为研究对象,在搭建序列图像获取平台的基础上,选取35幅、75幅、105幅序列图像进行三维重建的对比分析;同时从植株表型参数方面,对植株三维重建模型进行精度评价。结果表明：75幅图像序列的重建效果最好;不同图像序列的模型计算的植株高度相对误差（Relative Error, RE）均小于2.5%,决定系数（coefficient of determination, R2）均大于0.998;不同图像序列的模型提取叶片长和叶片宽的RE均小于2.89%,R2均大于0.958。因此,序列图像的数量与重建模型的效果有关,但二者并非呈正相关关系;序列图像的数量对重建叶片的长与宽的误差影响较小;SFM方法应用于结构比较复杂的植株的三维重建可以取得较好的重建效果。     

Binocular matching constraints from motion

Structure from motion and structure from stereo are two vision cues for achieving 3D reconstruction. The two cues have complementary strengths; while 3D reconstruction is accurate but correspondence establishment is difficult in the stereo cue, the reverse is true in the motion cue. This paper addresses how to combine the two cues when a stereo pair of cameras are available to capture image data for 3D reconstruction. The work is distinct in that, in contrast with the previous ones, it is not to exploit the redundancy in the image data for boosting the reconstruction accuracy, but to make the two vision cues complementary, preserving their strengths and avoiding their weaknesses. A mechanism is introduced that allows dense motion correspondences in the two separate image streams be transferred to dense binocular correspondences across the image streams, so that 3D can be reconstructed from the latter and accurate reconstruction is possible even with short motions of the stereo rig. Both the stereo correspondences and the motion of the stereo rig are assumed to be unknown in this work. Experiments involving real image data are presented to indicate the feasibility and robustness of the approach.     

Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography

A new approach is described for reconstructing coronary arteries from two sequences of projection images. The estimation of motion is performed on three-dimensional line segments (or centrelines), and is based on a ‘predictionprojection-optimization’ loop. The method copes with time varying properties, deformations and superpositions of vessels. Experiments using simulated and real data have been carried out. and the results found to be robust over a full cycle of a human heart. Local and global kinetic features can then be derived to obtain a greater insight on the cardiac functional state