Structure from Motion: Beyond the Epipolar Constraint

The classic approach to structure from motion entails a clear separation between motion estimation and structure estimation and between two-dimensional (2D) and three-dimensional (3D) information. For the recovery of the rigid transformation between different views only 2D image measurements are used. To have available enough information, most existing techniques are based on the intermediate computation of optical flow which, however, poses a problem at the locations of depth discontinuities. If we knew where depth discontinuities were, we could (using a multitude of approaches based on smoothness constraints) accurately estimate flow values for image patches corresponding to smooth scene patches; but to know the discontinuities requires solving the structure from motion problem first. This paper introduces a novel approach to structure from motion which addresses the processes of smoothing, 3D motion and structure estimation in a synergistic manner. It provides an algorithm for estimating the transformation between two views obtained by either a calibrated or uncalibrated camera. The results of the estimation are then utilized to perform a reconstruction of the scene from a short sequence of images.The technique is based on constraints on image derivatives which involve the 3D motion and shape of the scene, leading to a geometric and statistical estimation problem. The interaction between 3D motion and shape allows us to estimate the 3D motion while at the same time segmenting the scene. If we use a wrong 3D motion estimate to compute depth, we obtain a distorted version of the depth function. The distortion, however, is such that the worse the motion estimate, the more likely we are to obtain depth estimates that vary locally more than the correct ones. Since local variability of depth is due either to the existence of a discontinuity or to a wrong 3D motion estimate, being able to differentiate between these two cases provides the correct motion, which yields the least varying estimated depth as well as the image locations of scene discontinuities. We analyze the new constraints, show their relationship to the minimization of the epipolar constraint, and present experimental results using real image sequences that indicate the robustness of the method.     

Understanding the Behavior of SFM Algorithms: A Geometric Approach

We put forth in this paper a geometrically motivated motion error analysis which is capable of supporting investigation of global effect such as inherent ambiguities. This is in contrast with the usual statistical kinds of motion error analyses which can only deal with local effect such as noise perturbations, and where much of the results regarding global ambiguities are empirical in nature. The error expression that we derive allows us to predict the exact conditions likely to cause ambiguities and how these ambiguities vary with motion types such as lateral or forward motion. Given the erroneous 3-D motion estimates caused by the inherent ambiguities, it is also important to study the behavior of the resultant distortion in depth recovered under different motion-scene configurations. Such an investigation may alert us to the occurrence of ambiguities under different conditions and be more careful in picking the solution. Our formulation, though geometrically motivated, was also put to use in modeling the effect of noise and in revealing the strong influence of feature distribution. Experiments on both synthetic and real image sequences were conducted to verify the various theoretical predictions.     

基于纹理复杂度和运动方向的3D视频水印算法

3D视频的编码方法在HEVC（high efficiency video coding）的基础上有了较大的改动.针对这些改变,提出了一种基于纹理复杂度和运动方向的3D视频水印算法.首先,采用灰度共生矩阵和梯度矩阵分别对I帧中CTU（coding tree unit）和8 CU×8 CU（coding unit）纹理复杂度进行统计分析,将分析结果作为阈值选出P帧和B帧中纹理复杂度较高的8 CU×8 CU;其次,选择帧间预测模式为对称分割的块作为嵌入块,结合同位纹理块的运动矢量分量大小来确定嵌入的水印;最后,通过调制当前块运动矢量的搜索范围或修改区域最优运动矢量的大小来嵌入水印.本算法对不同视频序列平均嵌入容量达到613字节/帧,在不同QPs（quantization parameters）（25、30、35、40）值的重编码攻击下的平均误码率为13.31%,PSNR值平均下降仅为0.007 5 dB,对合成视点质量几乎没有影响,同时鲁棒性好.     

体育训练三维人体运动模拟与视频分析系统

在运动训练中引入数字图形图像技术,研制面向体育训练的三维人体运动仿真与视频分析系统,为我国体育健儿备战2008年奥运会提供强大的科技保证．首先介绍了研制系统的背景与意义,然后从视频分析与三维运动模拟两个方面概述了系统所具有的关键功能,最后较为详细地阐述了为实现这些功能所需要解决的关键问题及解决方法,包括视频运动人体提取与跟踪、三维人体运动模拟与仿真等．     

摄像机沿光轴运动下的场景三维重建

讨论了采用针孔摄像机进行摄像机沿光轴运动下的场景三维重建的方法．基于摄像机轴向运动的特点和性质,利用该方法找到图像间的缩放因子,进而解决了轴向运动下的特征匹配;采用Sturm的摄像机自标定方法得到摄像机的内外参数;从而实现了摄像机沿光轴运动下的场景三维重建．     

双目视觉下三维人体运动跟踪算法*

由于人体运动的复杂性,人体运动轨迹的快速改变和人体自遮挡现象经常发生,这给人体运动跟踪带来了很大的困难。针对此问题提出了一种基于三维Kalman滤波器和人体约束的人体运动跟踪算法。该算法首先利用外极线约束和灰度互相关法对二维标记点进行立体匹配,计算各个标记点的三维位置,从而构建得到三维标记点;然后利用三维Kalman滤波器对三维标记点进行跟踪;最后利用人体约束检验和修正跟踪结果。实验结果表明,该算法能有效地对复杂人体动作进行跟踪并能从跟踪错误中正确恢复。     

3D Human model adaptation by frame selection and shape–texture optimization

We present a novel approach for 3D human body shape model adaptation to a sequence of multi-view images, given an initial shape model and initial pose sequence. In a first step, the most informative frames are determined by optimization of an objective function that maximizes a shape–texture likelihood function and a pose diversity criterion (i.e. the model surface area that lies close to the occluding contours), in the selected frames. Thereafter, a batch-mode optimization is performed of the underlying shape- and pose-parameters, by means of an objective function that includes both contour and texture cues over the selected multi-view frames.Using above approach, we implement automatic pose and shape estimation using a three-step procedure: first, we recover initial poses over a sequence using an initial (generic) body model. Both model and poses then serve as input to the above mentioned adaptation process. Finally, a more accurate pose recovery is obtained by means of the adapted model.We demonstrate the effectiveness of our frame selection, model adaptation and integrated pose and shape recovery procedure in experiments using both challenging outdoor data and the HumanEva data set.     

一种强壮的三维运动盲水印算法

提出了一种基于DCT变换的三维运动盲水印算法，算法结合了扩展频谱和量化的思想，通过对DCT域中频系数进行适当量化扰动来嵌入水印信息。选择合理的量化步长能保证对原运动信号不产生明显的视觉破坏；同时通过扩展频谱，让每一位水印信息包含在多个频率系数中，能有效增加抗攻击能力。实验表明，该算法对高斯白噪声、重采样、运动光滑以及重排列等有较好的鲁棒性。     

Lie Algebra and System Identification Techniques for 3D Rigid Motion Estimation and Monocular Tracking

This paper addresses the parameters’ estimation of 2D and 3D transformations. For the estimation we present a method based on system identification theory, we named it the “A-method”. The transformations are considered as elements of the Lie group GL(n) or one of its subgroups. We represent the transformations in terms of their Lie Algebra elements. The Lie algebra approach assures to follow the shortest path or geodesic in the involved Lie group. To prove the potencial of our method, two experiments are presented. The first one is a monocular estimation of 3D rigid motion of an object in the visual space. With this aim, the six parameters of the rigid motion are estimated based on measurements of the six parameters of the affine transformation in the image. Secondly, we present the estimation of the affine or projective transformations involved in monocular region tracking. Jaime Ortegón-Aguilar received his degree in computer sciences at the Universidad Autonoma de Yucatan in Merida, Mexico in 2000. He earned his M.Sc. degree at the Cinvestav in Guadalajara, Mexico in 2002. He received his PhD degree from Cinvestav in 2006. His research interests include image processing, computer vision, robotics and applications of geometric algebra. Eduardo Jose Bayro-Corrochano gained his Ph.D. in Cognitive Computer Science in 1993 from the University of Wales at Cardiff. From 1995 to 1999 he has been Researcher and Lecturer at the Institute for Computer Science, Christian Albrechts University, Kiel, Germany, working on applications of geometric Clifford algebra to cognitive systems. At present is a full professor at CINVESTAV Unidad Guadalajara, México, Department of Electrical Engineering and Computer Science. His current research interest focuses on geometric methods for artificial perception and action systems. It includes geometric neural networks, visually guided robotics, humanoids, color image processing, Lie bivector algebras for early vision and robot maneuvering. He developed the quaternion wavelet transform for quaternion multi-resolution analysis using the phase concept. He is associate editor of Robotics and Journal of Advanced Robotic Systems and member of the editorial board of Journal of Pattern Recognition, Journal of Mathematical Imaging and Vision, Iberoamerican Journal of Computer and Systems and Journal of Theoretical and Numerical Approximation. He is editor and author of the following books: Geometric Computing for Perception Action Systems, E. Bayro-Corrochano, Springer Verlag, 2001; Geometric Algebra with Applications in Science and Engineering, E. Bayro-Corrochano and G. Sobczyk (Eds.), Birkhauser 2001; Handbook of Geometric Computing for Pattern Recognition, Computer Vision, Neurocomputing and Robotics, E. Bayro-Corrochano, Springer Verlag, 2005. He has published over 120 refereed journal, book chapters and conference papers. He is fellow of the IAPR society.     

基于序列图像的三维重构

提出了一种基于序列图像的三维重构方法。在相机标定的基础上,利用两幅图像间的匹配关系得到一个初始的三维模型,然后利用剩余图像与初始模型间的匹配关系,采用反投影映射的方法计算出剩余图像的外方位元素。最后经过稠密匹配、三角剖分和纹理渲染得到物体的三维模型。实验结果表明,相机标定精度较高的情况下,采用该算法能够得到较低失真的三维模型。     

基于子空间集成学习的3维人体运动识别

如何对3维运动数据进行自动识别,是提高数据利用效率和进行计算机动画创作的前提。为了有效地识别运动数据,需要把运动数据投影到非线性流型低维子空间中,先识别出运动的内在结构,然后对运动的各个关节点分别进行学习,最后基于集成学习的方法产生强的隐马尔可夫学习器,以便能够对一些常见的运动类型进行自动识别。实验结果表明,这种基于子空间集成的人体运动识别方法较传统方法的检索精度、检索速度均有较大提高。     

Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion,Shape and Reflectance

In this paper we study the problem of recovering the 3D shape, reflectance, and non-rigid motion properties of a dynamic 3D scene. Because these properties are completely unknown and because the scene's shape and motion may be non-smooth, our approach uses multiple views to build a piecewise-continuous geometric and radiometric representation of the scene's trace in space-time. A basic primitive of this representation is the dynamic surfel, which (1) encodes the instantaneous local shape, reflectance, and motion of a small and bounded region in the scene, and (2) enables accurate prediction of the region's dynamic appearance under known illumination conditions. We show that complete surfel-based reconstructions can be created by repeatedly applying an algorithm called Surfel Sampling that combines sampling and parameter estimation to fit a single surfel to a small, bounded region of space-time. Experimental results with the Phong reflectancemodel and complex real scenes (clothing, shiny objects, skin) illustrate our method's ability to explain pixels and pixel variations in terms of their underlying causes—shape, reflectance, motion, illumination, and visibility.     

点状特征柔性物体三维运动捕获方法

针对具有点状特征的柔性物体,提出了一种三维运动捕获方法.首先,该方法利用两个标定的高速摄像机拍摄柔性物体的运动视频,并对图像进行立体校正;然后,采用DOG （Difference Of Gaussian）算法获取点状特征的位置,并提取特征点极值;其次,在一定范围的窗口上搜索匹配对,匹配左右图像的特征点;再次,通过三角测量法进行三维重建;最后,利用搜索策略进行时间序列上的匹配,实现动态柔性物体的三维运动捕获,并计算空间坐标、速度、加速度参数.实验结果表明,相比于采用sift算法匹配特征点捕获柔性运动物体的方法,本方法精度更高.     

基于D3D的三维虚拟人运动显示

开发了一套基于Direct 3D技术的应用程序开发包DXVHSDK,它可以在虚拟环境中载入、驱动并显示一个或多个三维虚拟人,具有绘制速度快、可靠性高、可扩展性强等优点,能够灵活地驱动虚拟角色完成各种复杂的运动和操作,可以广泛应用于仿真、娱乐、智能人机交互等领域．在DXVHSDK的基础上,还开发了一个可以控制虚拟人运动的原型系统．     

基于区域的手指三维运动跟踪

提出了基于区域的多连接体(手指)的三维运动跟踪算法.该算法首先用多约束融合的方法以及手指的运动特性,得到初始帧手指的三维结构;然后根据刚性多连接体的运动模型以及相应的姿势约束模型,给出了这一特殊运动模型三维运动估计的优化算法,此算法能够鲁棒地估计手指的三维运动;最后利用区域跟踪的方法获取多连接体三维运动,并在真实的手指序列图象中实现了该算法.实验结果证实了该算法的有效性.     

单目视频人体三维运动高效恢复

为解决计算机图形学和视觉领域的人体数据运动获取问题,提出一种从无标记点的单目视频恢复三维人体运动的方法.首先对人体侧影进行分析,获取躯干和未端节点位置信息;然后进行三维姿态优化.根据人体骨架特点,提出一个有效且计算简单的目标函数以及一种迭代优化策略,极大地减少了优化过程的计算量;设计了一个新颖的姿态序列恢复流程,克服了误差累积等传统跟踪方法的缺点.实验结果表明,文中方法可以准确地对视频中的复杂人体运动进行三维恢.     

基于MTi系统的三维运动检测方法的研究

MTi(motion tracker instrument)系统是一款精度较高,灵活性好的测量姿态和航向的系统。结合MTi系统分析了其测量方法,以及其输入、输出量之间的转换关系。通过对被测物体不同运动姿态的测量,应用MATLAB软件编程对系统输出的数据进行分析,观察欧拉角针对不同运动姿态的变化规律。对机械手的三维运动进行测量分析,为采摘机械手的轨迹规划和运动控制提供试验基础。     

Motion Recovery by Integrating over the Joint Image Manifold

Recovery of epipolar geometry is a fundamental problem in computer vision. The introduction of the “joint image manifold” (JIM) allows to treat the recovery of camera motion and epipolar geometry as the problem of fitting a manifold to the data measured in a stereo pair. The manifold has a singularity and boundary, therefore special care must be taken when fitting it. Four fitting methods are discussed—direct, algebraic, geometric, and the integrated maximum likelihood (IML) based method. The first three methods are the exact analogues of three common methods for recovering epipolar geometry. The more recently introduced IML method seeks the manifold which has the highest “support,” in the sense that the largest measure of its points are close to the data. While computationally more intensive than the other methods, its results are better in some scenarios. Both simulations and experiments suggest that the advantages of IML manifold fitting carry over to the task of recovering epipolar geometry, especially when the extent of the data and/or the motion are small.     

Iterative computation of 3D plane parameters

Knowledge of the position and orientation of 3D planes that exist in a scene is very important for many machine vision-based tasks, such as navigation, self-localization and docking. In this paper we propose two iterative methods for robustly computing the plane parameters out of given image sequences. Both methods include the (limited) ability to determine unknown parameters of the 3D-camera motion.