轨迹空间中基于特征符号搜索算法的非刚体三维运动重建研究

现有研究一般是在正交约束条件下采用最小二乘法来求解三维运动轨迹基重建问题,然而这些算法的性能容易受噪声影响,产生不适定性。为此,通过利用离散线性的方法为三维运动搜索最优轨迹基系数和三维结构,建立了一种基于特征符号搜索算法的三维运动轨迹基重建方法。利用该方法,对一系列单目图像序列进行了重建实验研究,结果表明所提出的方法是可行的,并提高了重建算法的准确性。     

一个基于运动的视频检索系统

在视频摄取的过程中,相机运动在某种程度上反映了当前视频的部分重要信息,通过对摄像机运动的提取能够更好地实现对视频的浏览以及检索,本文针对现有的依据光流场的分布来求相机运动参数的算法中不足,给出了一种依靠运动矢量的分布来求相机运动参数的方法,该方法能够在相机聚焦中心不在成像平面中心点时检测出给定类型的相机运动,并具有较好的鲁棒性,最后根据该算法实现了一个基于运动的视频检索系统.     

一种基于Multi-Egocentric视频运动轨迹重建的多目标跟踪算法

Egocentric视频具有目标运动剧烈、遮挡频繁、目标尺度差异明显及视角时变性强的特点,给目标跟踪任务造成了极大的困难。本文从重建不同视角Egocentric视频中各目标的运动轨迹出发,提出一种基于Multi-Egocentric视频运动轨迹重建的多目标跟踪算法,该方法基于多视角同步帧之间的单应性约束解决目标遮挡和丢失问题,然后根据多视角目标空间位置约束关系通过轨迹重建进一步优化目标定位,并采用卡尔曼滤波构建目标运动模型优化目标运动轨迹,在BJMOT、EPLF-campus4数据集上的对比实验验证了本文算法在解决Multi-Egocentric视频多目标跟踪轨迹不连续问题的有效性。     

基于目标轨迹空间关系的视频摘要方法

碰撞现象是视频摘要中需要避免的问题,在轨迹重排时一般通过碰撞代价函数进行约束,但是现有视频摘要方法在轨迹重排优化过程中需要重复计算轨迹间的碰撞代价,存在大量冗余运算量,为此提出了一种基于目标轨迹空间关系的视频摘要方法.该方法通过分析目标轨迹间的空间关系,可以在轨迹重排前预先判断两条轨迹是否会发生碰撞,据此定义了3种轨迹关系,并给出了碰撞代价的快速计算方法,从而较好地降低了现有视频摘要方法优化过程中的冗余计算,提高了视频摘要中轨迹重排的运算速度.实验结果验证了所提方法的有效性.     

单目视频人体三维运动高效恢复

为解决计算机图形学和视觉领域的人体数据运动获取问题,提出一种从无标记点的单目视频恢复三维人体运动的方法.首先对人体侧影进行分析,获取躯干和未端节点位置信息;然后进行三维姿态优化.根据人体骨架特点,提出一个有效且计算简单的目标函数以及一种迭代优化策略,极大地减少了优化过程的计算量;设计了一个新颖的姿态序列恢复流程,克服了误差累积等传统跟踪方法的缺点.实验结果表明,文中方法可以准确地对视频中的复杂人体运动进行三维恢.     

体育视频序列中基于IMM的运动目标跟踪算法

在视频处理领域的运动目标跟踪问题中,卡尔曼滤波器（KF）与扩展卡尔曼滤波器（EKF）已经得到了广泛的应用,但在复杂背景或是目标高机动运动的情况下跟踪效果并不理想。提出一种基于交互多模型算法（IMM）,并采用去偏转换测量卡尔曼滤波器（CMKF-D）对运动目标进行跟踪的算法。该算法有效地解决了单一模型无法与运动特性相匹配的问题,并克服了KF、EKF对非线性模型线性化所引入的误差。以足球视频为例进行仿真实验,结果表明该算法有效地提高了视频序列中运动目标跟踪的准确率。     

基于运动估计的无人机视频与遥测同步方法

针对无人机遥测数据和视频由于通讯延迟、链路不稳等因素,视频和遥测数据无法一一对应起来,需要对其进行同步处理的技术需求,提出了基于运动估计的视频与遥测同步方法。对于无人机飞行获取的遥测数据和视频数据,分别进行运动估计,得到基于遥测数据的运动编码和基于视频数据的运动编码,然后基于分析得到的运动编码进行匹配,进而根据匹配结果,对遥测数据进行插值处理,实现遥测与视频的同步。通过对总体运动模式的匹配,在载荷和无人机的遥测协议具体内容未知的情况下,根据运动模式实现视频与遥测的同步。通过灰度投影的方法进行针对无人机遥测数据和视频由于通讯延迟、链路不稳等因素,视频和遥测数据无法一一对应起来,需要对其进行同步处理的技术需求,提出了基于运动估计的视频与遥测同步方法。对于无人机飞行获取的遥测数据和视频数据,分别进行运动估计,得到基于遥测数据的运动编码和基于视频数据的运动编码,然后基于分析得到的运动编码进行匹配,进而根据匹配结果,对遥测数据进行插值处理,实现遥测与视频的同步。通过对总体运动模式的匹配,在载荷和无人机的遥测协议具体内容未知的情况下,根据运动模式实现视频与遥测的同步。通过灰度投影的方法进行运动估计以完成视频的运动编码,大大提高了视频运动估计的效率和速度,最终实现视频与遥测的同步。实现对遥测数据的高精度匹配和插值,提高同步的精度。运动估计以完成视频的运动编码,大大提高了视频运动估计的效率和速度,最终实现视频与遥测的同步。实现对遥测数据的高精度匹配和插值,提高同步的精度。     

视频序列中的运动目标检测

针对室内安全监控的场景特点,提出一种稳健的运动目标检测方法,对每个背景像素使用多个矢量表示,利用当前帧和背景帧像素的色度分量差分进行运动检测,并选择性地更新背景模型.实验表明,该方法能有效消除背景中振动对象的干扰,提高运动目标检测的准确性,有利于消除由于虚假目标而导致的误报警.     

一个有效的基于运动的视频检索系统

在视频摄取的过程中,摄象机的运动在某种程度上反映了当前视频的部分语义特征,所以通过对摄象机运动的提取能够更好地实现对视频的浏览以及基于内容的检索。针对以上应用,本文提出了一种用运动矢量的分布来提取摄相机运动的方法,测试后表明,该方法能有效地分析出多种相机运动并具有较好的鲁棒性。同时我们以此为基础实现了一个基于相机运动的视频检索系统。     

基于图像序列的非刚体三维运动恢复

研究图像序列中非刚体的三维运动重建问题。介绍非刚体运动重建的两种重要算法:奇异值分解法和线性迭代法。对迭代算法所用的重构方法进行修改,在迭代过程中应用结果更为精确的重构方法来求解非刚体的模型和旋转矩阵。对真实图像序列的实验结果验证了该算法的有效性和精确性。     

Wide Baseline Matching between Unsynchronized Video Sequences

3D reconstruction of a dynamic scene from features in two cameras usually requires synchronization and correspondences between the cameras. These may be hard to achieve due to occlusions, different orientation, different scales, etc. In this work we present an algorithm for reconstructing a dynamic scene from sequences acquired by two uncalibrated non-synchronized fixed affine cameras. It is assumed that (possibly) different points are tracked in the two sequences. The only constraint relating the two cameras is that every 3D point tracked in one sequence can be described as a linear combination of some of the 3D points tracked in the other sequence. Such constraint is useful, for example, for articulated objects. We may track some points on an arm in the first sequence, and some other points on the same arm in the second sequence. On the other extreme, this model can be used for generally moving points tracked in both sequences without knowing the correct permutation. In between, this model can cover non-rigid bodies with local rigidity constraints. We present linear algorithms for synchronizing the two sequences and reconstructing the 3D points tracked in both views. Outlier points are automatically detected and discarded. The algorithm can handle both 3D objects and planar objects in a unified framework, therefore avoiding numerical problems existing in other methods. This work was done while the authors were PhD students in the School of Computer Science and Engineering, the Hebrew University of Jerusalem.     

Reconstruction of non-rigid 3D shapes from stereo-motion

Several non-rigid structure from motion methods have been proposed so far in order to recover both the motion and the non-rigid structure of an object. However, these monocular algorithms fail to give reliable 3D shape estimates when the overall rigid motion of the sequence is small. Aiming to overcome this limitation, in this paper we propose a novel approach for the 3D Euclidean reconstruction of deformable objects observed by an uncalibrated stereo rig. Using a stereo setup drastically improves the 3D model estimation when the observed 3D shape is mostly deforming without undergoing strong rigid motion. Our approach is based on the following steps. Firstly, the stereo system is automatically calibrated and used to compute metric rigid structures from pairs of views. Afterwards, these 3D shapes are aligned to a reference view using a RANSAC method in order to compute the mean shape of the object and to select the subset of points which have remained rigid throughout the sequence. The selected rigid points are then used to compute frame-wise shape registration and to robustly extract the motion parameters from frame to frame. Finally, all this information is used as initial estimates of a non-linear optimization which allows us to refine the initial solution and also to recover the non-rigid 3D model. Exhaustive results on synthetic and real data prove the performance of our proposal estimating motion, non-rigid models and stereo camera parameters even when there is no rigid motion in the original sequence.     

摄像机沿光轴运动下的场景三维重建

讨论了采用针孔摄像机进行摄像机沿光轴运动下的场景三维重建的方法．基于摄像机轴向运动的特点和性质,利用该方法找到图像间的缩放因子,进而解决了轴向运动下的特征匹配;采用Sturm的摄像机自标定方法得到摄像机的内外参数;从而实现了摄像机沿光轴运动下的场景三维重建．     

A Closed-Form Solution to Non-Rigid Shape and Motion Recovery

Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem (Bregler, C., Hertzmann, A., and Biermann, H. 2000. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Brand, M. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition) utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique. An arbitrary linear transformation of the bases produces another set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.     

Detailed Real-Time Urban 3D Reconstruction from Video

The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU’s to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.     

乒乓球的三维运动轨迹重建

主要基于图像序列对乒乓球的运动轨迹进行三维重建,并对乒乓球运动形态进行分析.首先对采集的图像进行立体校正,利用颜色识别和改进的霍夫圆检测算法提取出序列图像中乒乓球的圆心坐标;然后根据前后帧图像的特征点坐标差值在时间序列上匹配特征点;最后,利用三角测量法对匹配的特征点进行三维重建,并计算出乒乓球不同时刻的速度和加速度,实现了动态物体的三维运动重建.实验结果表明该三维运动重建方法提高了特征提取的准确性,有效地实现了时间序列上的匹配,获得了物体的三维运动数据.     

结合视觉惯性模组的室内三维布局鲁棒重建方法

针对使用传统单目相机的全自动三维重建方法结果精确度差和整体结构理解缺失等问题,提出一种结合视觉惯性里程计和由运动到结构的全自动室内三维布局重建系统.首先利用视觉里程计获得关键帧图像序列和对应空间位置姿态,并利用运动恢复结构算法计算精确相机位姿;然后利用多图视立体几何算法生成高质量稠密点云;最后基于曼哈顿世界假设,针对典型的现代建筑室内场景,设计一种基于规则的自底向上的布局重建方法,得到最终房间外轮廓布局.使用浙江大学CAD&CG实验室场景现场扫描数据集和人工合成的稠密点云数据集作为实验数据,在Ubuntu 16.04和PCL 1.9环境下进行实验.结果表明,文中方法对三维点云噪声容忍度高,能够有效地重建出室内场景的三维外轮廓布局.     

适用于单目视频的无标记三维人体运动跟踪

在无标记人体运动跟踪过程中,由于被跟踪目标缺乏明显的特征以及背景复杂而使得跟踪到的人体运动姿态与真实值偏差较大,不能进行长序列视频跟踪.针对这一现象,提出一种基于形变外观模板匹配进行单目视频的三维人体运动跟踪算法,其中所用的人体外观模型由三维人体骨骼模型及二维纸板模型组成.首先根据人体骨骼比例约束采用逆运动学计算出关节旋转欧拉角;然后利用正向运动学求得纸板模型中像素在三维空间中的坐标,将这些像素根据摄像机成像模型投影到二维图像中得到形变外观模板;最后采用直方图匹配得到人体运动跟踪结果.实验结果表明,该算法对于一些复杂的长序列人体运动能够得到较为理想的跟踪结果,可应用于人机交互和动画制作等领域.     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.