3D视频目标分割与快速跟踪

提出一种面向3D视频的目标分割与快速跟踪算法,主要分为空域分割和时域与视域跟踪两个主要步骤。空域分割采用基于改进C-V模型的方法。首先基于帧差法提取运动区域作为目标的初始轮廓,以减少迭代次数加快收敛速度。然后在传统的C-V模型中加入图像边缘信息以加强C-V模型对模糊边缘的处理能力,提高分割结果的主管视觉效果。时域和视域的目标跟踪采用基于改进生物地理优化的快速跟踪方法。实验结果表明所提分割与跟踪算法快速、高效。     

Interactive reconstruction via geometric probing

Geometric probing considers problems of determining a geometric structure or some aspect of that structure from the results of a mathematical or physical measuring device, a probe. The field of geometric probing is surveyed, with results ordered by a probing model. The emphasis is on interactive reconstruction, where the results of all previous measurements are used to determine the orientation of the next probe so it provides the maximum amount of information about the structure. Through interactive reconstruction, finite determination strategies exist for such diverse models as finger, X-ray, and half-plane probes     

2D facial expression recognition via 3D reconstruction and feature fusion

In this paper, a novel feature extraction method is proposed for facial expression recognition by extracting the feature from facial depth and 3D mesh alongside texture. Accordingly, the 3D Facial Expression Generic Elastic Model (3D FE-GEM) method is used to reconstruct an expression-invariant 3D model from the human face. Then, the texture, depth and mesh are extracted from the reconstructed face model. Afterwards, the Local Binary Pattern (LBP), proposed 3D High-Low Local Binary Pattern (3DH-LLBP) and Local Normal Binary Patterns (LNBPs) are applied to texture, depth and mesh of the face, respectively, to extract the feature from 2D images. Finally, the final feature vectors are generated through feature fusion and are classified by the Support Vector Machine (SVM). Convincing results are acquired for facial expression recognition on the CK+, CK, JAFFE and Bosphorus image databases compared to several state-of-the-art methods.     

Three-dimensional modeling from endoscopic video using geometric constraints via feature positioning

The endoscope is a popular imaging modality used in many preevaluations and surgical treatments, and is also one of the essential tools in minimally invasive surgery. However, regular endoscopes provide only 2-D images. Even though stereoendoscopy systems can display 3-D images, the real anatomical structure of the observed lesion is unavailable and can only be judged by the surgeon's imagination. In this paper, we present a constraint-based factorization method for reconstructing 3-D structures registered to the patient, from 2-D endoscopic images. The proposed method incorporates the geometric constraints from the tracked surgical instrument into the traditional factorization method based on frame-to-frame feature motion on the endoscopically viewed scene. Experiments with real and synthetic data demonstrate good real-scale 3-D extraction, with greater accuracy than is available from traditional methods. The reconstruction process can also be accomplished in a few seconds, making it suitable for on-line surgical applications to provide surgeons with additional 3-D shape information, critical distance monitoring and warnings.     

Scalable 3D video streaming over P2P networks with playback length changeable chunk segmentation

3D video distribution over P2P networks has been thought as a promising way for 3D video entering home. The convergence of scalable 3D video coding and P2P streaming can provide diverse 3D experiences for heterogeneous clients with high distribution efficiencies. However, the conventional chunk segmentation and scheduling algorithms originally aiming at the non-scalable 2D video streaming are not very efficient for scalable 3D video streaming over P2P networks due to the particular data characteristics of scalable 3D video. Based on this motivation, this paper first presents a playback length changeable 3D video chunk segmentation (PLC3DCS) algorithm to provide different error resilience strengths to video and depth as well as layers with different importance levels in the 3D video transmission. Then, a hybrid-priority based chunk scheduling (HPS) algorithm is proposed to be tied in with the proposed chunk segmentation algorithm to further promote the overall 3D video P2P streaming performance. The simulation results show that the proposed PLC3DCS algorithm with the corresponding HPS can increase the success delivery rates of chunks with more important levels, and further improve the user’s quality of 3D experience.     

Towards 3-D scene reconstruction from broadcast video

Three-dimensional (3-D) scene reconstruction from broadcast video is a challenging problem with many potential applications, such as 3-D TV, free-view TV, augmented reality or three-dimensionalization of two-dimensional (2-D) media archives. In this paper, a flexible and effective system capable of efficiently reconstructing 3-D scenes from broadcast video is proposed, with the assumption that there is relative motion between camera and scene/objects. The system requires no a priori information and input, other than the video sequence itself, and capable of estimating the internal and external camera parameters and performing a 3-D motion-based segmentation, as well as computing a dense depth field. The system also serves as a showcase to present some novel approaches for moving object segmentation, sparse and dense reconstruction problems. According to the simulations for both synthetic and real data, the system achieves a promising performance for typical TV content, indicating that it is a significant step towards the 3-D reconstruction of scenes from broadcast video.     

Computer-assisted registration, segmentation, and 3D reconstruction from images of neuronal tissue sections

Neuroscientists have studied the relationship between nerve cell morphology and function for over a century. To pursue these studies, they need accurate three-dimensional models of nerve cells that facilitate detailed anatomical measurement and the identification of internal structures. Although serial transmission electron microscopy has been a source of such models since the mid 1960s, model reconstruction and analysis remain very time consuming. The authors have developed a new approach to reconstructing and visualizing 3D nerve cell models from serial microscopy. An interactive system exploits recent computer graphics and computer vision techniques to significantly reduce the time required to build such models. The key ingredients of the system are a digital "blink comparator" for section registration, "snakes," or active deformable contours, for semiautomated cell segmentation, and voxel-based techniques for 3D reconstruction and visualization of complex cell volumes with internal structures.     

基于视觉里程计的单目红外视频三维重建

针对红外视频的特点,提出了一种基于直接法和稀疏法视觉里程计的单目红外视频三维重建方法。该方法首先通过对红外热像仪标定获得热像仪内参,然后构建直接法和稀疏法视觉里程计模型,视觉里程计前端执行帧管理和点管理的任务,利用滑动窗口并借助高斯-牛顿迭代对总光度误差进行优化,计算出直接法和稀疏法视觉里程计模型所依赖的所有变量,完成定位热像仪和建图的任务。通过实验证明了该方法能够实时实现对单目红外视频进行三维重建。     

基于改进特征的序列图像三维重建

近年来序列图像三维重建技术得到了重视,提出了很多算法.本文对Harris特征提取算法进行了改进,并在在序列图像中进行特征跟踪,在特征的稀疏重建和稠密重建后,使用估计样本一致性算法实现三维重建,提高了序列图像三维重建的精度.     

基于动作标准序列的3D视频人体动作识别

基于3D视频的人体动作识别近年来受到越来越广泛的关注。基于动态时间规整的算法考虑了动作的时序信息,并能较好地解决人体运动在时间上的不确定性,但是随着训练样本增加,效率会变得较低。本文提出了一种基于动作标准序列的动作识别方法。通过特征提取将3D动作视频样本构建为动作序列,在动态时间规整度量下将动作标准序列学习建模成一个序列平均的优化问题,并使用动态时间规整重心平均算法(DBA)求解。对于动作类别类中存在显著差异的场景,研究了多重动作标准序列学习,并针对无监督学习的情况,提出了DBA-K-means聚类算法。实验结果表明,该方法可进一步提高动作识别的效率和准确率。     

视频监控中运动图像序列三维重建

视频监控中运动图像序列三维重建一直是三维重建研究的重点,其重建效果会对运动图像的清晰度产生影响。但当前三维重建方法都是通过获取二维视频监控中的运动图像序列,通过基于Java Applet与Java Application编程,采用体绘制法完成视频监控中二维运动图像序列的三维重建,该方法无法保证重建后的图像质量,导致图像清晰度不高。为此,提出基于图像特征点提取与匹配的视频监控中运动图像序列三维重建方法。首先,对视频监控中运动图像序列特征点进行检测,并对特征点一定邻域内图像的纹理、结构以及其他特征进行统计,通过对运动图像序列特征点的特征比较,完成运动图像序列特征点提取与匹配;然后,对视频监控中运动图像序列的结构和运动初始化,并进行视频监控相机的自标定,实现对视频监控中运动图像序列的三维重建。实验结果表明,所提方法能够有效提高三维重建后视频监控中运动图像序列的清晰度,减少三维重建流程,提高运动图像序列重建效率,具有良好的使用价值。     

一种基于内容表示的图像序列运动分割算法

提出了一种MPEG－4标准所规范的基于内容表示的图像序列运动分割算法。首先进行图像序列二维运动估计并计算位移帧差,然后,将位移帧差作为将运动物体从静止背景中分离出来的标准进行分割;最后对分割结果进行形态滤波。实验结果显示,在物体存在整体运动的情况下,分割结果表现出了很好的准确度,可以成功地提取图像序列中的运动物体。     

Automatic 3D face synthesis using single 2D video frame

3D face synthesis has been extensively used in many applications over the last decade. Although many methods have been reported, automatic 3D face synthesis from a single video frame still remains unsolved. An automatic 3D face synthesis algorithm is proposed, which resolves a number of existing bottlenecks.     

An interactive 3D video system for human facial reconstruction and expression modeling

A 3D facial reconstruction and expression modeling system which creates 3D video sequences of test subjects and facilitates interactive generation of novel facial expressions is described. Dynamic 3D video sequences are generated using computational binocular stereo matching with active illumination and are used for interactive expression modeling. An individual’s 3D video set is annotated with control points associated with face subregions. Dragging a control point updates texture and depth in only the associated subregion so that the user generates new composite expressions unseen in the original source video sequences. Such an interactive manipulation of dynamic 3D face reconstructions requires as little preparation on the test subject as possible. Dense depth data combined with video-based texture results in realistic and convincing facial animations, a feature lacking in conventional marker-based motion capture systems.     

Statistical bias in 3-D reconstruction from a monocular video.

The present state-of-the-art in computing the error statistics in three-dimensional (3-D) reconstruction from video concentrates on estimating the error covariance. A different source of error which has not received much attention is the fact that the reconstruction estimates are often significantly statistically biased. In this paper, we derive a precise expression for the bias in the depth estimate, based on the continuous (differentiable) version of structure from motion (SfM). Many SfM algorithms, or certain portions of them, can be posed in a linear least-squares (LS) framework Ax = b. Examples include initialization procedures for bundle adjustment or algorithms that alternately estimate depth and camera motion. It is a well-known fact that the LS estimate is biased if the system matrix A is noisy. In SfM, the matrix A contains point correspondences, which are always difficult to obtain precisely; thus, it is expected that the structure and motion estimates in such a formulation of the problem would be biased. Existing results on the minimum achievable variance of the SfM estimator are extended by deriving a generalized Cramer-Rao lower bound. A detailed analysis of the effect of various camera motion parameters on the bias is presented. We conclude by presenting the effect of bias compensation on reconstructing 3-D face models from rendered images.     

Retrieving 3D information from video for robot control andsurveillance

A technique has been developed at Roke Manor Research Ltd. for recovering three-dimensional (3D) information from conventional video data. This is based on matching features in a scene observed from different viewpoints, either as a result of intrinsic camera or object motion or through the use of multiple cameras. The technique has been examined as a means of supplying guidance information to a robot manipulator and an autonomous vehicle. Results indicate that the method is capable of recognising basic surface structure in a variety of indoor and outdoor situations. The essential elements of the technique are described and a real-time system implementation is outlined. A surveillance application is also discussed whereby the derived 3D representation can be exploited in a data compression device to communicate concise 3d information to a remote operator or observer     

Three stage deep network for 3D human pose reconstruction by exploiting spatial and temporal data via its 2D pose

3D Human Pose Reconstruction (HPR) is a challenging task due to less availability of 3D ground truth data and projection ambiguity. To address these limitations, we propose a three-stage deep network having the workflow of 2D Human Pose Estimation (HPE) followed by 3D HPR; which utilizes the proposed Frame Specific Pose Estimation (FSPE), Multi-Stage Cascaded Feature Connection (MSCFC) and Feature Residual Connection (FRC) Sub-level Strategies. In the first stage, the FSPE concept with the MSCFC strategy has been used for 2D HPE. In the second stage, the basic deep learning concepts like convolution, batch normalization, ReLU, and dropout have been utilized with the FRC Strategy for spatial 3D reconstruction. In the last stage, LSTM deep architecture has been used for temporal refinement. The effectiveness of the technique has been demonstrated on MPII, Human3.6M, and HumanEva-I datasets. From the experiments, it has been observed that the proposed method gives competitive results to the recent state-of-the-art techniques.     

Semantics-aware transformer for 3D reconstruction from binocular images

Existing multi-view three-dimensional(3 D) reconstruction methods can only capture single type of feature from input view, failing to obtain fine-grained semantics for reconstructing the complex shapes. They rarely explore the semantic association between input views, leading to a rough 3 D shape. To address these challenges, we propose a semantics-aware transformer(SATF) for 3 D reconstruction. It is composed of two parallel view transformer encoders and a point cloud transformer decoder, and t...