M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

When occlusion is minimal, a single camera is generally sufficient to detect and track objects. However, when the density of objects is high, the resulting occlusion and lack of visibility suggests the use of multiple cameras and collaboration between them so that an object is detected using information available from all the cameras in the scene.In this paper, we present a system that is capable of segmenting, detecting and tracking multiple people in a cluttered scene using multiple synchronized surveillance cameras located far from each other. The system is fully automatic, and takes decisions about object detection and tracking using evidence collected from many pairs of cameras. Innovations that help us tackle the problem include a region-based stereo algorithm capable of finding 3D points inside an object knowing only the projections of the object (as a whole) in two views, a segmentation algorithm using bayesian classification and the use of occlusion analysis to combine evidence from different camera pairs.The system has been tested using different densities of people in the scene. This helps us determine the number of cameras required for a particular density of people. Experiments have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.     

Generalized Multiple Baseline Stereo and Direct Virtual View Synthesis Using Range-Space Search, Match, and Render

A new range-space approach is described for synergistic resolution of both stereovision and reflectance (visual) modeling problems simultaneously. This synergistic approach can be applied to arbitrary camera arrangements with different intrinsic and extrinsic parameters, image types, image resolutions, and image number. These images are analyzed in a step-wise manner to extract 3-D range measurements and also to render a customized perspective view. The entire process is fully automatic. An extensive and detailed experimental validation phase supports the basic feasibility and generality of the Range-Space Approach.     

基于宽基线立体视觉的远距离三维重建

针对月球车视觉导航中远距离场景的三维重建问题,提出一种基于相机自标定的宽基线立体视觉方法.该方法首先提取立体图对的尺寸不变性(SIFT)特征并进行匹配,得到特征点对应关系;再使用外极线几何约束描述立体图像校正的直射变换,对其Sampson误差使用Levenberg-Marquardt(LM)算法求得最小二乘最优解,完成相机参数估计和图像校正;对校正后的图像使用种子像素视差扩张算法进行宽基线立体匹配,根据匹配得到的视差完成目标场景三维重建.实验表明:该方法解决了月球车相机的现场标定难题,并能够解决宽基线立体视觉面临的光照差异、透视畸变、遮挡等问题,对远距离山脉等自然场景的三维重建效果良好,重建精度较高.     

基于全局相似性约束的多视点图像拼接

针对在大视差场景下进行图像拼接时容易出现的扭曲、失真、重影等现象,提出了一种基于全局相似性约束的多视点图像拼接算法。在传统细分网格模型的基础上,添加一个先验的全局相似约束项,对待拼接图像在整体上进行相似性转化,同时分别选取视角不同、亮度不同、复杂场景的三组图像,与已有算法的拼接效果进行对比,并采用质量评价方法对图像进行客观评价。结果表明：提出的方法在拼接效果上可以有效降低已有方法对大视差图像拼接时产生的扭曲和重影,且质量评价提高了10%以上;由于对图像的视角及场景深度无严格要求,因而具有良好的适应性,能够更自然地拼接大视差图像。     

融合语义先验和渐进式深度优化的宽基线3维场景重建

目的 基于视觉的3维场景重建技术已在机器人导航、航拍地图构建和增强现实等领域得到广泛应用。不过,当相机出现较大运动时则会使得传统基于窄基线约束的3维重建方法无法正常工作。方法 针对宽基线环境,提出了一种融合高层语义先验的3维场景重建算法。该方法在马尔可夫随机场（MRF）模型的基础上,结合超像素的外观、共线性、共面性和深度等多种特征对不同视角图像中各个超像素的3维位置和朝向进行推理,从而实现宽基线条件下的初始3维重建。与此同时,还以递归的方式利用高层语义先验对相似深度超像素实现合并,进而对场景深度和3维模型进行渐进式优化。结果 实验结果表明,本文方法在多种不同的宽基线环境,尤其是相机运动较为剧烈的情况下,依然能够取得比传统方法更为稳定而精确的深度估计和3维场景重建效果。结论 本文展示了在宽基线条件下如何将多元图像特征与基于三角化的几何特征相结合以构建出精确的3维场景模型。本文方法采用MRF模型对不同视角图像中超像素的3维位置和朝向进行同时推理,并结合高层语义先验对3维重建的过程提供指导。与此同时,还使用了一种递归式框架以实现场景深度的渐进式优化。实验结果表明,本文方法在不同的宽基线环境下均能够获得比传统方法更接近真实描述的3维场景模型。