基于自适应权重的全局图割立体匹配算法

针对立体匹配中存在纹理、遮挡区域和深度不连续的问题,提出一种基于自适应权重的全局图割立体匹配算法,一方面,采用单像素自适应权重加窗匹配能够减少深度不连续和稀疏纹理处匹配的误差;另一方面,对于图割中的平滑项表示和遮挡处理,使用一定平滑约束和遮挡约束构建能量函数而取得最优解。实验结果表明,该算法能保证匹配的可靠性。     

基于遮挡修补的TV-L1光流算法

当前比较流行的TV-L1光流算法,在不失精确度的前提下,能够利用双向求解机制来降低运算量,但无法有效地处理由间断、遮挡等因素造成的错误光流分量的缺陷。通过前向光流和后向光流的运动一致性理论来判断遮挡区域的光流分量,通过单调递减函数对遮挡区域进行处理,抑制了遮挡区域错误光流对邻域的扩散,提出了同帧邻域光流的横向修补和相邻帧光流的纵向修补。实验表明,该方法能够很好地处理遮挡情况,提高了光流的计算精度。     

一种基于最小割的稠密视差图恢复算法

针对经典最小割算法计算量大和适应性不足的问题,提出一种改进的基于网络最小割计算稠密深度图的全局优化方法.首先,根据视差变化与不连续区域之间的关系,定义了具有一定适应性的平滑约束和遮挡约束,然后使用网络最小割算法,求解遮挡情况下的稠密视差.其次,在分析最小割算法复杂性的基础上,给出了一种受限α-扩展(α-expansion)操作,该操作根据灰度连通性和特征点匹配的结果对每次网络构造的顶点进行控制,减少网络中顶点和边的数目,可有效提高计算效率.实验结果显示,该算法在保证视差恢复准确性的前提下,能以较快的速度计算出较理想的稠密视差图.     

基于区域边界约束和图割优化的稠密匹配算法

基于图割全局优化的稠密匹配算法中,平滑项表示和遮挡处理是关键问题.具有凸性平滑项的能量函数可以求得全局最优解,但所求结果在视差跳变处过于平滑;而具有非凸平滑项的能量函数虽保留了视差的非连续性,但目前只能使用循环算法求得次优解.为此,基于"视差跳变绝大部分发生在颜色的不连续处"这一设定,提出一种利用区域边界和边界像素间的约束构建能量函数的稠密匹配表示方法,使得该函数既能求得全局最优解,又能使最终结果满足平滑项的"非连续保留"性,且体现遮挡约束、顺序性约束,并显著提高计算效率,在速度和效果上取得较好平衡.     

一种基于PSO的改进光流算法

针对变分光流法无法有效检测由间断、遮挡等因素造成的错误光流分量的缺陷,提出一种基于PSO(Particle Swarm Optimization)的光流算法。该方法在Classic+NL算法模型的基础上计算出光流后,引入前向光流和后向光流的运动一致性理论来判断遮挡区域,并通过基于PSO的修补法来实现对遮挡区域错误光流的有效修补,同时,利用邻域光流修补法实现了再次修补。实验结果表明,该方法能有效克服由间断、遮挡等因素造成的错误光流分量的缺陷,更准确地刻画出光流,提高光流的计算精度。     

一种基于光流和能量的图像匹配算法

结合光流与图像信息,提出一种获取稠密视差的图像匹配算法.首先对于基线较大的左右图像,在多分辨率框架下采用由粗到精的策略计算光流,从而实现大偏移量时的光流获取.其次为了避免光流在图像边界上的不可靠性,通过光流计算所得的光流场作为初始视差图,采用基于能量的方法依据对应的图像梯度场对光流场内部进行平滑并保持边缘的不连续性,最终得到精准稠密的视差图.实验验证,该方法是一种行之有效的图像匹配算法.     

基于形状先验和图割的彩色图像分割

基于图割理论的图像分割方法在二值标号问题中可以获取全局最优解,而在多标号问题中可以获取带有很强特征的局部最优解。但对于含有噪声或遮挡物等复杂的图像,分割结果不完整,效果并不令人满意,提出了一种基于形状先验和图割的图像分割方法。以图割算法为基础,加入形状先验知识,使该算法包含更多约束信息,从而限制感兴趣区域的搜寻空间,能够更好地分割出完整的目标,增加了算法的精确度。针对形状的仿射变换,运用特征匹配算法进行处理,使算法更加具有灵活性,能够应对不同类型的情况。实验表明了该算法的有效性。     

面向多种场景的视频对象自动分割算法

针对当前应用于视频对象分割的图割方法容易在复杂环境、镜头移动、光照不稳定等场景下鲁棒性不佳的问题,提出了结合光流和图割的视频对象分割算法.主要思路是通过分析前景对象的运动信息,得到单帧图像上前景区域的先验知识,从而改善分割结果.论文首先通过光流场采集视频中动作信息,并提取出前景对象先验区域,然后结合前景和背景先验区域建立图割模型,实现前景对象分割.最后为提高算法在不同场景下的鲁棒性,本文改进了传统的测地显著性模型,并基于视频本征的时域平滑性,提出了基于混合高斯模型的动态位置模型优化机制.在两个标准数据集上的实验结果表明,所提算法与当前其他视频对象分割算法相比,降低了分割结果的错误率,有效提高了在多种场景下的鲁棒性.     

局部平衡的判别近邻嵌入算法

判别近邻嵌入算法（discriminant neighborhood embedding,DNE）通过构造邻接图,使得在投影子空间中能够保持原始数据的局部结构,能有效地发现最佳判别方向。但是它有两方面的不足：一方面不能标识样本点的近邻样本点位置信息,从而不能更好地保持邻域结构;另一方面当数据不均衡时,不能实现子空间中类内聚合或者类间分离的目的,这不利于分类。为此提出了一种新的有监督子空间学习算法--局部平衡的判别近邻嵌入算法（locality-balanced DNE,LBDNE）。在构建邻接图时,局部平衡的判别近邻嵌入算法分别建立同类邻接图和异类邻接图,并通过引入一个控制参数,有效地平衡了类内与类间的关系。该算法与其他经典算法相比,在人脸识别问题上具有较高的识别率,充分说明了局部平衡的判别近邻嵌入算法能够有效地处理识别问题。     

一种基于图割理论的快速立体匹配算法

针对传统的立体匹配算法中存在的低纹理区域和遮挡区域匹配精度低、实时性不好等问题,提出了一种基于图割理论的立体匹配算法.把图像分割成色彩单一的不同区域;计算初始视差图,利用可靠点求取各分割区域的平面模板参数,对模板参数相同的相邻区域进行融合;构造全局能量函数,采用图割算法求取全局能量最小的视差最优分配.实验结果表明,该算法对低纹理区域和遮挡区域均有较好的匹配结果,能够满足高精度、高实时性的要求.     

抗多遮挡物干扰的光场深度信息估计算法

针对光场的深度信息估计中，由遮挡带来的干扰，造成遮挡处的深度值估计精度低的问题，提出一种抗多遮挡物干扰的光场深度信息估计算法。对场景点的angular patch图像进行多遮挡物分析，分析遮挡物的位置分布特性。基于分类的思想提出改进AP（Affinity Propagation）聚类算法将场景点的angular patch图像进行像素点分类，将遮挡物和场景点分离。对分离遮挡物后的angular patch图像提出联合像素强度信息熵及中心方差的目标函数，最小化该函数，求得场景点的初始深度值估计。对初始深度值估计提出基于MAP-MRF（最大后验估计的马尔可夫随机场）框架的平滑约束能量函数进行平滑优化，并采用图割算法（Graph Cut Algorithm）求解，得到场景的最终深度值估计。实验结果表明，相较于现有深度信息估计算法，所提算法提升了遮挡处的估计精度。     

Detecting Occlusions as an Inverse Problem

Occlusions generally become apparent when integrated over time because violations of the brightness-constancy constraint of optical flow accumulate in occluded areas. Based on this observation, we propose a variational model for occlusion detection that is formulated as an inverse problem. Our forward model adapts the brightness constraint of optical flow to emphasize occlusions by exploiting their temporal behavior, while spatio-temporal regularizers on the occlusion set make our model robust to noise and modeling errors. In terms of minimization, we approximate the resulting variational problem by a sequence of convex optimizations and develop efficient algorithms to solve them. Our experiments show the benefits of the proposed formulation, both forward model and regularizers, in comparison to the state-of-the-art techniques that detect occlusion as the residual of optical-flow estimation.     

Dual space directional occlusion

Current real-time ambient or directional occlusion approximation methods are either screen space or object space based. Both methods suffer from drawbacks such as time incoherence and occlusion popping for screen space methods or loss of detailed occlusion effects caused by geometry simplification for object space methods. We present an algorithm that combines both methods to overcome these drawbacks. To avoid over or underestimations during the combination, we use the Spherical Harmonics representation of the directional occlusion information. We therefore combine “Screen Space Spherical Harmonics Occlusion” (Herholz et al. in VMV, 2012) with “Interactive Voxel Cone Tracing” (CT) (Crassin et al. in Comput. Graph. Forum, 2011). The result is a directional occlusion approximation including both occlusions from distant or not directly visible objects and detailed occlusions effects from fine geometrical structures. To increase the quality of CT for occlusion sampling, we also present several extensions such as view dependent cascaded voxelization and a method for voxel coverage estimation.     

自适应成本量的抗遮挡光场深度估计算法

目的 光场相机通过一次成像同时记录场景的空间信息和角度信息,获取多视角图像和重聚焦图像,在深度估计中具有独特优势。遮挡是光场深度估计中的难点问题之一,现有方法没有考虑遮挡或仅仅考虑单一遮挡情况,对于多遮挡场景点,方法失效。针对遮挡问题,在多视角立体匹配框架下,提出了一种对遮挡鲁棒的光场深度估计算法。方法 首先利用数字重聚焦算法获取重聚焦图像,定义场景的遮挡类型,并构造相关性成本量。然后根据最小成本原则自适应选择最佳成本量,并求解局部深度图。最后利用马尔可夫随机场结合成本量和平滑约束,通过图割算法和加权中值滤波获取全局优化深度图,提升深度估计精度。结果 实验在HCI合成数据集和Stanford Lytro Illum实际场景数据集上展开,分别进行局部深度估计与全局深度估计实验。实验结果表明,相比其他先进方法,本文方法对遮挡场景效果更好,均方误差平均降低约26.8%。结论 本文方法能够有效处理不同遮挡情况,更好地保持深度图边缘信息,深度估计结果更准确,且时效性更好。此外,本文方法适用场景是朗伯平面场景,对于含有高光的非朗伯平面场景存在一定缺陷。     

基于光流的快速人体姿态估计

针对目前深度学习领域人体姿态估计算法计算复杂度高的问题,提出了一种基于光流的快速人体姿态估计算法.在原算法的基础上,首先利用视频帧之间的时间相关性,将原始视频序列分为关键帧和非关键帧分别处理（相邻两关键帧之间的图像和前向关键帧组成一个视频帧组,同一视频帧组内的视频帧相似）,仅在关键帧上运用人体姿态估计算法,并通过轻量级光流场将关键帧识别结果传播到其他非关键帧.其次针对视频中运动场的动态特性,提出一种基于局部光流场的自适应关键帧检测算法,以根据视频的局部时域特性确定视频关键帧的位置.在OutdoorPose和HumanEvaI数据集上的实验结果表明,对于存在背景复杂、部件遮挡等问题的视频序列中,所提算法较原算法检测性能略有提升,检测速度平均可提升89.6%.     

Motion segmentation using occlusions

We examine the key role of occlusions in finding independently moving objects instantaneously in a video obtained by a moving camera with a restricted field of view. In this problem, the image motion is caused by the combined effect of camera motion (egomotion), structure (depth), and the independent motion of scene entities. For a camera with a restricted field of view undergoing a small motion between frames, there exists, in general, a set of 3D camera motions compatible with the observed flow field even if only a small amount of noise is present, leading to ambiguous 3D motion estimates. If separable sets of solutions exist, motion-based clustering can detect one category of moving objects. Even if a single inseparable set of solutions is found, we show that occlusion information can be used to find ordinal depth, which is critical in identifying a new class of moving objects. In order to find ordinal depth, occlusions must not only be known, but they must also be filled (grouped) with optical flow from neighboring regions. We present a novel algorithm for filling occlusions and deducing ordinal depth under general circumstances. Finally, we describe another category of moving objects which is detected using cardinal comparisons between structure from motion and structure estimates from another source (e.g., stereo).     

Motion estimation and segmentation

In the general structure-from-motion (SFM) problem involving several moving objects in a scene, the essential first step is to segment moving objects independently. We attempt to deal with the problem of optical flow estimation and motion segmentation over a pair of images. We apply a mean field technique to determine optical flow and motion boundaries and present a deterministic algorithm. Since motion discontinuities represented by line process are embedded in the estimation of the optical flow, our algorithm provides accurate estimates of optical flow especially along motion boundaries and handles occlusion and multiple motions. We show that the proposed algorithm outperforms other well-known algorithms in terms of estimation accuracy and timing.     

Multiple people tracking and pose estimation with occlusion estimation

Simultaneously tracking poses of multiple people is a difficult problem because of inter-person occlusions and self occlusions. This paper presents an approach that circumvents this problem by performing tracking based on observations from multiple wide-baseline cameras. The proposed global occlusion estimation approach can deal with severe inter-person occlusions in one or more views by exploiting information from other views. Image features from non-occluded views are given more weight than image features from occluded views. Self occlusion is handled by local occlusion estimation. The local occlusion estimation is used to update the image likelihood function by sorting body parts as a function of distance to the cameras. The combination of the global and the local occlusion estimation leads to accurate tracking results at much lower computational costs. We evaluate the performance of our approach on a pose estimation data set in which inter-person and self occlusions are present. The results of our experiments show that our approach is able to robustly track multiple people during large movement with severe inter-person occlusions and self occlusions, whilst maintaining near real-time performance.     

Motion from occlusions

In computer vision, occlusions are almost always seen as undesirable singularities that pose difficult challenges to image motion analysis problems, such as optic flow computation, motion segmentation, disparity estimation, or egomotion estimation. However, it is well known that occlusions are extremely powerful cues for depth or motion perception, and could be used to improve those methods.

In this paper, we propose to recover camera motion information based uniquely on occlusions, by observing two specially useful properties: occlusions are independent of the camera rotation, and reveal direct information about the camera translation.

We assume a monocular observer, undergoing general rotational and translational motion in a static environment. We present a formal model for occlusion points and develop a method suitable for occlusion detection. Through the classification and analysis of the detected occlusion points, we show how to retrieve information about the camera translation (FOE). Experiments with real images are presented and discussed in the paper.