Stereo and motion correspondences using nonlinear optimization method

This paper presents a new approach for using stereo and motion correspondences to solve the problem of tracking multiple independently moving features. In this approach, quantitative relational structure (QRS) is proposed as a framework for the integration of stereo–motion correspondences. The similarity function, tightly coupled to stereo and motion cues, is constructed on QRS, and then energy function E² consisting of the similarity function is defined. The tracking problem can be converted into the maximization problem of the energy function E². The stereo and motion correspondences that maximize E² are recovered by applying an extended graduated assignment algorithm. The relaxation labeling method is also presented for the comparison with the proposed method. Experimental results are presented to illustrate the performance of the proposed method.     

Video segmentation algorithm based on superpixel link weight model

Multimedia Tools and Applications - Based on the traditional segmentation algorithms, this paper proposes unsupervised video segmentation approach. The proposed algorithm applies superpixel to...     

基于超像素的点追踪方法

目的由于当前大多数的追踪算法都是使用目标外观模型和特征进行目标的匹配,在长时间的目标追踪过程中,目标的尺度和形状均会发生变化,再加上计算机视觉误差,都会导致追踪的失误。提出一种高效的目标模型用于提高追踪的效率和成功率。方法采用分割后提取的目标特征来进行建模表示外观结构,利用图像分割的方法,将被追踪的目标区域分割成多个超像素块,结合SIFT特征,形成词汇本,并计算每个词在词汇本中的权值,作为目标的外观模型。利用外观模型确定目标对象的关键点位置后,通过使用金字塔Lucas-Kanade追踪器预测关键点在下一帧图像中的位置,并移动追踪窗口位置。结果结合点位移的加权计算有效地克服目标尺度和形状变化产生的问题。结论实验结果表明在目标发生形变或光照变化的情况下,算法也能准确地、实时地追踪到目标。     

Integrated Person Tracking Using Stereo,Color, and Pattern Detection

We present an approach to real-time person tracking in crowded and/or unknown environments using integration of multiple visual modalities. We combine stereo, color, and face detection modules into a single robust system, and show an initial application in an interactive, face-responsive display. Dense, real-time stereo processing is used to isolate users from other objects and people in the background. Skin-hue classification identifies and tracks likely body parts within the silhouette of a user. Face pattern detection discriminates and localizes the face within the identified body parts. Faces and bodies of users are tracked over several temporal scales: short-term (user stays within the field of view), medium-term (user exits/reenters within minutes), and long term (user returns after hours or days). Short-term tracking is performed using simple region position and size correspondences, while medium and long-term tracking are based on statistics of user appearance. We discuss the failure modes of each individual module, describe our integration method, and report results with the complete system in trials with thousands of users.     

Unsupervised Template Warp Consistency for Implicit Surface Correspondences

Unsupervised template discovery via implicit representation in a category of shapes has recently shown strong performance. At the core, such methods deform input shapes to a common template space which allows establishing correspondences as well as implicit representation of the shapes. In this work we investigate the inherent assumption that the implicit neural field optimization naturally leads to consistently warped shapes, thus providing both good shape reconstruction and correspondences. Contrary to this convenient assumption, in practice we observe that such is not the case, consequently resulting in sub-optimal point correspondences. In order to solve the problem, we re-visit the warp design and more importantly introduce explicit constraints using unsupervised sparse point predictions, directly encouraging consistency of the warped shapes. We use the unsupervised sparse keypoints in order to further condition the deformation warp and enforce the consistency of the deformation warp. Experiments in dynamic non-rigid DFaust and ShapeNet categories show that our problem identification and solution provide the new state-of-the-art in unsupervised dense correspondences.     

一种健壮的超像素跟踪算法

在目标跟踪中,传统的超像素跟踪算法在发生遮挡等情况后,会将非目标超像素标记为目标加入到特征空间. 在对候选样本置信度计算中,利用特征空间中最近邻超像素来划定样本中超像素的簇归属会产生错误;而依据的近邻超像素数量过多时,又会造成分类误差的积累. 为解决上述问题,本文提出一种健壮的超像素跟踪算法. 本算法以贝叶斯算法为框架,首先,将前几帧进行超像素切割,提取特征并使用均值漂移聚类算法和基于超像素的外观表示模型进行分类和计算类置信度,放入特征空间中. 其次,根据接下来几帧的平均中心误差确定最佳近邻数目. 最后,在跟踪过程中,对获取帧的指定区域进行超像素切割,提取特征、进行软分类和计算置信度;根据上一帧目标位置进行高斯采样,累加样本内超像素置信度,获得样本置信度;在发生严重遮挡时,不进行滑动窗口更新和外观模型修改,使用当前模型继续跟踪. 与传统的最近邻超像素算法相比,本算法能够有效提升跟踪成功率和降低平均中心误差.     

自适应紧致特征的超像素目标跟踪

目的 针对现有的超像素目标跟踪算法（RST）对同一类中分别属于目标和相似干扰物体的超像素块赋予相同特征置信度,导致难以区分目标和相似干扰物的问题,为此提出自适应紧致特征的超像素目标跟踪算法（ACFST）。方法 该方法在每帧的目标搜索区域内构建适合目标大小的自适应紧致搜索区域,并将该区域内外的特征置信度分别保持不变和降低。处于背景中的相似干扰物体会被该方法划分到紧致搜索区域外,其特征置信度被降低。当依据贝叶斯推理框架求出对应最大后验概率的目标时,紧致搜索区域外的特征置信度低,干扰物体归属目标的程度也低,不会被误判为目标。结果 在具有与目标相似干扰物体的两个视频集进行测试,本文ACFST跟踪算法与RST跟踪算法相比,平均中心误差分别缩减到5.4像素和7.5像素,成功率均提高了11%,精确率分别提高了10.6%和21.6%,使得跟踪结果更精确。结论 本文提出构建自适应紧致搜索区域,并通过设置自适应的参数控制紧致搜索区域变化,减少因干扰物体与目标之间相似而带来的误判。在具有相似物体干扰物的视频集上验证了本文算法的有效性,实验结果表明,本文算法在相似干扰物体靠近或与目标部分重叠时,能够保证算法精确地跟踪到目标,提高算法的跟踪精度,具有较强的鲁棒性,使得算法更能适应背景杂乱、目标遮挡、形变等复杂环境。     

基于分层置信度传播的光流估计方法

置信度传播算法作为一种有效的寻找图像间对应点的方法,近年来被广泛应用于光流估计.但是在估计大位移高精度光流时,将置信度传播直接应用于原图像会导致标签空间过大和处理时间过长的问题.为了克服这个缺点,我们提出了一种基于分层置信度传播的算法来估计高精度大位移光流.本文方法将输入图像视作马尔科夫随机场,为了提高效率,在超像素和像素两个层面上执行置信度传播.我们将超像素层得到的基础位移结果作为粗略的位移参考值,可以有效地减小像素层置信度传播的标签空间,并在有限的标签空间内得到高精度的光流估计结果.MPI Sintel光流数据集上的实验结果显示本文提出的方法在精度和速度上都取得了较好的结果.     

A loop-consistency measure for dense correspondences in multi-view video

Many applications in computer vision and computer graphics require dense correspondences between images of multi-view video streams. Most state-of-the-art algorithms estimate correspondences by considering pairs of images. However, in multi-view videos, several images capture nearly the same scene. In this article we show that this redundancy can be exploited to estimate more robust and consistent correspondence fields. We use the multi-video data structure to establish a confidence measure based on the consistency of the correspondences in a loop of three images. This confidence measure can be applied after flow estimation is terminated to find the pixels for which the estimate is reliable. However, including the measure directly into the estimation process yields dense and highly accurate correspondence fields. Additionally, application of the loop consistency confidence measure allows us to include sparse feature matches directly into the dense optical flow estimation. With the confidence measure, spurious matches can be successfully suppressed during optical flow estimation while correct matches contribute to increase the accuracy of the flow.     

Robust visual tracking via bag of superpixels

The Bag of Words (BoW) model is one of the most popular and effective image representation methods and has been drawn increasing interest in computer vision filed. However, little attention is paid on it in visual tracking. In this paper, a visual tracking method based on Bag of Superpixels (BoS) is proposed. In BoS, the training samples are oversegmented to generate enough superpixel patches. Then K-means algorithm is performed on the collected patches to form visual words of the target and a superpixel codebook is constructed. Finally the tracking is accomplished via searching for the highest likelihood between candidates and codebooks within Bayesian inference framework. In this process, an effective updating scheme is adopted to help our tracker resist occlusions and deformations. Experimental results demonstrate that the proposed method outperforms several state-of-the-art trackers.     

基于超像素联接权模型的视频分割算法

视频图像分割是图像处理领域的一个热门问题。在传统分割算法的基础上,提出了一种新的无监督的视频分割算法。该算法采用超像素对运动前景进行表示,定义联接权概念来描述超像素属于同一物体的可能性,并利用当前帧的静态特征与前后帧的关联特征进行联接权计算。为优化超像素间匹配关系的搜索,算法引入了超像素颜色特征匹配约束与运动关联性匹配约束的机制。分别在简单场景和复杂场景进行了视频分割实验,简单场景下,算法保证了较高的召回率与稳定的准确率;复杂场景下,算法完成了人群中单个人的切分。大量实验结果表明,该算法能够实现视频图像的分割,并且能有效解决过分割问题。     

Guided Sampling via Weak Motion Models and Outlier Sample Generation for Epipolar Geometry Estimation

The problem of automatic robust estimation of the epipolar geometry in cases where the correspondences are contaminated with a high percentage of outliers is addressed. This situation often occurs when the images have undergone a significant deformation, either due to large rotation or wide baseline of the cameras. An accelerated algorithm for the identification of the false matches between the views is presented. The algorithm generates a set of weak motion models (WMMs). Each WMM roughly approximates the motion of correspondences from one image to the other. The algorithm represents the distribution of the median of the geometric distances of a correspondence to the WMMs as a mixture model of outlier correspondences and inlier correspondences. The algorithm generates a sample of outlier correspondences from the data. This sample is used to estimate the outlier rate and to estimate the outlier pdf. Using these two pdfs the probability that each correspondence is an inlier is estimated. These probabilities enable guided sampling. In the RANSAC process this guided sampling accelerates the search process. The resulting algorithm when tested on real images achieves a speedup of between one or two orders of magnitude. This work was supported partly by grant 01-99-08430 of the Israeli Space Agency through the Ministry of Science Culture and Sports of Israel.     

Feature correspondence and motion recovery in vehicle planar navigation

This paper describes a strategy to feature point correspondence and motion recovery in vehicle navigation. A transformation of the image plane is proposed that keeps the motion of the vehicle on a plane parallel to the transformed image plane. This permits to define linear tracking filters to estimate the real-world positions of the features, and allows us to select the matches that accomplish the rigidity of the scene by a Hough transform. Candidate correspondences are selected by similarity, taking into account the smoothness of motion. Further processing brings out the final matching. The methods have been tested in a real application.     

Unsupervised Learning for Graph Matching

Graph matching is an essential problem in computer vision that has been successfully applied to 2D and 3D feature matching and object recognition. Despite its importance, little has been published on learning the parameters that control graph matching, even though learning has been shown to be vital for improving the matching rate. In this paper we show how to perform parameter learning in an unsupervised fashion, that is when no correct correspondences between graphs are given during training. Our experiments reveal that unsupervised learning compares favorably to the supervised case, both in terms of efficiency and quality, while avoiding the tedious manual labeling of ground truth correspondences. We verify experimentally that our learning method can improve the performance of several state-of-the art graph matching algorithms. We also show that a similar method can be successfully applied to parameter learning for graphical models and demonstrate its effectiveness empirically.     

Foreground Focus: Unsupervised Learning from Partially Matching Images

We present a method to automatically discover meaningful features in unlabeled image collections. Each image is decomposed into semi-local features that describe neighborhood appearance and geometry. The goal is to determine for each image which of these parts are most relevant, given the image content in the remainder of the collection. Our method first computes an initial image-level grouping based on feature correspondences, and then iteratively refines cluster assignments based on the evolving intra-cluster pattern of local matches. As a result, the significance attributed to each feature influences an image’s cluster membership, while related images in a cluster affect the estimated significance of their features. We show that this mutual reinforcement of object-level and feature-level similarity improves unsupervised image clustering, and apply the technique to automatically discover categories and foreground regions in images from benchmark datasets.     

SoftPOSIT: Simultaneous Pose and Correspondence Determination

The problem of pose estimation arises in many areas of computer vision, including object recognition, object tracking, site inspection and updating, and autonomous navigation when scene models are available. We present a new algorithm, called SoftPOSIT, for determining the pose of a 3D object from a single 2D image when correspondences between object points and image points are not known. The algorithm combines the iterative softassign algorithm (Gold and Rangarajan, 1996; Gold et al., 1998) for computing correspondences and the iterative POSIT algorithm (DeMenthon and Davis, 1995) for computing object pose under a full-perspective camera model. Our algorithm, unlike most previous algorithms for pose determination, does not have to hypothesize small sets of matches and then verify the remaining image points. Instead, all possible matches are treated identically throughout the search for an optimal pose. The performance of the algorithm is extensively evaluated in Monte Carlo simulations on synthetic data under a variety of levels of clutter, occlusion, and image noise. These tests show that the algorithm performs well in a variety of difficult scenarios, and empirical evidence suggests that the algorithm has an asymptotic run-time complexity that is better than previous methods by a factor of the number of image points. The algorithm is being applied to a number of practical autonomous vehicle navigation problems including the registration of 3D architectural models of a city to images, and the docking of small robots onto larger robots.     

Superpixel-based active contour model for unsupervised change detection from satellite images

This study proposes a superpixel-based active contour model (SACM) for unsupervised change detection from satellite images. The accuracy of change detection produced by the traditional active contour model suffers from the trade-off parameter. The SACM is designed to address this limitation through the incorporation of the spatial and statistical information of superpixels. The proposed method mainly consists of three steps. First, the difference image is created with change vector analysis method from two temporal satellite images. Second, statistical region merging method is applied on the difference image to produce a superpixel map. Finally, SACM is designed based on the superpixel map to detect changes from the difference image. The SACM incorporates spatial and statistical information and retains the accurate shapes and outlines of superpixels. Experiments were conducted on two data sets, namely Landsat-7 Enhanced Thematic Mapper Plus and SPOT 5, to validate the proposed method. Experimental results show that SACM reduces the effects of the trade-off parameter. The proposed method also increases the robustness of the traditional active contour model for input parameters and improves its effectiveness. In summary, SACM often outperforms some existing methods and provides an effective unsupervised change detection method.