一种动态场景下基于时空信息的视频对象提取算法

在实际应用中,许多视频序列具有运动背景,使得从其中提取视频对象变得复杂,为此提出了一种基于运动估计和图形金字塔的动态场景下的视频对象提取算法。该算法首先引入了相位相关法求取运动向量,因避免了视频序列中光照变化的影响,故可提高效率和稳健性;接着再根据参数模型进行全局运动估计来得到最终运动模板;然后利用图形金字塔算法对当前模板内图像区域进行空间分割,最终提取出语义视频对象。与现有算法相比,对于从具有动态场景的视频流中提取运动对象的情况,由于使用该算法能有效地避开精准背景补偿,因而不仅节省了计算量,而且提取出来的语义对象精度较高。实验表明,无论是对动态场景中刚性还是非刚性运动物体的分割,该算法都具有较好的效果。     

Fast and accurate global motion compensation

Video understanding has attracted significant research attention in recent years, motivated by interest in video surveillance, rich media retrieval and vision-based gesture interfaces. Typical methods focus on analyzing both the appearance and motion of objects in video. However, the apparent motion induced by a moving camera can dominate the observed motion, requiring sophisticated methods for compensating for camera motion without a priori knowledge of scene characteristics. This paper introduces two new methods for global motion compensation that are both significantly faster and more accurate than state of the art approaches. The first employs RANSAC to robustly estimate global scene motion even when the scene contains significant object motion. Unlike typical RANSAC-based motion estimation work, we apply RANSAC not to the motion of tracked features but rather to a number of segments of image projections. The key insight of the second method involves reliably classifying salient points into foreground and background, based upon the entropy of a motion inconsistency measure. Extensive experiments on established datasets demonstrate that the second approach is able to remove camera-based observed motion almost completely while still preserving foreground motion.     

基于运动显著性的移动镜头下的运动目标检测

针对移动镜头下的运动目标检测中的背景建模复杂、计算量大等问题,提出一种基于运动显著性的移动镜头下的运动目标检测方法,在避免复杂的背景建模的同时实现准确的运动目标检测。该方法通过模拟人类视觉系统的注意机制,分析相机平动时场景中背景和前景的运动特点,计算视频场景的显著性,实现动态场景中运动目标检测。首先,采用光流法提取目标的运动特征,用二维高斯卷积方法抑制背景的运动纹理;然后采用直方图统计衡量运动特征的全局显著性,根据得到的运动显著图提取前景与背景的颜色信息;最后,结合贝叶斯方法对运动显著图进行处理,得到显著运动目标。通用数据库视频上的实验结果表明,所提方法能够在抑制背景运动噪声的同时,突出并准确地检测出场景中的运动目标。     

Optimal Image and Video Closure by Superpixel Grouping

Detecting independent objects in images and videos is an important perceptual grouping problem. One common perceptual grouping cue that can facilitate this objective is the cue of contour closure, reflecting the spatial coherence of objects in the world and their projections as closed boundaries separating figure from background. Detecting contour closure in images consists of finding a cycle of disconnected contour fragments that separates an object from its background. Searching the entire space of possible groupings is intractable, and previous approaches have adopted powerful perceptual grouping heuristics, such as proximity and co-curvilinearity, to constrain the search. We introduce a new formulation of the problem, by transforming the problem of finding cycles of contour fragments to finding subsets of superpixels whose collective boundary has strong edge support (few gaps) in the image. Our cost function, a ratio of a boundary gap measure to area, promotes spatially coherent sets of superpixels. Moreover, its properties support a global optimization procedure based on parametric maxflow. Extending closure detection to videos, we introduce the concept of spatiotemporal closure. Analogous to image closure, we formulate our spatiotemporal closure cost over a graph of spatiotemporal superpixels. Our cost function is a ratio of motion and appearance discontinuity measures on the boundary of the selection to an internal homogeneity measure of the selected spatiotemporal volume. The resulting approach automatically recovers coherent components in images and videos, corresponding to objects, object parts, and objects with surrounding context, providing a good set of multiscale hypotheses for high-level scene analysis. We evaluate both our image and video closure frameworks by comparing them to other closure detection approaches, and find that they yield improved performance.     

Image classification without segmentation using a hybrid pyramid kernel

Image classification usually requires complicated segmentation to separate foreground objects from the background scene. However, the statistical content of a background scene can actually provide very useful information for classification. In this paper, we propose a new hybrid pyramid kernel which incorporates local features extracted from both dense regular grids and interest points for image classification, without requiring segmentation. Features extracted from dense regular grids can better capture information about the background scene, while interest points detected at corners and edges can better capture information about the salient objects. In our algorithm, these two local features are combined in both the spatial and the feature-space domains, and are organized into pyramid representations. In order to obtain better classification accuracy, we fine-tune the parameters involved in the similarity measure, and we determine discriminative regions by means of relevance feedback. From the experimental results, we observe that our algorithm can achieve a 6.37 % increase in performance as compared to other pyramid-representation-based methods. To evaluate the applicability of the proposed hybrid kernel to large-scale databases, we have performed a cross-dataset experiment and investigated the effect of foreground/background features on each of the kernels. In particular, the proposed hybrid kernel has been proven to satisfy Mercer’s condition and is efficient in measuring the similarity between image features. For instance, the computational complexity of the proposed hybrid kernel is proportional to the number of features.     

基于前景增强与背景抑制的显著性物体检测

显著性物体检测的关键在于准确地突出前景区域,多数传统方法在处理复杂背景图像时效果不理想。针对上述问题,提出了一种基于前景增强与背景抑制的显著性物体检测方法。首先,利用简单线性迭代聚类(SLIC)将图像进行分割得到多个超像素区域,通过区域间的对比和边界信息分别获得图像的显著区域与背景种子,并通过计算得到基于区域间对比和基于背景的两幅显著图。然后,在两幅图像中运用Seam Carving和Graph based的图像分割法区分显著与非显著区域,进而得到前景增强与背景抑制模板。最终,融合两幅显著图与模板得到最终的显著图。在公开数据集MSRA 1000上对算法进行验证,结果表明,所提算法与7种主流算法相比具有更好的查准率和查全率。     

面向多种场景的视频对象自动分割算法

针对当前应用于视频对象分割的图割方法容易在复杂环境、镜头移动、光照不稳定等场景下鲁棒性不佳的问题,提出了结合光流和图割的视频对象分割算法.主要思路是通过分析前景对象的运动信息,得到单帧图像上前景区域的先验知识,从而改善分割结果.论文首先通过光流场采集视频中动作信息,并提取出前景对象先验区域,然后结合前景和背景先验区域建立图割模型,实现前景对象分割.最后为提高算法在不同场景下的鲁棒性,本文改进了传统的测地显著性模型,并基于视频本征的时域平滑性,提出了基于混合高斯模型的动态位置模型优化机制.在两个标准数据集上的实验结果表明,所提算法与当前其他视频对象分割算法相比,降低了分割结果的错误率,有效提高了在多种场景下的鲁棒性.     

一种动态场景下运动对象分割新算法

视频运动对象分割是计算机视觉和视频处理的基本问题。在摄像机存在全局运动的动态场景下,准确分割运动对象依然是难点和热点问题。本文提出一种基于全局运动补偿和核密度检测的动态场景下视频运动对象分割算法。首先,提出匹配加权的全局运动估计补偿算法,消除动态场景下背景运动对运动对象分割的影响;其次,采用非参数核密度估计方法分别估计各像素属于前景与背景的概率密度,通过比较属于前景和属于背景的概率及形态学处理得到运动对象分割结果。实验结果证明,该方法实现简单,有效地提高了动态场景下运动对象分割的准确性。     

基于背景配准的矿井危险区域视频目标检测算法

针对矿井危险区域视频监控视场背景复杂,难以实现视频目标精确提取的问题,提出了一种基于背景配准的视频目标检测算法。该算法实现步骤:提取SIFT特征点,计算特征点区域H-S光流矢量;通过区域运动特性分析提取出背景运动区域,对背景运动区域特征点做帧间匹配;计算仿射参数,配准差分后提取出精确的目标区域。实验结果表明,该算法能够去除前景目标特征点对背景配准的影响,可获得较为精确的目标区域。     

联合聚焦度和传播机制的光场图像显著性检测

目的 图像显著性检测方法对前景与背景颜色、纹理相似或背景杂乱的场景,存在背景难抑制、检测对象不完整、边缘模糊以及方块效应等问题。光场图像具有重聚焦能力,能提供聚焦度线索,有效区分图像前景和背景区域,从而提高显著性检测的精度。因此,提出一种基于聚焦度和传播机制的光场图像显著性检测方法。方法 使用高斯滤波器对焦堆栈图像的聚焦度信息进行衡量,确定前景图像和背景图像。利用背景图像的聚焦度信息和空间位置构建前/背景概率函数,并引导光场图像特征进行显著性检测,以提高显著图的准确率。另外,充分利用邻近超像素的空间一致性,采用基于K近邻法（K-nearest neighbor,K-NN）的图模型显著性传播机制进一步优化显著图,均匀地突出整个显著区域,从而得到更加精确的显著图。结果 在光场图像基准数据集上进行显著性检测实验,对比3种主流的传统光场图像显著性检测方法及两种深度学习方法,本文方法生成的显著图可以有效抑制背景区域,均匀地突出整个显著对象,边缘也更加清晰,更符合人眼视觉感知。查准率达到85.16%,高于对比方法,F度量（F-measure）和平均绝对误差（mean absolute error,MAE）分别为72.79%和13.49%,优于传统的光场图像显著性检测方法。结论 本文基于聚焦度和传播机制提出的光场图像显著性模型,在前/背景相似或杂乱背景的场景中可以均匀地突出显著区域,更好地抑制背景区域。     

结合全卷积网络的无监督视频目标分割

对视频中的目标进行像素级分割是计算机视觉领域的研究热点，完全没有用户标注的无监督视频分割对分割算法提出了更高的要求。近几年在分割中常使用基于帧间运动信息进行建模的方法，即用光流等运动信息预测目标轮廓，再结合颜色等特征建立模型进行分割。针对这些方法产生的前景背景混淆以及边缘粗糙等问题，本文提出结合全卷积网络的视频目标分割方法。首先通过全卷积网络预测视频序列中显著目标的轮廓，结合光流获得的运动显著性标签进行修正，然后建立时间-空间图模型，运用图割的方法获得最终的预测标签。在SegTrack v2以及DAVIS这2个通用数据集上进行评估，结果表明本文方法较基于帧间运动信息的方法在分割效果上有明显的提高。     

Realistic action recognition with salient foreground trajectories

Human action recognition has great potential in many applications relevant to artificial intelligence, which can accelerate some research on expert and intelligent systems, such as feature selection. To improve the performance on human action recognition in realistic scenarios, a novel Salient Foreground Trajectory extraction method based on saliency detection and low-rank matrix recovery is proposed to learn the discriminative features from complicated video context. Specifically, a new trajectory saliency combining appearance saliency and motion saliency is proposed to divide the dense trajectories into salient trajectories and non-salient ones. The salient trajectories are approximately corresponding to the interested foreground region, while the non-salient subset is mainly composed of the dominating background trajectories. Furthermore, according to the low rank property of background motion, if the video has background motion, the background trajectory subspace is further constructed on the non-salient trajectory subset via low-rank matrix recovery method. Then the possible background trajectories in the salient subset could be subtracted. Finally, the resulting salient foreground trajectory features are encoded by the approach of Bag of Features or Fisher Vector for action classification. Experiments on KTH, UCF Sports and Olympic Sports have shown that the proposed Salient Foreground Trajectory method is effective and achieves comparable results to the state of the art.     

自适应混合高斯背景模型的改进

对自适应混合高斯背景模型进行了改进，将背景重构和前景消融时间控制机制整合到传统自适应混合高斯背景模型中，以提高运动分割的质量。背景重构算法从含有运动物体的动态场景视频序列中重构静态背景图像，然后用重构的静态背景图像初始化自适应混合高斯背景模型；而前景消融时间控制机制则使运动物体停止时的前景消融时间独立于背景模型的学习速率，从而可以根据需要调节前景消融的持续时间。实验结果表明了算法的有效性。     

Multilayer background modeling under occlusions

A multilayer background modeling technique is presented for video surveillance. Rather than simply classifying all features in a scene as either dynamically moving foreground or long-lasting, stationary background, a temporal model is used to place each scene object in time relative to each other. Foreground objects that become stationary are registered as layers on top of the background layer. In this process of layer formation, the algorithm deals with ”fake objects” created by moved background, and noise created by dynamic background and moving foreground objects. Objects that leave the scene are removed based on the occlusion reasoning among layers. The technique allows us to understand and visualize a scene with multiple objects entering, leaving, and occluding each other at different points in time. This scene understanding leads to a richer representation of temporal scene events than traditional foreground/background segmentation. The technique builds on a low-cost background modeling technique that makes it suitable for embedded, real-time platforms.     

相机抖动场景中数据驱动的背景图像更新算法

视频序列中的运动目标检测是计算机视觉领域的重要研究课题.背景减除是运动目标检测的有效方法,但相机抖动会对背景提取带来极大干扰,从而可能造成传统基于模型的图像处理方法模型失真.本文提出了相机抖动场景下前景图像提取的数据驱动背景图像更新控制算法.首先利用Harris特征检测进行背景补偿以消除抖动干扰.然后利用无模型自适应控制方法,建立单入单出控制系统来表示背景图像并进行实时更新.最后运用背景减除法提取运动目标前景图像.本文方法与传统基于模型方法进行了不同视频序列的对比仿真.实验结果表明,本文方法可以有效处理相机抖动场景下的运动目标检测问题,目标前景图像分离效果更加接近真实场景.     

基于背景差分的核密度估计前景检测方法

针对非参数核密度估计算法前景检测不够精确、运算量大的问题,提出了一种基于背景差分图像的核密度估计前景检测方法。该方法结合了单高斯模型和核密度估计模型进行初始背景建模,利用背景差分图像,过滤掉非动态背景区域,对动态背景区域采用核密度估计进行像素分类。同时,对非动态背景区域,采用渐进式更新;对动态背景区域,采用非参数核密度估计进行更新。实验结果表明,该算法能够精确地分割出前景目标,减少了误检噪声,降低了运算量。     

Layered Representation of a Video Shot with Mosaicing

: This paper presents a motion segmentation method useful for representing efficiently a video shot as a static mosaic of the background plus sequences of the objects moving in the foreground. This generates an MPEG-4 compliant, layered representation useful for video coding, editing and indexing. First, a mosaic of the static background is computed by estimating the dominant motion of the scene. This is achieved by tracking features over the video sequence and using a robust technique that discards features attached to the moving objects. The moving objects get removed in the final mosaic by computing the median of the grey levels. Then, segmentation is obtained by taking the pixelwise difference between each frame of the original sequence and the mosaic of the background. To discriminate between the moving object and noise, temporal coherence is exploited by tracking the object in the binarised difference image sequence. The automatic computation of the mosaic and the segmentation procedure are illustrated with real sequences experiments. Examples of coding and content-based manipulation are also shown. Received: 31 August 2000, Received in revised form: 18 April 2001, Accepted: 20 July 2001     

结合l1/2范数与显著性约束的背景减除

传统背景减除模型在背景静止和前景对象移动较快时提取到的前景效果较好,但当背景变化或前景对象移动缓慢时容易将动态背景误判为前景或检测出的前景有较多空洞。针对传统背景减除模型在动态背景和前景对象移动缓慢条件下存在前景检测精度下降的问题,提出一种基于l_1/2范数与显著性约束的背景减除模型。将观测数据分为低秩背景、运动前景和动态干扰3类,利用l_1/2范数约束运动前景加强前景稀疏性,有效抑制动态背景对前景提取造成的干扰,提高运动前景在动态背景中的检测精度。引入视频每一帧的显著性约束,通过对每一帧图像进行低秩稀疏分解来检测移动缓慢的目标。实验结果表明,该模型对于复杂场景具有较强的适应能力,可有效去除动态背景对前景的干扰,快速检测出移动缓慢的前景对象,相比于l_1/1/2-RPCA背景减除模型的平均查全率、查准率和调和平均值分别提升了9、14和10个百分点。     

Video-based salient object detection via spatio-temporal difference and coherence

Salient object detection aims to extract the attractive objects in images and videos. It can support various robotics tasks and multimedia applications, such as object detection, action recognition and scene analysis. However, efficient detection of salient objects in videos still faces many challenges as compared to that in still images. In this paper, we propose a novel video-based salient object detection method by exploring spatio-temporal characteristics of video content, i.e., spatial-temporal difference and spatial-temporal coherence. First, we initialize the saliency map for each keyframe by deriving spatial-temporal difference from color cue and motion cue. Next, we generate the saliency maps of other frames by propagating the saliency intra and inter frames with the constraint of spatio-temporal coherence. Finally, the saliency maps of both keyframes and non-keyframes are refined in the saliency propagation. In this way, we can detect salient objects in videos efficiently by exploring their spatio-temporal characteristics. We evaluate the proposed method on two public datasets, named SegTrackV2 and UVSD. The experimental results show that our method outperforms the state-of-the-art methods when taking account of both effectiveness and efficiency.