结合核密度估计和边缘信息的运动对象分割算法

针对前景与背景具有相似颜色时的运动对象分割问题,提出一种结合核密度估计和边缘信息的分割算法.在前景和背景建模阶段使用颜色信息的基础上,引入边缘信息来构造前景和背景的概率模型;然后在马尔可夫随机场框架下引入与概率模型有关的似然能量项,以及反映空域连续性和时域一致性的能量项,并利用图切割方法来获得可靠的运动对象分割结果.实验结果证明,对于前景与背景具有相似颜色的视频序列,该算法降低了对象分割误差,显著地提高了整个序列中对象分割的鲁棒性.     

基于时空分析的视频前景提取

为了从包含动态背景或者非平移运动前景的视频中提取完整的前景区域,提出一种视频分割算法。首先,将视频中单个像素的变化过程视为离散时间信号,运用时间轴的Gabor滤波对时域信息进行分析,将视频粗分为前景和背景;然后,运用均值漂移算法对前景和背景做颜色聚类分析,分析空域的颜色关联信息,分别建立全局颜色模型和局部颜色模型;最后,运用双重标记法提取视频前景。该算法综合考虑视频的时域信息和空域信息。在多个视频库的测试结果表明,该算法可以显著提高前景区域提取的精度,特别是对于背景动态变化或者前景发生非平移运动的视频。     

Foreground prediction for bilayer segmentation of videos

Given a segmentation result (an alpha matte or a binary mask) of the former frame, foreground prediction is a process of estimating the probability that each pixel in the current frame belongs to the foreground. It plays a very important role in bilayer segmentation of videos, especially videos with non-static backgrounds. In this paper, a new foreground prediction algorithm which is called opacity propagation is proposed. It can propagate the opacity values of the former frame to the current frame by minimizing a cost function that is constructed by assuming the spatiotemporally local color smoothness of the video. Optical flow and probability density estimation based on a local color model are employed to find the corresponding pixels of two adjacent frames. An OPSIC (opacity propagation with sudden illumination changes) algorithm is also proposed which is an improvement of our proposed opacity propagation algorithm because it adds a simple color transformation model. As far as we know, this is the first algorithm that can predict the foreground accurately when the illumination changes suddenly. The opacity map (OM) generated by the opacity propagation algorithm is usually more accurate than the previously used probability map (PM). The experiments demonstrate the effectiveness of our algorithm.     

Robust foreground object segmentation from handheld camera videos with occlusion map

Extracting foreground objects from videos captured by a handheld camera has emerged as a new challenge. While existing approaches aim to exploit several clues such as depth and motion to extract the foreground layer, there are limitations in handling partial movement and cast shadow. In this paper, we bring a novel perspective to address these two issues by utilizing occlusion map introduced by object and camera motion and taking the advantage of interactive image segmentation methods. For partial movement, we treat each video frame as an image and synthesize “seeding” user interactions (i.e., user manually marking foreground and background) with both forward and backward occlusion maps to leverage the advances in high quality interactive image segmentation. For cast shadow, we utilize a paired region based shadow detection method to further refine initial segmentation results by removing detected shadow regions. Experimental results from both qualitative evaluation and quantitative evaluation on the Hopkins dataset demonstrate both the effectiveness and the efficiency of our proposed approach.     

融合双目多维感知特征的立体视频显著性检测

目的 立体视频能提供身临其境的逼真感而越来越受到人们的喜爱,而视觉显著性检测可以自动预测、定位和挖掘重要视觉信息,可以帮助机器对海量多媒体信息进行有效筛选。为了提高立体视频中的显著区域检测性能,提出了一种融合双目多维感知特性的立体视频显著性检测模型。方法 从立体视频的空域、深度以及时域3个不同维度出发进行显著性计算。首先,基于图像的空间特征利用贝叶斯模型计算2D图像显著图;接着,根据双目感知特征获取立体视频图像的深度显著图;然后,利用Lucas-Kanade光流法计算帧间局部区域的运动特征,获取时域显著图;最后,将3种不同维度的显著图采用一种基于全局-区域差异度大小的融合方法进行相互融合,获得最终的立体视频显著区域分布模型。结果 在不同类型的立体视频序列中的实验结果表明,本文模型获得了80%的准确率和72%的召回率,且保持了相对较低的计算复杂度,优于现有的显著性检测模型。结论 本文的显著性检测模型能有效地获取立体视频中的显著区域,可应用于立体视频/图像编码、立体视频/图像质量评价等领域。     

A dynamic conditional random field model for foreground and shadow segmentation

This paper proposes a dynamic conditional random field (DCRF) model for foreground object and moving shadow segmentation in indoor video scenes. Given an image sequence, temporal dependencies of consecutive segmentation fields and spatial dependencies within each segmentation field are unified by a dynamic probabilistic framework based on the conditional random field (CRF). An efficient approximate filtering algorithm is derived for the DCRF model to recursively estimate the segmentation field from the history of observed images. The foreground and shadow segmentation method integrates both intensity and gradient features. Moreover, models of background, shadow, and gradient information are updated adaptively for nonstationary background processes. Experimental results show that the proposed approach can accurately detect moving objects and their cast shadows even in monocular grayscale video sequences.     

Prior-Free Dependent Motion Segmentation Using Helmholtz-Hodge Decomposition Based Object-Motion Oriented Map

Motion segmentation in moving camera videos is a very challenging task because of the motion dependence between the camera and moving objects. Camera motion compensation is recognized as an effective approach. However, existing work depends on prior-knowledge on the camera motion and scene structure for model selection. This is not always available in practice. Moreover, the image plane motion suffers from depth variations, which leads to depth-dependent motion segmentation in 3D scenes. To solve these problems, this paper develops a prior-free dependent motion segmentation algorithm by introducing a modified Helmholtz-Hodge decomposition (HHD) based object-motion oriented map (OOM). By decomposing the image motion (optical flow) into a curl-free and a divergence-free component, all kinds of camera-induced image motions can be represented by these two components in an invariant way. HHD identifies the camera-induced image motion as one segment irrespective of depth variations with the help of OOM. To segment object motions from the scene, we deploy a novel spatio-temporal constrained quadtree labeling. Extensive experimental results on benchmarks demonstrate that our method improves the performance of the state-of-the-art by 10%~20% even over challenging scenes with complex background.     

Semantic Plane-Structure based motion detection with a nonstationary camera

This paper presents a novel method to accurately detect moving objects from a video sequence captured using a nonstationary camera. Although common methods provide effective motion detection for static backgrounds or through only planar-perspective transformation, many detection errors occur when the background contains complex dynamic interferences or the camera undergoes unknown motions. To solve this problem, this study proposed a motion detection method that incorporates temporal motion and spatial structure. In the proposed method, first, spatial semantic planes are segmented, and image registration based on stable background planes is applied to overcome the interferences of the foreground and dynamic background. Thus, the estimated dense temporal motion ensures that small moving objects are not missed. Second, motion pixels are mapped on semantic planes, and then, the spatial distribution constraints of motion pixels, regional shapes and plane semantics, which are integrated into a planar structure, are used to minimise false positives. Finally, based on the dense temporal motion and spatial structure, moving objects are accurately detected. The experimental results on CDnet dataset, Pbi dataset, Aeroscapes dataset, and other challenging self-captured videos under difficult conditions, such as fast camera movement, large zoom variation, video jitters, and dynamic background, revealed that the proposed method can remove background movements, dynamic interferences, and marginal noises and can effectively obtain complete moving objects.© 2017 ElsevierInc.Allrightsreserved.     

Real-time video object segmentation in H.264 compressed domain

In this study the authors proposed a real-time video object segmentation algorithm that works in the H.264 compressed domain. The algorithm utilises the motion information from the H.264 compressed bit stream to identify background motion model and moving objects. In order to preserve spatial and temporal continuity of objects, Markov random field (MRF) is used to model the foreground field. Quantised transform coefficients of the residual frame are also used to improve segmentation result. Experimental results show that the proposed algorithm can effectively extract moving objects from different kinds of sequences. The computation time of the segmentation process is merely about 16 ms per frame for CIF size frame, allowing the algorithm to be applied in real-time applications.     

Video frame rate up conversion under inconsistent camera motion

In this paper, we address the problem of video frame rate up-conversion (FRC) in the compressed domain. FRC is often recognized as video temporal interpolation. This problem is very challenging when targeted for video sequences with inconsistent camera and object motion, such as sports videos. A novel compressed domain motion compensation scheme is presented and applied in this paper, aiming at up-sampling frame rates in sports videos. MPEG-2 encoded motion vectors (MVs) are utilized as inputs in the proposed algorithm. The decoded MVs undergo a cumulative spatiotemporal interpolation. An iterative rejection scheme based on the dense motion vector field (MVF) and the generalized affine motion model is exploited to detect global camera motion. Subsequently, the foreground object separation is performed by additionally examining the temporal consistency of the output of iterative rejections. This consistency check process helps coalesce the resulting foreground blocks and weed out the unqualified blocks. Finally, different compensation strategies for the camera and object motions are applied to interpolate the new frames. Illustrative examples are provided to demonstrate the efficacy of the proposed approach. Experimental results are compared with the popular block and non-block based frame interpolation approaches.

Density-based multifeature background subtraction with support vector machine

Background modeling and subtraction is a natural technique for object detection in videos captured by a static camera, and also a critical preprocessing step in various high-level computer vision applications. However, there have not been many studies concerning useful features and binary segmentation algorithms for this problem. We propose a pixelwise background modeling and subtraction technique using multiple features, where generative and discriminative techniques are combined for classification. In our algorithm, color, gradient, and Haar-like features are integrated to handle spatio-temporal variations for each pixel. A pixelwise generative background model is obtained for each feature efficiently and effectively by Kernel Density Approximation (KDA). Background subtraction is performed in a discriminative manner using a Support Vector Machine (SVM) over background likelihood vectors for a set of features. The proposed algorithm is robust to shadow, illumination changes, spatial variations of background. We compare the performance of the algorithm with other density-based methods using several different feature combinations and modeling techniques, both quantitatively and qualitatively.     

Background subtraction by combining Temporal and Spatio-Temporal histograms in the presence of camera movement

Background subtraction is the classical approach to differentiate moving objects in a scene from the static background when the camera is fixed. If the fixed camera assumption does not hold, a frame registration step is followed by the background subtraction. However, this registration step cannot perfectly compensate camera motion, thus errors like translations of pixels from their true registered position occur. In this paper, we overcome these errors with a simple, but effective background subtraction algorithm that combines Temporal and Spatio-Temporal approaches. The former models the temporal intensity distribution of each individual pixel. The latter classifies foreground and background pixels, taking into account the intensity distribution of each pixels’ neighborhood. The experimental results show that our algorithm outperforms the state-of-the-art systems in the presence of jitter, in spite of its simplicity.     

Abnormal event detection and localization in crowded scenes based on PCANet

Detecting and localizing abnormal events in crowded scenes still remains a challenging task among computer vision community. An unsupervised framework is proposed in this paper to address the problem. Low-level features and optical flows (OF) of video sequences are extracted to represent motion information in the temporal domain. Moreover, abnormal events usually occur in local regions and are closely linked to their surrounding areas in the spatial domain. To extract high-level information from local regions and model the relationship in spatial domain, the first step is to calculate optical flow maps and divide them into a set of non-overlapping sub-maps. Next, corresponding PCANet models are trained using the sub-maps at same spatial location in the optical flow maps. Based on the block-wise histograms extracted by PCANet models, a set of one-class classifiers are trained to predict the anomaly scores of test frames. The framework is completely unsupervised because it utilizes only normal videos. Experiments were carried out on UCSD Ped2 and UMN datasets, and the results show competitive performance of this framework when compared with other state-of-the-art methods.     

基于背景分类的运动目标检测算法

针对光线暗、对比度和分辨率低的监控视频,提出了一种基于背景分类的运动目标检测算法。 首先用视频第一帧图像HSV空间的色度H和亮度V作为背景特征进行初始化,建立两种包含色度和亮度特征的背景模型类,即初始化得到的原始背景类和受光照或者其他因素影响得到的在原始背景周围波动的背景波动类,利用这两个背景模型进行前景检测和背景更新。为提高前景检测的准确率,背景模型的更正加入背景更正机制和权重机制,使得背景中样本的数量根据背景的实际情况处在一种动态的变化中,提高前景分割的效率。用不同场景下的监控视频进行算法对比实验,结果证明,该算法获得的前景完整清晰,视频处理的速度较快。提出的算法简单实用,对噪声干扰表现出良好的鲁棒性。     

Augmented Panoramic Video

Many video sequences consist of a locally dynamic background containing moving foreground subjects. In this paper we propose a novel way of re‐displaying these sequences, by giving the user control over a virtual camera frame. Based on video mosaicing, we first compute a static high quality background panorama. After segmenting and removing the foreground subjects from the original video, the remaining elements are merged into a dynamic background panorama, which seamlessly extends the original video footage. We then re‐display this augmented video by warping and cropping the panorama. The virtual camera can have an enlarged field‐of‐view and a controlled camera motion. Our technique is able to process videos with complex camera motions, reconstructing high quality panoramas without parallax artefacts, visible seams or blurring, while retaining repetitive dynamic elements.     

Bilayer segmentation of webcam videos using tree-based classifiers

This paper presents an automatic segmentation algorithm for video frames captured by a (monocular) webcam that closely approximates depth segmentation from a stereo camera. The frames are segmented into foreground and background layers that comprise a subject (participant) and other objects and individuals. The algorithm produces correct segmentations even in the presence of large background motion with a nearly stationary foreground. This research makes three key contributions: First, we introduce a novel motion representation, referred to as "motons," inspired by research in object recognition. Second, we propose estimating the segmentation likelihood from the spatial context of motion. The estimation is efficiently learned by random forests. Third, we introduce a general taxonomy of tree-based classifiers that facilitates both theoretical and experimental comparisons of several known classification algorithms and generates new ones. In our bilayer segmentation algorithm, diverse visual cues such as motion, motion context, color, contrast, and spatial priors are fused by means of a conditional random field (CRF) model. Segmentation is then achieved by binary min-cut. Experiments on many sequences of our videochat application demonstrate that our algorithm, which requires no initialization, is effective in a variety of scenes, and the segmentation results are comparable to those obtained by stereo systems.     

PWP3D: Real-Time Segmentation and Tracking of 3D Objects

We formulate a probabilistic framework for simultaneous region-based 2D segmentation and 2D to 3D pose tracking, using a known 3D model. Given such a model, we aim to maximise the discrimination between statistical foreground and background appearance models, via direct optimisation of the 3D pose parameters. The foreground region is delineated by the zero-level-set of a signed distance embedding function, and we define an energy over this region and its immediate background surroundings based on pixel-wise posterior membership probabilities (as opposed to likelihoods). We derive the differentials of this energy with respect to the pose parameters of the 3D object, meaning we can conduct a search for the correct pose using standard gradient-based non-linear minimisation techniques. We propose novel enhancements at the pixel level based on temporal consistency and improved online appearance model adaptation. Furthermore, straightforward extensions of our method lead to multi-camera and multi-object tracking as part of the same framework. The parallel nature of much of the processing in our algorithm means it is amenable to GPU acceleration, and we give details of our real-time implementation, which we use to generate experimental results on both real and artificial video sequences, with a number of 3D models. These experiments demonstrate the benefit of using pixel-wise posteriors rather than likelihoods, and showcase the qualities, such as robustness to occlusions and motion blur (and also some failure modes), of our tracker.     

结合形态学和假设检验的视频对象分割

视频对象分割是当前图像和视频处理的热点和难点之一。文章首先采用形态算子和改进的watershed算法对图像序列进行空间分割,然后利用F检测算法进行帧间变化检测,将时空分割结果结合起来,得到初始的变化检测模板。通过相应的基于二值形态算子的后处理,得到最终的分割结果。整个过程基本是对灰度图像和二值模板的形态处理,简单易行。实验结果表明该算法可以较好地分离前景和背景,定位和分割视频对象。     

一种基于时空联合的视频对象分割方法

视频对象分割在基于内容的视频编码和视频检索中均有重要的应用.为此,针对视频对象分割,提出了一种时域和空域信息融合的视频对象分割方案,该方案首先对时域分割采用基于F-假设检验的方法来得到初始的变化检测模板,然后通过与基于形态学的空域分割融合来获得最终的运动对象.实验结果表明,该方案计算比较简单,能较好地将前景运动对象从静止或运动、简单或复杂的背景中分离出来,且定位精度较好.