Independent Component Analysis-Based Background Subtraction for Indoor Surveillance

In video surveillance, detection of moving objects from an image sequence is very important for target tracking, activity recognition, and behavior understanding. Background subtraction is a very popular approach for foreground segmentation in a still scene image. In order to compensate for illumination changes, a background model updating process is generally adopted, and leads to extra computation time. In this paper, we propose a fast background subtraction scheme using independent component analysis (ICA) and, particularly, aims at indoor surveillance for possible applications in home-care and health-care monitoring, where moving and motionless persons must be reliably detected. The proposed method is as computationally fast as the simple image difference method, and yet is highly tolerable to changes in room lighting. The proposed background subtraction scheme involves two stages, one for training and the other for detection. In the training stage, an ICA model that directly measures the statistical independency based on the estimations of joint and marginal probability density functions from relative frequency distributions is first proposed. The proposed ICA model can well separate two highly-correlated images. In the detection stage, the trained de-mixing vector is used to separate the foreground in a scene image with respect to the reference background image. Two sets of indoor examples that involve switching on/off room lights and opening/closing a door are demonstrated in the experiments. The performance of the proposed ICA model for background subtraction is also compared with that of the well-known FastICA algorithm.      

Robust bilayer video segmentation by adaptive propagation of global shape and local appearance

Segmenting semantic objects of interest from video has long been an active research topic, with a wide range of potential applications. In this paper, we present a bilayer video segmentation method robust to abrupt motion and change in appearance for both the foreground and background. Specifically, based on a few manually segmented keyframes, the proposed method propagates the global shape of the foreground as priors to adjacent frames by applying branch-and-mincut [1], which jointly estimates what is optimal among a set of shapes along with its pose and the corresponding segmentation in the current image. Based on this preliminary segmentation we determine two types of local regions likely to have erroneous results, and apply a probabilistic framework where shape and appearance cues are adaptively emphasized for local refinement. With each successive frame segmentation, the set of shapes applied as priors are incrementally updated. Experimental results support the robustness of the proposed method for obstacles such as background clutter, motion, and appearance changes, from only a small number of user segmented keyframes.     

Compressed sensing based foreground detection vector for object detection in Wireless Visual Sensor Networks

Compressed sensing based background subtraction (CS-BS) plays a significant role in video surveillance applications in Wireless Visual Sensor Networks. This paper implements a CS-BS framework with a novel thresholding strategy to detect the anomaly with fewer measurements in a secured indoor environment. In CS-BS, the CS is performed on the difference frame which is sparse, thereby reducing energy, memory and bandwidth. In this framework, a foreground threshold is proposed based on the measurement matrix to extract the moving object from a scene. The performance of the CS-BS framework with FDV is evaluated using metrics such as detection accuracy, energy complexity, percentage of reduction in samples and measurements. The proposed CS-BS framework with hybrid matrix based FDV achieves around 95.8% reduction of measurements and 91% reduction of samples.     

后处理在数字抠像中的应用与解析

数字抠像是将一幅图像中的前景物体与背景进行分离的问题,它的关键在于Alpha通道的计算.以往通过采样方法求得的Alpha中,由于采用逐点计算的离散化方式,求解出的Alpha通常不连续,并且包含很多噪声,因而需要对Alpha进行后处理,这不仅会增强Alpha在视觉上的平滑性,而且能够进一步提高Alpha的精确度.在目前国际上,有关数字抠像后处理领域已经进行了许多研究,但缺少相关的综述性文献,并且对后处理后的Alpha如何进行定量的评价也仍未系统解决.本文首先将数字抠像中的后处理方法分为2类:与仿射类方法相结合的方式及自平滑方式,其次,对两类方法进行了全面的总结和梳理,并对方法的优缺点进行了分析,对将来研究方向提出了建议,最后,针对后处理后的Alpha结果进行了全面的量化比较,弥补了传统方法基本上仅在视觉层面上进行比较的缺陷.     

Region-level motion-based background modeling and subtraction using MRFs.

This paper presents a new approach to automatic segmentation of foreground objects from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the classification problem as a graph labeling over a region adjacency graph based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, a temporal coherence is also maintained by modeling it as the prior energy. On the other hand, color distributions of two neighboring regions are taken into consideration to impose spatial coherence. Then, the a priori energy of MRFs takes both spatial and temporal coherence into account to maintain the continuity of our segmentation. Finally, a labeling is obtained by maximizing the a posteriori energy of the MRFs. Under such formulation, we integrate two different kinds of techniques in an elegant way to make the foreground detection more accurate. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.     

Unsupervised and reliable image matting based on modified spectral matting

Spectral matting is the state-of-the-art image matting and also a milestone in theoretic matting research. For spectral matting without user intervention, the accuracy of alpha matte is low and the computational cost is high. Therefore, this paper presents a modified version of spectral matting to greatly increase the accuracy of alpha matte and effectively reduce the computational cost. In the proposed modified spectral matting, palette-based component classification is used to obtain reliable foreground and background components. Next, the corresponding matting components are obtained via a linear transformation of the smallest eigenvectors of the matting Laplacian matrix. Finally, the matting components of the foreground and the unknown regions are combined to from the complete alpha matte based on minimizing the matte cost. Moreover, image composition with consistency of color temperature is used to obtain the realistic image composition. Experimental results show that the proposed method outperforms the state-of-the-art methods based on spectral matting.     

基于场景感知的运动目标检测方法

背景减除法是一种主要的运动目标检测框架,但在复杂环境中构建一种初始模型建立周期短、可靠性高、鲁棒性好的模型仍是一大难题.本文从场景感知的角度出发,在背景减除框架的基础上提出一种目标检测方法.该方法根据前两帧中稳定的结构信息感知背景中潜在的前景区域,在第二帧建立初始模型时利用最近邻域背景像素点代替可能的前景像素点,提高了初始模型可靠性;结合颜色信息和二进制特征提出了像素点二级分类判决机制,并通过感知像素点邻域内的纹理复杂度自适应调整局部判决阈值和更新频率;在模型更新阶段提出处理误判的反馈机制.在公开视频序列上同几种流行检测算法的实验对比结果证明了本文算法的有效性和优越性.     

光照突变下融合多类特征的场景分割方法

为解决场景模型在快速光照变化下失效的问题,提出了一种新的前景目标分割方法。该方法共包括三个步骤。首先,利用全局光照函数建立高斯混合模型;其次,提取当前帧中的纹理、ZNCC 及轮廓特征;最后,将提取到的特征分两阶段与高斯混合模型进行融合(第一阶段:融合纹理及ZNCC 特征;第二阶段:融合轮廓特征),得到最终的场景分割结果。实验结果表明:该算法具有较好的鲁棒性,并且相较于基于全局光照建模的方法具有更高的精度值及召回值。     

Automatic, robust global motion estimation using clustering

Global motion estimation (GME) is a vital part of many video compression and computer vision applications. However, the large moving foreground objects that are present in many video scenes make the task of GME more challenging. In this paper, we propose an automatic, efficient, and robust approach for GME that addresses the issue of large foreground objects. The proposed GME algorithm is based on two key ideas: a new clustering technique, to automate the initial segmentation of background and foreground blocks, and a modified Lorentzian estimator, to reduce the impact of any remaining foreground blocks on the GME process. We also apply an up-sampling technique to the estimated motion parameters to remove any errors caused by under-sampling during the warping process. These ideas provide a significant improvement in performance when combined into a common framework. Simulation results and analyses demonstrate the improved performance of our proposed algorithm over other state-of-the-art methods.     

Time efficient ghost removal for motion detection in visual surveillance systems

Segmentation of foreground objects is an important and essential task for many systems that aim to carry out motion tracking, object classification, event detection and is used in applications such as traffic monitoring and analysis, access control to special areas, human and vehicle identification and the detection of anomalous behaviour. The most common approach for detecting moving objects is background subtraction, in which each frame of a video sequence is compared against a background model. A large number of background subtraction algorithms have been proposed [1], but problems remain for moving object identification under certain conditions. One of the toughest problems in background subtraction is caused by the detection of false objects when an object that belongs to the background (e.g. after staying stationary for some time) starts to move away. This generates what are called `ghosts?. It is important to address the problem because ghost objects will adversely affect many tasks such as object classification, tracking and event analysis (e.g. abandoned item detection). This Letter focuses on the problem of ghost identification and elimination. We used a state-of-the-art industrial tracker which includes basic background subtraction and object tracking. Then we included our ghost detection algorithm into the basic tracker to identify and eliminate ghosts. Finally, we systematically evaluated and compared performance on urban traffic video sequences.     

Maritime filtering for images and videos

Maritime signal processing technologies have emerged as an important area of study because of the increasing popularity of autonomous ships and automatic maritime surveillance systems. However, the various techniques developed for detecting or tracking objects remain unable to address various maritime noise challenges that cause several types of false positives in maritime visual surveillance. Maritime signal processing is challenging because of the prevalence of noise sources such as severe dynamic backgrounds, wakes, and reflections, owing to the complex, unconstrained, and diverse nature of such scenes caused by the surface properties of water. Moreover, few studies have investigated specific maritime noise filtering as a general integrated processing approach with image and video technologies in the context of maritime visual surveillance. In this study, we propose a novel maritime noise prior (MNP) based on a dark channel prior and observations of the characteristics of the sea. A general maritime filtering technique is developed to suppress noise originating from the properties of water in maritime images and videos. The proposed method employs a noniterative, nonlinear, and simple maritime filtering approach based on MNP that does not require specialized knowledge of application scene conditions or structure. We conducted image and video experiments by applying our approach to three publicly available databases. In experiments with color images, our method successfully filtered related background noise and water, i.e., severe boat wakes and reflections, while preserving objects other than water in color images. In the experiments with video sequences, the results demonstrated that the proposed filter improved the overall performance of state-of-the-art background subtraction (BS) algorithms from 36.60%–50.63%. By combining BS algorithms and filtering to enhance foreground detection in video sequences, the proposed method ensures the universal applicability and flexibility required to eliminate noise from images and videos obtained in challenging maritime environments. The results indicate that the proposed method is appropriate for maritime surveillance applications implementing image segmentation and foreground detection, and it can potentially increase the accuracy of maritime visual surveillance.     

Aerial video surveillance and exploitation

There is growing interest in performing aerial surveillance using video cameras. Compared to traditional framing cameras, video cameras provide the capability to observe ongoing activity within a scene and to automatically control the camera to track the activity. However, the high data rates and relatively small field of view of video cameras present new technical challenges that must be overcome before such cameras can be widely used. In this paper, we present a framework and details of the key components for real-time, automatic exploitation of aerial video for surveillance applications. The framework involves separating an aerial video into the natural components corresponding to the scene. Three major components of the scene are the static background geometry, moving objects, and appearance of the static and dynamic components of the scene. In order to delineate videos into these scene components, we have developed real time, image-processing techniques for 2-D/3-D frame-to-frame alignment, change detection, camera control, and tracking of independently moving objects in cluttered scenes. The geo-location of video and tracked objects is estimated by registration of the video to controlled reference imagery, elevation maps, and site models. Finally static, dynamic and reprojected mosaics may be constructed for compression, enhanced visualization, and mapping applications     

航空视频图像地理定位处理技术分析

在飞行平台上安装视频摄像机实施对地面进行观测、勘测、监视越来越多地被用于城市建设、森林植被勘测、边境安全监视、反恐和缉毒等各行各业中。在各种实际的应用中,不但要求获取清晰的视频图像,更要求对视频图像中的地面目标和场景实现地理定位。描述了目前使用的直接定位处理技术和精确定位处理技术,给出了直接定位技术原理和误差分析,并对如何实现航空视频图像精确定位处理方法提出了处理模型和流程。     

Video representation with three-dimensional entities

Very low bit-rate coding requires new paradigms that go well beyond pixel- and frame-based video representations. We introduce a novel content-based video representation using tridimensional entities: textured object models and pose estimates. The multiproperty object models carry stochastic information about the shape and texture of each object present in the scene. The pose estimates define the position and orientation of the objects for each frame. This representation is compact. It provides alternative means for handling video by manipulating and compositing three-dimensional (3-D) entities. We call this representation tridimensional video compositing, or 3DVC for short. We present the 3DVC framework and describe the methods used to construct incrementally the object models and the pose estimates from unregistered noisy depth and texture measurements. We also describe a method for video frame reconstruction based on 3-D scene assembly, and discuss potential applications of 3DVC to video coding and content-based handling. 3DVC assumes that the objects in the scene are rigid and segmented. By assuming segmentation, we do not address the difficult questions of nonrigid segmentation and multiple object segmentation. In our experiments, segmentation is obtained via depth thresholding. It is important to notice that 3DVC is independent of the segmentation technique adopted. Experimental results with synthetic and real video sequences where compression ratios in the range of 1:150-1:2700 are achieved demonstrate the applicability of the proposed representation to very low bit-rate coding     

Video object segmentation in rainy situations based on difference scheme with object structure and color analysis

A scheme based on a difference scheme using object structures and color analysis is proposed for video object segmentation in rainy situations. Since shadows and color reflections on the wet ground pose problems for conventional video object segmentation, the proposed method combines the background construction-based video object segmentation and the foreground extraction-based video object segmentation where pixels in both the foreground and background from a video sequence are separated using histogram-based change detection from which the background can be constructed and detection of the initial moving object masks based on a frame difference mask and a background subtraction mask can be further used to obtain coarse object regions. Shadow regions and color-reflection regions on the wet ground are removed from the initial moving object masks via a diamond window mask and color analysis of the moving object. Finally, the boundary of the moving object is refined using connected component labeling and morphological operations. Experimental results show that the proposed method performs well for video object segmentation in rainy situations.     

运动人体轮廓检测算法的分析与改进

在视频序列的人体运动分析中,实时分割出运动的人体,是研究的关键步骤。为了克服不均匀光照、前景运动缓慢、背景中存在摇摆的树叶等因素对检测带来的影响,提出了一种背景减除法与帧间差分相结合的运动目标检测方法。该方法首先通过基于帧差法的背景模型建立方法建立背景图像,再结合背景减除与带有权值的帧间差分检测运动目标,降低目标物体对速度和环境干扰的敏感性。最后通过形态学梯度运算操作消除外界噪声的影响。实验结果表明,本文提出的算法计算简单,对环境适应能力较强,是一种有效的运动人体检测方法。     

Robust moving shadow detection with hierarchical mixture of MLP experts

Detection and elimination of the shadows of moving objects in video sequences have been one of the major challenges in tracking applications. Since moving shadows cannot be removed from foreground by motion-based background subtraction methods, they lead to confusion and error in moving object tracking. In this paper, a novel classification method based on hierarchical mixture of experts learning for detecting shadows from foreground is proposed. A hierarchical mixture of MLP experts method (HMME) with semi-supervised teacher-directed learning (SSP-HMME) is used. It contains a two-level mixture of experts (ME) system. The main superiority of this method is that it is more robust than state-of-the-art methods in all types of indoor and outdoor environments. The robustness is against the number of light sources, illumination conditions, surface orientations, object sizes, etc., and it is estimated using accuracy rates. The video set has been collected from 7 different datasets. The results of experiments in outdoor and indoor environments show the validity of the method in the improvement on the accuracy of both detection and discrimination rate for moving shadows in video sequences. The results of the experiments show the accuracy rate of 89 % in average in different indoor and outdoor environmental conditions that is about 6 % better than current state-of-the-art methods.     

Automatic Segmentation of Moving Objects in Video Sequences for Indoor and Outdoor Applications

1　IntroductionAutomaticsegmentationofmovingobjectsfromvideosequencesisadifficultandchallengingproblemincomputervisionsystems.Ithasmanyapplicationssuchasvideosurveillance,trafficmonitoring ,peopletrackingandvideocommunication[1～4] .Italsoplaysanimportantroleinsupportingcontent basedimagecoding,especiallyaftertheemergenceofthevideocodingstandardMPEG 4[5～ 1 4 ] .Therearealotofresearchworksonmovingob jectssegmentationandextraction .Thesealgorithmscanberoughlyclassifiedintotwocategories:inter …     

Vehicle Tracking and Speed Measurement system (VTSM) based on novel feature descriptor: Diagonal Hexadecimal Pattern (DHP)

Object detection and tracking is an important and active research area in computer vision community. The proposed Vehicle Tracking and Speed Measurement (VTSM) system can find out speed parameters of the vehicles. Speed parameters are used to take judgment on accidents at a low cost. The main objective of this paper is to develop an algorithm that can detect foreground, track specified object and calculate speed parameter of the object. Identifying stationary background from moving objects in a video is a critical task. To achieve superior foreground detection quality across unconstrained scenarios, a novel dynamic background subtraction and object tracking algorithm using a novel Diagonal Hexadecimal Pattern (DHP) is proposed. Metric F-score and MOTA are used to measure the performance of the proposed system. From the results, it is observed that the proposed system gives good results for the background subtraction and tracking.