Fusion of color and infrared video for moving human detection

We approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, the moving people in a static scene provide cues to address this problem. In this paper, we propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction.     

Change detection for moving object segmentation with robust background construction under Wronskian framework

Although background subtraction techniques have been used for several years in vision systems for moving object detection, many of them fail to provide good results in presence of noise, illumination variation, non-static background, etc. A basic requirement of background subtraction scheme is the construction of a stable background model and then comparing each incoming image frame with it so as to detect moving objects. The novelty of the proposed scheme is to construct a stable background model from a given video sequence dynamically. The constructed background model is compared with different image frames of the same sequence to detect moving objects. In the proposed scheme the background model is constructed by analyzing a sequence of linearly dependent past image frames in Wronskian framework. The Wronskian based change detection model is further used to detect the changes between the constructed background scene and the considered target frame. The proposed scheme is an integration of Gaussian averaging and Wronskian change detection model. Gaussian averaging uses different modes which arise over time to capture the underlying richness of background, and it is an approach for background building by considering temporal modes. Similarly, Wronskian change detection model uses a spatial region of support in this regard. The proposed scheme relies on spatio-temporal modes arising over time to build the appropriate background model by considering both spatial and temporal modes. The results obtained by the proposed model is found to provide accurate shape of moving objects. The effectiveness of the proposed scheme is verified by comparing the results with those of some of the existing state of the art background subtraction techniques on public benchmark databases. We found that the average F-measure is significantly improved by the proposed scheme from that of the state-of-the-art techniques.     

An improved statistical approach for moving object detection in thermal video frames

In a video surveillance system, background modeling is assumed to be a fundamental technique for moving object detection. The surveillance system based on thermal video overcomes many challenges, such as background variations, varying light intensity, external illumination source, and so on. This paper presents a new method for background modeling and background subtraction. The method utilizes the combined approach of Fisher's Linear Discriminant and Relative Entropy for pixel based classification and detection of moving objects in thermal video frames. The experimental results show the higher average value of various performance indicators like Accuracy, ROC, and F-measure. In contrast, the percentage of false classification and total error is minimum and also has lesser execution time. The method outperforms when compared with the other existing methods.

     

A new thermal infrared and visible spectrum images-based pedestrian detection system

In this paper, we propose a hybrid system for pedestrian detection, in which both thermal and visible images of the same scene are used. The proposed method is achieved in two basic steps: (1) Hypotheses generation (HG) where the locations of possible pedestrians in an image are determined and (2) hypotheses verification (HV), where tests are done to check the presence of pedestrians in the generated hypotheses. HG step segments the thermal image using a modified version of OTSU thresholding technique. The segmentation results are mapped into the corresponding visible image to obtain the regions of interests (possible pedestrians). A post-processing is done on the resulting regions of interests to keep only significant ones. HV is performed using random forest as classifier and a color-based histogram of oriented gradients (HOG) together with the histograms of oriented optical flow (HOOF) as features. The proposed approach has been tested on OSU Color-Thermal, INO Video Analytics and LITIV data sets and the results justify its effectiveness.

     

基于混合高斯模型的运动目标检测

提出了一种新的基于HSV颜色空间的阴影检测和误判检测的自适应背景模型运动目标检测算法,并将其应用于运动物体的分割。该算法较好地解决了背景模型的提取、更新、背景扰动、外界光照变化等问题。实验结果表明了该算法的实时性、可靠性和准确性较好。     

Entropy based region selection for moving object detection

This article addresses a problem of moving object detection by combining two kinds of segmentation schemes: temporal and spatial. It has been found that consideration of a global thresholding approach for temporal segmentation, where the threshold value is obtained by considering the histogram of the difference image corresponding to two frames, does not produce good result for moving object detection. This is due to the fact that the pixels in the lower end of the histogram are not identified as changed pixels (but they actually correspond to the changed regions). Hence there is an effect on object background classification. In this article, we propose a local histogram thresholding scheme to segment the difference image by dividing it into a number of small non-overlapping regions/windows and thresholding each window separately. The window/block size is determined by measuring the entropy content of it. The segmented regions from each window are combined to find the (entire) segmented image. This thresholded difference image is called the change detection mask (CDM) and represent the changed regions corresponding to the moving objects in the given image frame. The difference image is generated by considering the label information of the pixels from the spatially segmented output of two image frames. We have used a Markov Random Field (MRF) model for image modeling and the maximum a posteriori probability (MAP) estimation (for spatial segmentation) is done by a combination of simulated annealing (SA) and iterated conditional mode (ICM) algorithms. It has been observed that the entropy based adaptive window selection scheme yields better results for moving object detection with less effect on object background (mis) classification. The effectiveness of the proposed scheme is successfully tested over three video sequences.     

Fusion of infrared and visible images using neutrosophic fuzzy sets

Fusion of infrared and visible image is a technology which combines information from two different sensors for the same scene. It also gives extremely effective information complementation, which is widely used for the monitoring systems and military fields. Due to limited field depth in an imaging device, visible images can’t identify some targets that may not be apparent due to poor lighting conditions or because that the background color is similar to the target. To deal with this problem, a simple and efficient image fusion approach of infrared and visible images is proposed to extract target’s details from infrared images and enhance the vision in order to improve the performance of monitoring systems. This method depends on maximum and minimum operations in neutrosophic fuzzy sets. Firstly, the image is transformed from its spatial domain to the neutrosophic domain which is described by three membership sets: truth membership, indeterminacy membership, and falsity membership. The indeterminacy in the input data is handled to provide a comprehensive fusion result. Finally, deneutrosophicised process is made which means that the membership values are retransformed into a normal image space. At the end of the study, experimental results are applied to evaluate the performance of this approach and compare it to the recent image fusion methods using several objective evaluation criteria. These experiments demonstrate that the proposed method achieves outstanding visual performance and excellent objective indicators.

     

Physical models for moving shadow and object detection in video

Current moving object detection systems typically detect shadows cast by the moving object as part of the moving object. In this paper, the problem of separating moving cast shadows from the moving objects in an outdoor environment is addressed. Unlike previous work, we present an approach that does not rely on any geometrical assumptions such as camera location and ground surface/object geometry. The approach is based on a new spatio-temporal albedo test and dichromatic reflection model and accounts for both the sun and the sky illuminations. Results are presented for several video sequences representing a variety of ground materials when the shadows are cast on different surface types. These results show that our approach is robust to widely different background and foreground materials, and illuminations.     

A unified saliency detection framework for visible and infrared images

Multimedia Tools and Applications - Conventional saliency detection algorithms usually achieve good detection performance at the cost of high computational complexity, and most of them focus on...     

Salient object detection via robust dictionary representation

Multimedia Tools and Applications - The theory of sparse and low-rank representation has worked competitive performance in the field of salient object detection. Generally, the salient object is...     

Fast moving object detection with non-stationary background

The detection of moving objects under a free-moving camera is a difficult problem because the camera and object motions are mixed together and the objects are often detected into the separated components. To tackle this problem, we propose a fast moving object detection method using optical flow clustering and Delaunay triangulation as follows. First, we extract the corner feature points using Harris corner detector and compute optical flow vectors at the extracted corner feature points. Second, we cluster the optical flow vectors using K-means clustering method and reject the outlier feature points using Random Sample Consensus algorithm. Third, we classify each cluster into the camera and object motion using its scatteredness of optical flow vectors. Fourth, we compensate the camera motion using the multi-resolution block-based motion propagation method and detect the objects using the background subtraction between the previous frame and the motion compensated current frame. Finally, we merge the separately detected objects using Delaunay triangulation. The experimental results using Carnegie Mellon University database show that the proposed moving object detection method outperforms the existing other methods in terms of detection accuracy and processing time.     

一种改进的运动目标精确检测算法

针对传统混合高斯背景模型在多变场景下因背景模型更新不及时而存在的误检、漏检等不足,提出一种改进算法.该算法首先通过在高斯分布匹配过程中结合帧间差分获取的帧间未变化区域与变化区域判断像素点的区域类别,然后根据不同的像素区域类别执行不同的背景更新策略,使背景的更新及时准确地反映背景的变化.实验结果表明,该改进混合高斯背景模型算法能有效地解决因目标和背景相互转化而出现的拖尾、影子以及运动目标空洞等问题.     

Gender recognition from visible and thermal infrared facial images

Most present research of gender recognition focuses on visible facial images, which are sensitive to illumination changes. In this paper, we proposed hybrid methods for gender recognition by fusing visible and thermal infrared images. First, the active appearance model is used to extract features from visible images, as well as local binary pattern features and several statistical temperature features are extracted from thermal infrared images. Then, feature selection is performed by using the F-test statistic. Third, we propose using Bayesian Networks to perform explicit and implicit fusion of visible and thermal infrared image features. For explicit fusion, we propose two Bayesian Networks to perform decision-level and feature-level fusion. For implicit fusion, we propose using features from one modality as privileged information to improve gender recognition by another modality. Finally, we evaluate the proposed methods on the Natural Visible and Infrared facial Expression spontaneous database and the Equinox face database. Experimental results show that both feature-level and decision-level fusion improve the gender recognition performance, compared to that achieved from one modality. The proposed implicit fusion methods successfully capture the role of privileged information of one modality, thus enhance the gender recognition from another modality.     

基于双阈值的运动目标检测

目前在运动目标实时检测中.主要是运用差分法进行检测.但背景差分受光照、环境影响较大,需要实时更新背景,而帧间差分容易出现空洞和误检.结合背景差分和帧间差分,采用双阈值对运动目标进行分割,能对背景进行实时更新,有效的避免了空洞和误检,并且在机场的运动目标检测中取得了较好的效果.     

Local-to-global background modeling for moving object detection from non-static cameras

Multimedia Tools and Applications - This paper investigates efficient and robust moving object detection from non-static cameras. To tackle the motion of background caused by moving cameras and to...     

帧间差分与边缘纹理相融合的运动目标检测

提出了一种利用边缘纹理进行背景建模,结合三帧差分算法,获取目标轮廓的运动目标检测方法,进一步通过区域填充得到目标前景.该方法对光线变化不敏感,对阴影去除有比较好的效果,改进了边缘纹理差分和帧间差分的缺陷,取得了比较完整准确的运动目标前景.     

Granulation,rough entropy and spatiotemporal moving object detection

A new spatio-temporal segmentation approach for moving object(s) detection and tracking from a video sequence is described. Spatial segmentation is carried out using rough entropy maximization, where we use the quad-tree decomposition, resulting in unequal image granulation which is closer to natural granulation. A three point estimation based on Beta Distribution is formulated for background estimation during temporal segmentation. Reconstruction and tracking of the object in the target frame is performed after combining the two segmentation outputs using its color and shift information. The algorithm is more robust to noise and gradual illumination change, because their presence is less likely to affect both its spatial and temporal segments inside the search window. The proposed methods for spatial and temporal segmentation are seen to be superior to several related methods. The accuracy of reconstruction has been significantly high.     

Robust moving object detection against fast illumination change

To solve the problem due to fast illumination change in a visual surveillance system, we propose a novel moving object detection algorithm for which we develop an illumination change model, a chromaticity difference model, and a brightness ratio model. When fast illumination change occurs, background pixels as well as moving object pixels are detected as foreground pixels. To separate detected foreground pixels into moving object pixels and false foreground pixels, we develop a chromaticity difference model and a brightness ratio model that estimates the intensity difference and intensity ratio of false foreground pixels, respectively. These models are based on the proposed illumination change model. Based on experimental results, the proposed method shows excellent performance under various illumination change conditions while operating in real-time.     

基于光谱和亮度信息的红外与可见光图像融合方法

针对红外与可见光图像融合,提出了一种基于区域能量和修正的视觉特征对比度的低频融合规则,以尽可能多地保留红外图像的热目标信息及可见光图像的光谱信息。实验结果表明,在非下采样轮廓波变换和小波变换域,新的融合规则能都充分利用源图像的互补和冗余信息,使融合图像具有更好的主观视觉效果。     

Improved background modeling for real-time spatio-temporal non-parametric moving object detection strategies

Answering to the growing demand of machine vision applications for the latest generation of electronic devices endowed with camera platforms, several moving object detection strategies have been proposed in recent years. Among them, spatio-temporal based non-parametric methods have recently drawn the attention of many researchers. These methods, by combining a background model and a foreground model, achieve high-quality detections in sequences recorded with non-completely static cameras and in scenarios containing complex backgrounds. However, since they have very high memory and computational associated costs, they apply some simplifications in the background modeling process, therefore decreasing the quality of the modeling.