Sparse representation for robust abnormality detection in crowded scenes

In crowded scenes, the extracted low-level features, such as optical flow or spatio-temporal interest point, are inevitably noisy and uncertainty. In this paper, we propose a fully unsupervised non-negative sparse coding based approach for abnormality event detection in crowded scenes, which is specifically tailored to cope with feature noisy and uncertainty. The abnormality of query sample is decided by the sparse reconstruction cost from an atomically learned event dictionary, which forms a sparse coding bases. In our algorithm, we formulate the task of dictionary learning as a non-negative matrix factorization (NMF) problem with a sparsity constraint. We take the robust Earth Mover's Distance (EMD), instead of traditional Euclidean distance, as distance metric reconstruction cost function. To reduce the computation complexity of EMD, an approximate EMD, namely wavelet EMD, is introduced and well combined into our approach, without losing performance. In addition, the combination of wavelet EMD with our approach guarantees the convexity of optimization in dictionary learning. To handle both local abnormality detection (LAD) and global abnormality detection, we adopt two different types of spatio-temporal basis. Experiments conducted on four public available datasets demonstrate the promising performance of our work against the state-of-the-art methods.     

Anomaly detection in crowded scenes using motion energy model

Multimedia Tools and Applications - We present a new method for detection of abnormal behaviors in crowded scenes. Based on statistics of low-level feature—optical flow, which describes human...     

基于Faster R-CNN的密集人群检测算法

为提高拥挤场景下的人群检测准确率,提出一种基于改进Faster R-CNN的密集人群检测算法。首先,在特征提取阶段添加空间与通道注意力机制,使用加强的双向特征金字塔网络（S-BiFPN）替代原网络中的多尺度特征金字塔（FPN）,使网络对重要特征进行自主学习并加强对图像深层特征的提取;其次,引入多实例预测（MIP）算法对实例进行预测,以避免模型对拥挤场景下的目标造成漏检;最后,对模型中的非极大值抑制（NMS）进行优化,并额外增设一个交并比（IoU）阈值,以对检测结果的干扰项进行精确抑制。在开源的密集人群检测数据集上进行测试的结果显示,相较于原Faster R-CNN算法,所提算法的平均精度（AP）提升5.6%,Jaccard指数值提升3.2%。所提算法具有较高检测精度和稳定性,可以满足密集场景人群检测的需求。     

An efficient subsequence search for video anomaly detection and localization

This paper presents a novel framework for anomaly event detection and localization in crowded scenes. For anomaly detection, one-class support vector machine with Bayesian derivation is applied to detect unusual events. We also propose a novel event representation, called subsequence, which refers to a time series of spatial windows in proximity. Unlike recent works encoded an event with a 3D bounding box which may contain irrelevant information, e.g. background, a subsequence can concisely capture the unstructured property of an event. To efficiently locate anomalous subsequences in a video space, we propose the maximum subsequence search. The proposed search algorithm integrates local anomaly scores into a global consistent detection so that the start and end of an abnormal event can be determined under false and missing detections. Experimental results on two public datasets show that our method is robust to the illumination change and achieve at least 80% localization rate which approximately doubles the accuracy of recent works. This study concludes that anomaly localization is crucial in finding abnormal events.     

拥挤场景下基于密集轨迹对准及其运动影响描述符的异常活动检测

针对现有异常活动检测算法对拥挤场景下的目标跟踪和描述能力不足的问题,文中提出基于密集轨迹对准及其运动影响描述符的算法,捕捉视频目标运动的关键信息.密集轨迹保证对视频运动目标的有效提议,沿着轨迹的方向提取与轨迹对准的运动影响描述符.最后提出完整框架,准确检测全局和局部的异常活动.在UCSD公共数据集上的实验证明文中方法性能较优.     

SCOOP: A Real-Time Sparsity Driven People Localization Algorithm

Detecting and tracking people in scenes monitored by cameras is an important step in many application scenarios such as surveillance, urban planning or behavioral studies to name a few. The amount of data produced by camera feeds is so large that it is also vital that these steps be performed with the utmost computational efficiency and often even real-time. We propose SCOOP, a novel algorithm that reliably localizes people in camera feeds, using only the output of a simple background removal technique. SCOOP can handle a single or many video feeds. At the heart of our technique there is a sparse model for binary motion detection maps that we solve with a novel greedy algorithm based on set covering. We study the convergence and performance of the algorithm under various degradation models such as noisy observations and crowded environments, and we provide mathematical and experimental evidence of both its efficiency and robustness using standard datasets. This clearly shows that SCOOP is a viable alternative to existing state-of-the-art people localization algorithms, with the marked advantage of real-time computations.     

A system for learning statistical motion patterns

Analysis of motion patterns is an effective approach for anomaly detection and behavior prediction. Current approaches for the analysis of motion patterns depend on known scenes, where objects move in predefined ways. It is highly desirable to automatically construct object motion patterns which reflect the knowledge of the scene. In this paper, we present a system for automatically learning motion patterns for anomaly detection and behavior prediction based on a proposed algorithm for robustly tracking multiple objects. In the tracking algorithm, foreground pixels are clustered using a fast accurate fuzzy k-means algorithm. Growing and prediction of the cluster centroids of foreground pixels ensure that each cluster centroid is associated with a moving object in the scene. In the algorithm for learning motion patterns, trajectories are clustered hierarchically using spatial and temporal information and then each motion pattern is represented with a chain of Gaussian distributions. Based on the learned statistical motion patterns, statistical methods are used to detect anomalies and predict behaviors. Our system is tested using image sequences acquired, respectively, from a crowded real traffic scene and a model traffic scene. Experimental results show the robustness of the tracking algorithm, the efficiency of the algorithm for learning motion patterns, and the encouraging performance of algorithms for anomaly detection and behavior prediction.     

Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models

We propose a novel unsupervised learning framework to model activities and interactions in crowded and complicated scenes. Hierarchical Bayesian models are used to connect three elements in visual surveillance: low-level visual features, simple "atomic" activities, and interactions. Atomic activities are modeled as distributions over low-level visual features, and multi-agent interactions are modeled as distributions over atomic activities. These models are learnt in an unsupervised way. Given a long video sequence, moving pixels are clustered into different atomic activities and short video clips are clustered into different interactions. In this paper, we propose three hierarchical Bayesian models, Latent Dirichlet Allocation (LDA) mixture model, Hierarchical Dirichlet Process (HDP) mixture model, and Dual Hierarchical Dirichlet Processes (Dual-HDP) model. They advance existing language models, such as LDA [1] and HDP [2]. Our data sets are challenging video sequences from crowded traffic scenes and train station scenes with many kinds of activities co-occurring. Without tracking and human labeling effort, our framework completes many challenging visual surveillance tasks of board interest such as: (1) discovering typical atomic activities and interactions; (2) segmenting long video sequences into different interactions; (3) segmenting motions into different activities; (4) detecting abnormality; and (5) supporting high-level queries on activities and interactions.     

Scene Aware Detection and Block Assignment Tracking in crowded scenes

How far can human detection and tracking go in real world crowded scenes? Many algorithms often fail in such scenes due to frequent and severe occlusions as well as viewpoint changes. In order to handle these difficulties, we propose Scene Aware Detection (SAD) and Block Assignment Tracking (BAT) that incorporate with some available scene models (e.g. background, layout, ground plane and camera models). The SAD is proposed for accurate detection through utilizing 1) camera model to deal with viewpoint changes by rectifying sub-images, 2) a structural filter approach to handle occlusions based on a feature sharing mechanism in which a three-level hierarchical structure is built for humans, and 3) foregrounds for pruning negative and false positive samples and merging intermediate detection results. Many detection or appearance based tracking systems are prone to errors in occluded scenes because of failures of detectors and interactions of multiple objects. Differently, the BAT formulates tracking as a block assignment process, where blocks with the same label form the appearance of one object. In the BAT, we model objects on two levels, one is the ensemble level to measure how it is like an object by discriminative models, and the other one is the block level to measure how it is like a target object by appearance and motion models. The main advantage of BAT is that it can track an object even when all the part detectors fail as long as the object has assigned blocks. Extensive experiments in many challenging real world scenes demonstrate the efficiency and effectiveness of our approach.     

An automated day-time cloud detection technique applied to MSG-SEVIRI data over Western Europe

Since February 2003, the Spinning Enhanced Visible and Infrared Imager (SEVIRI) aboard the first Meteosat Second Generation (MSG) satellite has provided radiance data in 12 spectral bands for a full Earth hemisphere every 15 minutes. This high frame rate renders it an excellent tool for studies of atmospheric transport of pollutants, aerosol and clouds. TNO (Netherlands Organisation for Applied Scientific Research) is currently developing an algorithm for the retrieval of aerosol properties from MSG-SEVIRI observations over cloud-free scenes. This requires rigorous cloud screening for which a fast and stand-alone algorithm is developed. The detection technique described in this paper, which is based on the ATSR-2 (Along Track Scanning Radiometer 2) cloud screening algorithm, can be easily implemented, and satisfactorily identifies clouds. The study presented here focuses on Western Europe for the year 2006. Cloud detection results are compared to the KNMI/MF (Royal Netherlands Meteorological Institute/Meteo-France) and the Moderate Resolution Imaging Spectroradiometer (MODIS) cloud detection algorithms. According to the statistics, the results obtained with our algorithm show good agreement (>80%) with these data sets.     

Segmenting images using localized histograms and region merging

A working system for segmenting images of complex scenes is presented. The system integrates techniques that have evolved out of many years of research in low-level image segmentation at the University of Massachusetts and elsewhere. This paper documents the result of this historical evolution. Segmentations produced by the system are used extensively in related image interpretation research.The system first produces segmentations based upon an analysis of spatially localized feature histograms. These initial segmentations are then simplified using a region merging algorithm. Parameter selection for the local histogram segmentation algorithm is facilitated by mapping the multidimensional parameter space to a one-dimensional parameter which regulates region fragmentation. An extension of this algorithm to multiple features is also presented. Experience with roughly 100 images from different domains has shown the system to be robust and effective. Samples of these results are included.     

A Real-Time Algorithm for Small Group Detection in Medium Density Crowds

In this paper, we focus on the task of small group detection in crowded scenarios. Small groups are widely considered as one of the basic elements in crowds, so it is a major challenge to distinguish group members from the individuals in the crowd. It is also a basic problem in video surveillance and scene understanding. We propose a solution for this task, which could run in real time and could work in both low and medium density crowded scenes. In particular, we build a social force based collision avoidance model on each individual for goal direction prediction, and employ the predicted goal directions instead of traditional positions and velocities in collective motion detection to find group members. We evaluate our approach over three datasets including tens of challenging crowded scenarios. The experimental results demonstrate that our proposed approach is not only highly accurate but also improves the practical property performance compared to other state-of-the-art methods.     

Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes

Tracking pedestrians is a vital component of many computer vision applications, including surveillance, scene understanding, and behavior analysis. Videos of crowded scenes present significant challenges to tracking due to the large number of pedestrians and the frequent partial occlusions that they produce. The movement of each pedestrian, however, contributes to the overall crowd motion (i.e., the collective motions of the scene's constituents over the entire video) that exhibits an underlying spatially and temporally varying structured pattern. In this paper, we present a novel Bayesian framework for tracking pedestrians in videos of crowded scenes using a space-time model of the crowd motion. We represent the crowd motion with a collection of hidden Markov models trained on local spatio-temporal motion patterns, i.e., the motion patterns exhibited by pedestrians as they move through local space-time regions of the video. Using this unique representation, we predict the next local spatio-temporal motion pattern a tracked pedestrian will exhibit based on the observed frames of the video. We then use this prediction as a prior for tracking the movement of an individual in videos of extremely crowded scenes. We show that our approach of leveraging the crowd motion enables tracking in videos of complex scenes that present unique difficulty to other approaches.     

3D Motion Completion in Crowded Scenes

Crowded motions refer to multiple objects moving around and interacting such as crowds, pedestrians and etc. We capture crowded scenes using a depth scanner at video frame rates. Thus, our input is a set of depth frames which sample the scene over time. Processing such data is challenging as it is highly unorganized, with large spatio‐temporal holes due to many occlusions. As no correspondence is given, locally tracking 3D points across frames is hard due to noise and missing regions. Furthermore global segmentation and motion completion in presence of large occlusions is ambiguous and hard to predict. Our algorithm utilizes Gestalt principles of common fate and good continuity to compute motion tracking and completion respectively. Our technique does not assume any pre‐given markers or motion template priors. Our key‐idea is to reduce the motion completion problem to a 1D curve fitting and matching problem which can be solved efficiently using a global optimization scheme. We demonstrate our segmentation and completion method on a variety of synthetic and real world crowded scanned scenes.     

一种新的基于局部轮廓特征的目标检测方法

针对复杂场景中背景复杂、目标周围噪声多及目标只占图像中较小部分而难于检测的问题,提出一种新的基于局部轮廓特征的检测目标方法.该方法首先利用改进的全局概率边界算法 (Globalized probability of boundary, gPb) 算法提取图像的轮廓,然后应用最大类间方差法 (Otsu)进行自动阈值处理得到图像的显著性轮廓; 再提取显著性轮廓的k邻近大致直线轮廓段(k connected roughly straight contour segments, kAS),并以kAS作为局部特征,用于复杂场景中的目标检测.该算法结合 gPb 算法和 Otsu 提取轮廓的显著性轮廓,去除了目标附近的大量噪声边界,有效地提高了检测效率.同时,在检测阶段,测试集与 训练集中提取的不相关特征数目也得到较大减少,从而提高了检测的精度.多组实验结果均表明本文方法的有效性.     

Anomaly detection using sparse reconstruction in crowded scenes

Multimedia Tools and Applications - In this paper, we propose an algorithm of anomaly detection in crowded scenes by using sparse representation over the normal bases. First, the histogram of...     

基于主成分分析网络的车牌检测方法

车牌识别是智能交通系统的核心技术,车牌检测是车牌识别技术中至关重要的一步。传统的车牌检测方法多利用浅层的人工特征,在复杂场景下的车牌检测率不高。基于主成分分析网络的车牌检测算法,能够无监督地逐级提取车牌深层特征,可有效提高算法的鲁棒性。算法首先采用Sobel算子边缘检测和边缘对称性分析获取车牌候选区域;然后将候选区域输入到主成分分析网络中进行车牌深度特征提取,并利用支持向量机实现对车牌区域的判别;最后采用非极大值抑制算法标记最佳车牌检测区域。利用收集的复杂场景下的车辆图像对所提方法的参数进行分析,并将其与传统方法进行比较。实验结果表明,所提算法的鲁棒性高,性能优于传统的车牌检测方法。     

Unsupervised segmentation of unknown objects in complex environments

This paper presents a novel object segmentation approach for highly complex indoor scenes. Our approach starts with a novel algorithm which partitions the scene into distinct regions whose boundaries accurately conform to the physical object boundaries in the scene. Next, we propose a novel perceptual grouping algorithm based on local cues (e.g., 3D proximity, co-planarity, and shape convexity) to merge these regions into object hypotheses. Our extensive experimental evaluations demonstrate that our object segmentation results are superior compared to the state-of-the-art methods.