Abnormal visual event detection based on multi-instance learning and autoregressive integrated moving average model in edge-based Smart City surveillance

The abnormal visual event detection is an important subject in Smart City surveillance where a lot of data can be processed locally in edge computing environment. Real-time and detection effectiveness are critical in such an edge environment. In this paper, we propose an abnormal event detection approach based on multi-instance learning and autoregressive integrated moving average model for video surveillance of crowded scenes in urban public places, focusing on real-time and detection effectiveness. We propose an unsupervised method for abnormal event detection by combining multi-instance visual feature selection and the autoregressive integrated moving average model. In the proposed method, each video clip is modeled as a visual feature bag containing several subvideo clips, each of which is regarded as an instance. The time-transform characteristics of the optical flow characteristics within each subvideo clip are considered as a visual feature instance, and time-series modeling is carried out for multiple visual feature instances related to all subvideo clips in a surveillance video clip. The abnormal events in each surveillance video clip are detected using the multi-instance fusion method. This approach is verified on publically available urban surveillance video datasets and compared with state-of-the-art alternatives. Experimental results demonstrate that the proposed method has better abnormal event detection performance for crowded scene of urban public places with an edge environment.     

基于隐马尔可夫模型的视频异常场景检测

视频技术的广泛应用带来海量的视频数据,仅依靠人力对监控视频中的异常进行检测是不太可能的。异常行为的自动化检测在公共安全等领域的地位极其重要。提出一种综合考虑目标特性和时空上下文的异常检测方法,该方法利用光流纹理图描述移动物体的刚性特征,建立基于隐马尔可夫模型HMM的时间上下文异常检测模型。在此基础上,提取异常目标的Radon特征,以支持向量机SVM的异常预分类结果为基础,通过HMM建立异常场景的空间上下文分类模型。该模型在公共数据集UCSD PED2上进行了实验验证,结果表明,本算法不仅在异常检测方面优于已有算法,而且还能给出异常分类。     

Hierarchical abnormal event detection by real time and semi-real time multi-tasking video surveillance system

In this paper, we describe how to detect abnormal human activities taking place in an outdoor surveillance environment. Human tracks are provided in real time by the baseline video surveillance system. Given trajectory information, the event analysis module will attempt to determine whether or not a suspicious activity is currently being observed. However, due to real-time processing constrains, there might be false alarms generated by video image noise or non-human objects. It requires further intensive examination to filter out false event detections which can be processed in an off-line fashion. We propose a hierarchical abnormal event detection system that takes care of real time and semi-real time as multi-tasking. In low level task, a trajectory-based method processes trajectory data and detects abnormal events in real time. In high level task, an intensive video analysis algorithm checks whether the detected abnormal event is triggered by actual humans or not.     

Hierarchical crowd analysis and anomaly detection

ObjectiveThis work proposes a novel approach to model the spatiotemporal distribution of crowd motions and detect anomalous events.MethodsWe first learn the regions of interest (ROIs) which inform the behavioral patterns by trajectory analysis with Hierarchical Dirichlet Processes (HDP), so that the main trends of crowd motions can be modeled. Based on the ROIs, we then build a series of histograms both on global and local levels as the templates for the observed movement distribution, which statistically describes time-correlated crowd events. Once the template has been built hierarchically, we import real data containing the discrete trajectory observations from video surveillance and detect abnormal events for individuals and for crowds.ResultsExperimental results show the effectiveness of our approach, which is able to analyze and extract the crowd motion information from observed trajectory dataset, and achieve the anomaly detection at the hierarchical levels.ConclusionThe proposed hierarchical approach can learn the moving trends of crowd both in global and local area and describe the crowd behaviors in statistical way, which build a template for pedestrian movement distribution that allows for the detection of time-correlated abnormal crowd events.     

融合自编码器和one-class SVM的异常事件检测

目的 在自动化和智能化的现代生产制造过程中,视频异常事件检测技术扮演着越来越重要的角色,但由于实际生产制造中异常事件的复杂性及无关生产背景的干扰,使其成为一项非常具有挑战性的任务。很多传统方法采用手工设计的低级特征对视频的局部区域进行特征提取,然而此特征很难同时表示运动与外观特征。此外,一些基于深度学习的视频异常事件检测方法直接通过自编码器的重构误差大小来判定测试样本是否为正常或异常事件,然而实际情况往往会出现一些原本为异常的测试样本经过自编码得到的重构误差也小于设定阈值,从而将其错误地判定为正常事件,出现异常事件漏检的情形。针对此不足,本文提出一种融合自编码器和one-class支持向量机（support vector machine,SVM）的异常事件检测模型。方法 通过高斯混合模型（Gaussian mixture model,GMM）提取固定大小的时空兴趣块（region of interest,ROI）;通过预训练的3维卷积神经网络（3D convolutional neural network,C3D）对ROI进行高层次的特征提取;利用提取的高维特征训练一个堆叠的降噪自编码器,通过比较重构误差与设定阈值的大小,将测试样本判定为正常、异常和可疑3种情况之一;对自编码器降维后的特征训练一个one-class SVM模型,用于对可疑测试样本进行二次检测,进一步排除异常事件。结果 本文对实际生产制造环境下的机器人工作场景进行实验,采用AUC （area under ROC）和等错误率（equal error rate,EER）两个常用指标进行评估。在设定合适的误差阈值时,结果显示受试者工作特征（receiver operating characteristic,ROC）曲线下AUC达到91.7%,EER为13.8%。同时,在公共数据特征集USCD （University of California,San Diego） Ped1和USCD Ped2上进行了模型评估,并与一些常用方法进行了比较,在USCD Ped1数据集中,相比于性能第2的方法,AUC在帧级别和像素级别分别提高了2.6%和22.3%;在USCD Ped2数据集中,相比于性能第2的方法,AUC在帧级别提高了6.7%,从而验证了所提检测方法的有效性与准确性。结论 本文提出的视频异常事件检测模型,结合了传统模型与深度学习模型,使视频异常事件检测结果更加准确。     

聚众事件的建模与实现

在视频监控领域聚众等异常事件检测有着广泛的应用前景,然而相关研究在国内发展还比较缓慢。在这里给出了基于隐马尔科夫模型的聚众事件的检测方法,其简单过程如下:首先在高斯混合模型检测出目标的基础上,针对聚众事件视频序列的特性,完成了关于帧图像二元组的特征提取;然后,在合理选择初始模型的前提下使用Baum-Welch算法训练聚众事件的隐马尔科夫模型;最后通过实拍的视频序列验证其有效性。     

Sniffer-Net: quantitative evaluation of smoke in the wild based on spatial–temporal motion spectrum

Smoke detection plays an essential role in the wild video surveillance systems for abnormal events warning. In this paper, we introduced a dedicated neural network structure named Sniffer-Net to simultaneously extract smoke dynamic feature robustly and evaluate the smoke concentration accurately. Firstly, we utilize an improved LiteFlowNet to estimate the global optical flow from image sequence. Meanwhile, a Marr–Hildreth method is brought up and fused into this network to distinguish and eliminate occluded regions from global flow map. Then, an evaluation module based on Context-Encoder network is put forward specially to quantify smoke concentration levels. This network, following the improved LiteFlowNet, is modified through replacing the loss function and removing the multiscale scheme and trained to infer approximate smoke optical flow behind occlusion regions. Starting from the statistical view, the irregular RGB/HSV feature spaces are converted into a specific quantitative evaluation space. As a result, the whole evaluation system is responsible to transform the distribution of irregular smoke motion feature into a quantified form of representation. In turn, this transformation endows the system with a novel numerical standard for smoke concentration evaluation. Finally, an accuracy assessment method is applied to compare the results of detected smoke concentration with the human experience prior model, which feedback the accuracy and false detection rate of system algorithm. In the experiments of five smoke datasets, our proposed smoke detection approach is superior to other state-of-the-art methods, and concentration algorithm achieves the satisfactory performance of 97.3% accuracy on some specialized dataset.

     

基于多任务CNN的监控视频中异常行人快速检测

在近年来社会公共安全受到广泛关注的情况下,如何利用监控视频对异常行人进行监督,预防危险事件的发生成为了一个热门课题.异常行人是指与普通行人在外观上有明显异常性区别的人,例如用头盔大面积遮挡面部或低头躲避摄像头,考虑到异常行人的特征主要集中在头面部,本文提出一种基于多任务卷积神经网络和单类支持向量机的针对头面部特征的异常行人快速检测方法.首先进行头面部区域的检测,然后使用多任务卷积神经网络提取头面部区域的特征,之后使用单类支持向量机判断是正常行人还是异常行人.此外,本文还针对卷积神经网络设计了一种卷积核拆分方法,加快了特征提取的速度,最终实验表明,本文提出的算法能够快速有效的检测出监控视频中的异常行人.     

视频监控系统中的一种信息存储新方法

视频监控系统中信息存储是一个重要和有待解决的问题。本文分析了目前视频监控系统中信息存储方面存在的不足,提出了一种基于DSP和USB技术的视频监控系统信息存储方法,在软件实现上运用了动目标检测,动目标分割和图像相似度比较等算法。基于该方法的视频监控系统只记录有动目标存在时的图像,解决了视频监控系统需存储大量视频信息的问题,给DSP扩展的USBHOST接口,方便了信息的存取。     

Crowd behaviors analysis and abnormal detection based on surveillance data

Crowd analysis and abnormal trajectories detection are hot topics in computer vision and pattern recognition. As more and more video monitoring equipments are installed in public places for public security and management, researches become urgent to learn the crowd behavior patterns through the trajectories obtained by the intelligent video surveillance technology. In this paper, the FCM (Fuzzy c-means) algorithm is adopted to cluster the source points and sink points of trajectories that are deemed as critical points into several groups, and then the trajectory clusters can be acquired. The feature information statistical histogram for each trajectory cluster which contains the motion information will be built after refining them with Hausdorff distances. Eventually, the local motion coherence between test trajectories and refined trajectory clusters will be used to judge whether they are abnormal.     

基于视频监控参考量的异常行为检测研究

为了解决传统算法难以检测一般动态场景情形下人体运动目标的问题,文中提出了一种新的人体运动异常行为的检测方法,该方法组合利用视频监控各个的参考量。文中针对视频序列中人的行为进行分析,目的是检测出人的异常行为,具体涉及：人体运动目标的检测、跟踪与提取,异常行为检测等。文中阐述了异常行为检测的相关概念,介绍了视频监控参考量各个参数的计算方法,探讨了异常行为检测与分类技术的关系。结合异常行为检测与分类的相似性,提出了基于视频监控参考量的算法的异常行为检测方法,给出了其计算方法,并确定了检测的过程,分析该方法的特点和优势。     

Surveillance Robot Utilizing Video and Audio Information

For the aging population, surveillance in household environments has become more and more important. In this paper, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients (MFCC) is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as “falling down” and “running”. Also, a 88.17% accuracy rate is achieved in the detection of abnormal audio information like “crying”, “groan”, and “gun shooting”. To lower the false alarms by abnormal sound detection system, the passive acoustic location device directs the robot to the scene where abnormal events occur and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of master.     

Shadow detection of moving objects based on multisource information in Internet of things

Abstract

Moving object detection is an important part in intelligent video surveillance under the banner of Internet of things. The detection of moving target’s shadow is also an important step in moving object detection. On the accuracy of shadow detection will affect the detection results of the object directly. Based on the variety of shadow detection method, we find that only using one feature can’t make the result of detection accurately. Then we present a new method for shadow detection which contains colour information, the invariance of optical and texture feature. Through the comprehensive analysis of the detecting results of three kinds of information, the shadow was effectively determined. It gets ideal effect in the experiment when combining advantages of various methods.     

融合门控自注意力机制的生成对抗网络视频异常检测

目的 视频异常行为检测是当前智能监控技术的研究热点之一,在社会安防领域具有重要应用。如何通过有效地对视频空间维度信息和时间维度信息建模来提高异常检测的精度仍是目前研究的难点。由于结构优势,生成对抗网络目前广泛应用于视频异常检测任务。针对传统生成对抗网络时空特征利用率低和检测效果差等问题,本文提出一种融合门控自注意力机制的生成对抗网络进行视频异常行为检测。方法 在生成对抗网络的生成网络U-net部分引入门控自注意力机制,逐层对采样过程中的特征图进行权重分配,融合U-net网络和门控自注意力机制的性能优势,抑制输入视频帧中与异常检测任务不相关背景区域的特征表达,突出任务中不同目标对象的相关特征表达,更有效地针对时空维度信息进行建模。采用LiteFlownet网络对视频流中的运动信息进行提取,以保证视频序列之间的连续性。同时,加入强度损失函数、梯度损失函数和运动损失函数加强模型检测的稳定性,以实现对视频异常行为的检测。结果 在CUHK （Chinese University of Hong Kong） Avenue、UCSD （University of California,San Diego） Ped1和UCSD Ped2等视频异常事件数据集上进行实验。在CUHK Avenue数据集中,本文方法的AUC （area under curve）为87.2%,比同类方法高2.3%;在UCSD Ped1和UCSD Ped2数据集中,本文方法的AUC值均高于同类其他方法。同时,设计了4个消融实验并对实验结果进行对比分析,本文方法具有更高的AUC值。结论 实验结果表明,本文方法更适合视频异常检测任务,有效提高了异常行为检测任务模型的稳定性和准确率,且采用视频序列帧间运动信息能够显著提升异常行为检测性能。     

An efficient fire detection method based on orientation feature

This paper proposes a novel method for reliable fire detection. The burning fire usually causes rich moving features in terms of directions, which can offer the best chance to distinguish between the fire region and the non-fire one. Motivated by this observation, we design a novel orientation feature to represent this characteristic. Based on this feature, a method is proposed to detect the fire efficiently. First, fire color is utilized to extract the fire candidate areas from the surveillance video. Then, the direction is obtained by computing the optical flow for each pixel in the candidate area. The directions are discretized to four parts. By counting the percentage of pixels whose moving directions fall into these four parts in a period of time, and combining with the two parameters, i.e., both of the number of frames without the moving directions and the number of consecutive frames in the candidate area, we use these six parameters as the fire orientation feature. In the end, by training a support vector machine (SVM) classifier with the input of our fire orientation feature, the candidate area is judged whether it is a fire. Our main contribution is that we design the novel fire orientation feature. The feature can not only characterize the fire intrinsic dynamic properties accurately but also is very efficient. Compared with the art-of-state methods, the experimental results confirm that our approach significantly improves the accuracy of fire detection and impressively decreases the false alarm rate. The detection speed of our approach is also very competitive with the art-of-state fire detection methods.     

轨迹和多标签超图配对融合的视频复杂事件检测

视频数据中包含丰富的运动事件信息,从中检测复杂事件,分析其中的高层语义信息,已成为视频研究领域的热点之一。视频复杂事件检测,主要对事件中多语义概念进行检测分析,对多运动目标的特征进行描述,发现底层特征与高层语义概念间的关系,旨在从各类视频特征及相关的原始视频数据中自动提取视频复杂事件中语义概念模式,实现“跨越语义鸿沟”的目标。在超图理论的基础上,提出了针对运动目标特征分别构建轨迹超图和多标签超图,并对其进行配对融合,用于检测视频复杂事件。实验结果证明,同其他方法如基于普通图的事件检测方法和基于超图的多标签半监督学习方法相比,新方法在检测复杂事件结果中具有更高的平均查准率和平均查全率。     

基于HEVC的车辆异常事件检测

当前传统交通事故检测和查阅主要通过人工监测的方法,这种方法效率低且实时性差,本文提出一种基于最新压缩域视频编码标准HEVC（High-efficiency video coding）的车辆异常事件检测方法。首先对HEVC码流中提取出的运动矢量信息进行运动矢量累积迭代和中值滤波的预处理,之后根据提取出的块划分信息和运动矢量信息计算运动对象的运动强度,然后根据运动强度值和八连通区域法提取出运动对象,最后根据空间距离法和运动强度判别法检测出视频序列中发生的车辆异常事件。实验证明,该方法可以准确地检测出视频序列中发生的车辆异常事件;对于有着快速移动的运动目标以及多个运动目标的视频效果更好。     

基于TMS320DM8168硬件平台的智能网络视频监控系统

针对传统的模拟视频监控和数字视频监控存在需要专人监视的缺点,提出了一种基于TMS320DM8168硬件平台的智能视频监控解决方案,突出视频监控系统的智能识别、智能控制的特性,实现由专人监控向智能监控的跨越.该课题是以单芯片TMS320DM8168 为核心实现16路D1视频数据的采集、H.264编解码处理、数据存储和传输以及音视频数据回放等基本功能,同时在智能特性方面实现了摄像头的遮挡检测和运动目标检测的功能.     

运动目标三维轨迹可视化与关联分析方法

随着治安监控系统的普及,越来越多的监控摄像头被安装在各个交通道路和公共场所中,每天都产生大量的监控视频.如今,监控视频分析工作主要是采用人工观看的方式来排查异常,以这种方式来分析视频内容耗费大量的人力和时间.目前,关于视频分析方面的研究大多是针对目标个体的异常行为检测和追踪,缺乏针对对象之间的关联关系的分析,对视频中的一些对象和场景之间的关联关系等还没有较为有效的表示和分析方法.针对这一现状,提出一种基于运动目标三维轨迹的关联视频可视分析方法来辅助人工分析视频,首先对视频资料进行预处理,获取各个目标对象的运动轨迹信息,由于二维轨迹难以处理轨迹的自相交、循环运动和停留等现象,并且没有时间信息就难以对同一空间内多个对象轨迹进行的关联性分析,于是结合时间维度对轨迹进行三维化扩展.该方法支持草图交互方式来操作,在分析过程中进行添加草图注释来辅助分析.可结合场景和对象的时空关系对轨迹进行关联性计算,得出对象及场景之间的关联模型,通过对对象在各个场景出现状况的统计,结合人工预先设定的规则,可实现对异常行为报警,辅助用户决策.