An expert video surveillance system to identify and mitigate shoplifting in megastores

Shoplifting has got serious concern because of a steep surge in these types of cases all around. People are found stealing the items from the store without being noticed, either by putting them in bags or hiding objects inside clothes. CCTV cameras are generally installed at any such site, but evidences suggest that these cameras are not very effective unless the video feeds are constantly monitored. Therefore, we intend to build an automated and intelligent surveillance system to catch these shoplifters by identifying their stealing actions. This article proposes a deep neural network-based solution to identify these shoplifting activities. The model proposed uses a dual-stream fusion-based network that effectively binds appearance and motion dynamics in the temporal domain to efficiently identify the shoplifting actions. The deep Inception V3 model is used to extract activity-specific body posture features from video streams through two deep neural network pipelines, one each corresponding to appearance and motion information. Next, a recurrent neural network, namely Long Short Term Memory (LSTM) network, is used to build a temporal relation between features extracted from consecutive frames in order to distinguish human stealing actions accurately. Added to it, this article introduces a shoplifting dataset synthesized in our lab, which contains normal human actions and object stealing actions. The proposed methodology supported with experimental results demonstrates encouraging outcomes with the accuracy achieved up to 91.48%, which outperforms other existing methods.

     

Continuous human action recognition in real time

This paper discusses the task of continuous human action recognition. By continuous, it refers to videos that contain multiple actions which are connected together. This task is important to applications like video surveillance and content based video retrieval. It aims to identify the action category and detect the start and end key frame of each action. It is a challenging task due to the frequent changes of human actions and the ambiguity of action boundaries. In this paper, a novel and efficient continuous action recognition framework is proposed. Our approach is based on the bag of words representation. A visual local pattern is regarded as a word and the action is modeled by the distribution of words. A generative translation and scale invariant probabilistic Latent Semantic Analysis model is presented. The continuous action recognition result is obtained frame by frame and updated from time to time. Experimental results show that this approach is effective and efficient to recognize both isolated actions and continuous actions.     

人群密集度下GAN的视频异常行为检测进展

视频异常检测作为计算机视觉的重要分支,是智能监控系统中一项极具挑战性的任务,通常是指自动识别视频中的异常目标、行为或事件,对保障公共安全起着至关重要的作用。生成对抗网络是一种新兴的无监督方法,不仅可以用于生成图像,且其独特的对抗性学习思想在异常检测领域也显示出良好的发展潜力。介绍了生成对抗网络的框架结构;根据场景密度以及行为发生的对象,从个体行为异常、群体异常两个方面论述了生成对抗网络在视频异常检测领域的研究现状,分别基于重构和预测的方法对个体异常行为检测和群体异常行为检测作进一步阐述;简要介绍了视频异常检测的常用数据集;最后对未来发展作出了展望。     

可变场所的异常行为识别方法

在视觉分析中,人的同一动作在不同场景下会有截然不同的理解.为了判断在不同场景中行为是否为异常,在监控系统中使用双层词包模型来解决这个问题.把视频信息放在第1层包中,把场景动作文本词放在第2层包中.视频由一系列时空兴趣点组成的时空词典表示,动作性质由在指定场景下的动作文本词集合来确定.使用潜在语义分析概率模型(pLSA)不但能自动学习时空词的概率分布,找到与之对应的动作类别,也能在监督情况下学习在规定场景下运动文本词概率分布并区分出对应异常或正常行动结果.经过训练学习后,该算法可以识别新视频在相应场景下行为的异常或正常.     

针对视频监控系统隐蔽式攻击及蜜罐防御

基于视频监控系统网络化和智能化发展带来的风险,研究其隐蔽式网络攻击问题,目的在于调研大量隐蔽式网络攻击案例,总结针对视频监控系统的隐蔽式攻击特异性。结合蜜罐技术在检测网络攻击行为和发现攻击线索等方面的独特优势,梳理针对视频监控系统隐蔽式攻击的蜜罐防御方法。针对监控视频蜜罐在视觉场景部署上的不足,介绍了一种深度场景伪造防御框架,将生成式AI大模型与视频监控蜜罐相结合。最后提出了面向视频监控系统的蜜罐防御技术的发展方向。     

面向智能监控的行为识别

目的 为了进一步提高智能监控场景下行为识别的准确率和时间效率,提出了一种基于YOLO（you only look once：unified,real-time object detection）并结合LSTM（long short-term memory）和CNN（convolutional neural network）的人体行为识别算法LC-YOLO（LSTM and CNN based on YOLO）。方法 利用YOLO目标检测的实时性,首先对监控视频中的特定行为进行即时检测,获取目标大小、位置等信息后进行深度特征提取;然后,去除图像中无关区域的噪声数据;最后,结合LSTM建模处理时间序列,对监控视频中的行为动作序列做出最终的行为判别。结果 在公开行为识别数据集KTH和MSR中的实验表明,各行为平均识别率达到了96.6%,平均识别速度达到215 ms,本文方法在智能监控的行为识别上具有较好效果。结论 提出了一种行为识别算法,实验结果表明算法有效提高了行为识别的实时性和准确率,在实时性要求较高和场景复杂的智能监控中有较好的适应性和广泛的应用前景。     

多媒体技术研究:2013——面向智能视频监控的视觉感知与处理

目的 随着视频监控技术的日益成熟和监控设备的普及,视频监控应用日益广泛,监控视频数据量呈现出爆炸性的增长,已经成为大数据时代的重要数据对象。然而由于视频数据本身的非结构化特性,使得监控视频数据的处理和分析相对困难。面对大量摄像头采集的监控视频大数据,如何有效地按照视频的内容和特性去传输、存储、分析和识别这些数据,已经成为一种迫切的需求。方法 本文面向智能视频监控中大规模视觉感知与智能处理问题,围绕监控视频编码、目标检测与跟踪、监控视频增强、视频运动与异常行为识别等4个主要研究方向,系统阐述2013年度的技术发展状况,并对未来的发展趋势进行展望。结果 中国最新制定的国家标准AVS2在对监控视频的编码效率上比最新国际标准H.265/HEVC高出一倍,标志着我国的视频编码技术和标准在视频监控领域已经实现跨越;视频运动目标检测跟踪的研究主要集中在有效特征提取和分类器训练等方面,机器学习等方法的引入,使得基于多实例学习、稀疏表示的运动目标检测跟踪成为研究的热点;监控视频质量增强主要包括去雾、去夜色、去雨雪、去模糊和超分辨率增强等多方面的内容,现有的算法均是对某类图像清晰化效果较好,而对其他类则相对较差,普适性不高;现有的智能动作分析与异常行为识别技术虽然得到了不断发展,算法的性能也在不断提高,但是从实用角度,除了简单的特定或可控场景外,还没有太多成熟的应用系统。结论 随着大数据时代的到来,智能视频监控的需求将日益迫切,面对众多挑战的同时,该研究领域将迎来前所未有的重大机遇,必将产生越来越多可以实用的研究成果。     

智能视频监控系统异常行为检测算法研究综述

随着公共安全需求的快速增长,监控摄像头数量不断增多,视频监控数据呈爆炸式增长;传统的视频监控系统难以对如此海量的数据进行理解分析,因此智能视频监控系统应运而生;作为一个跨学科的研究领域,智能视频监控系统异常行为检测技术迎来重大机遇的同时也面临不少挑战;为了更好地研究智能视频监控系统异常行为检测算法,梳理了相关研究并从原理上对不同算法进行分类,对基于能量、基于聚类、基于重构、基于推断以及基于深度学习几个不同依据的算法进行对比分析,归纳了各类算法的分支研究方向,接着简要介绍了异常行为检测常用的公开数据集,最后讨论了目前异常行为检测算法所面临的挑战并针对性地提出了未来智能视频监控系统异常行为检测算法的可行研究方向。     

基于手部轨迹识别的ATM智能视频监控系统

为实时阻止针对自动取款机的犯罪行为发生,设计一种基于手部轨迹识别的ATM智能视频监控系统。对于采集所得的监控区域内的视频图像,利用混合高斯背景建模方法为视频图像建立背景模型,通过背景剪除法和跟踪算法得到监控区域内的人体信息,分析进入监控区域的人体面积变化情况,由此判断是否有异常行为发生,存在异常则报警,否则采用基于颜色空间的皮肤检测算法和位置约束检测人手部分,利用隐马尔可夫模型对分段的手部运动轨迹分别进行匹配识别,进一步判断是否存在犯罪行为。实验结果表明,该方法对于犯罪行为的识别率能达到88%。     

城市轨道交通智能视频分析关键技术综述

轨道交通是改善城市公共交通状况的有效途径。随着城市轨道交通的快速建设,人们对城市轨道交通的安全问题越来越重视。智能视频分析技术通过对监控视频流的实时分析,对场景中的各种目标进行检测、分类、跟踪,并分析和判断目标的行为,从而能在异常情况发生时可以及时报警、主动防范,提高处理突发事件的效率。主要研究了智能视频分析技术在轨道交通智能视频监控系统中的应用背景及技术框架,总结了智能视频分析中的关键技术的不同实现方法及其常见的算法。     

Using eye movement data to infer human behavioral intentions

Behavior-directed intentions can be revealed by certain biological signals that precede behaviors. This study used eye movement data to infer human behavioral intentions. Participants were asked to view pictures while operating under different intentions, which necessitated cognitive search and affective appraisal. Intentions regarding the pictures were non-specific or specific, specific intentions were cognitive or affective, and affective intentions were to evaluate either the positive or negative emotions expressed by the individuals depicted. The affective task group made more fixations and had a larger average pupil size than the cognitive task group. The positive appreciation group made more and shorter fixations, on average, than the negative appreciation group. However, support vector machine algorithms revealed low classification accuracy. This was due to large inter-individual variance and psychological factors underlying intentions. We demonstrated improvement in classification accuracy using individual repeated measures data, which helped infer participants’ self-selected intentions.     

Early detection of human actions—A hybrid approach

Early detection of human actions is essential in a wide spectrum of applications ranging from video surveillance to health-care. While human action recognition has been extensively studied, little attention is paid to the problem of detecting ongoing human action early, i.e. detecting an action as soon as it begins, but before it finishes. This study aims at training a detector to be capable of recognizing a human action when only partial action sample is seen. To do so, a hybrid technique is proposed in this work which combines the benefits of computer vision as well as fuzzy set theory based on the fuzzy Bandler and Kohout's sub-triangle product (BK subproduct). The novelty lies in the construction of a frame-by-frame membership function for each kind of possible movement. Detection is triggered when a pre-defined threshold is reached in a suitable way. Experimental results on a publicly available dataset demonstrate the benefits and effectiveness of the proposed method.     

Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models

A challenging problem in human action understanding is to jointly segment and recognize human actions from an unseen video sequence, where one person performs a sequence of continuous actions.     

基于智能监控的中小人群异常行为检测

针对人群异常行为检测实时性较差、分类算法识别率不高、特征量较少的问题,提出一种基于智能监控的中小人群异常行为检测算法。首先,利用快速群体密度检测算法,提取人群数量变化信息;其次,利用改进的Lucas-Kanande光流法提取视频中人群的平均动能、人群方向熵、人群距离势能;最后,利用极限学习机(ELM)算法对人群行为进行分类。使用UMN公共数据集进行测试,ELM算法对中小人群异常行为分析比中高密度人群异常行为检测算法和基于KOD能量特征的群体异常行为检测算法识别率分别高出7.13个百分点和5.89个百分点,并且人数密度估计部分平均每帧图像处理耗时相比中高密度人群异常行为检测算法减少了106 ms(近1/3)。实验结果表明:基于智能监控的中小人群异常行为检测算法能有效提高异常帧识别率和实时性。     

New framework that uses patterns and relations to understand terrorist behaviors

Terrorism is a complex phenomenon with high uncertainties in user strategy. The uncertain nature of terrorism is a main challenge in the design of counter-terrorism policy. Government agencies (e.g., CIA, FBI, NSA, etc.) cannot always use social media and telecommunications to capture the intentions of terrorists because terrorists are very careful in the use of these environments to plan and prepare attacks. To address this issue, this research aims to propose a new framework by defining the useful patterns of suicide attacks to analyze the terrorist activity patterns and relations, to understand behaviors and their future moves, and finally to prevent potential terrorist attacks. In the framework, a new network model is formed, and the structure of the relations is analyzed to infer knowledge about terrorist attacks. More specifically, an Evolutionary Simulating Annealing Lasso Logistic Regression (ESALLOR) model is proposed to select key features for similarity function. Subsequently, a new weighted heterogeneous similarity function is proposed to estimate the relationships among attacks. Moreover, a graph-based outbreak detection is proposed to define hazardous places for the outbreak of violence. Experimental results demonstrate the effectiveness of our framework with high accuracy (more than 90% accuracy) for finding patterns when compared with that of actual terrorism events in 2014 and 2015. In conclusion, by using this intelligent framework, governments can understand automatically how terrorism will impact future events, and governments can control terrorists’ behaviors and tactics to reduce the risk of future events.     

Human body segmentation based on shape constraint

Human body segmentation is essential for many practical applications, e.g., video surveillance analysis in intelligent urban. However, existing methods mainly suffer from various human poses. In this paper, we try to address this issue by introducing human shape constraint. First, human pose estimation is performed, and locations of human body parts are determined. Contrast to the previous work, we just use the human body parts with high precision. Then we combines the star convexity and the human body parts’ locations as shape constraint. The final segmentation results are acquired through the optimization step. Comprehensive and comparative experimental results demonstrate that the proposed method achieves promising performance and outperforms many state-of-the-art methods over publicly available challenging datasets.     

Suspicious human activity recognition: a review

Suspicious human activity recognition from surveillance video is an active research area of image processing and computer vision. Through the visual surveillance, human activities can be monitored in sensitive and public areas such as bus stations, railway stations, airports, banks, shopping malls, school and colleges, parking lots, roads, etc. to prevent terrorism, theft, accidents and illegal parking, vandalism, fighting, chain snatching, crime and other suspicious activities. It is very difficult to watch public places continuously, therefore an intelligent video surveillance is required that can monitor the human activities in real-time and categorize them as usual and unusual activities; and can generate an alert. Recent decade witnessed a good number of publications in the field of visual surveillance to recognize the abnormal activities. Furthermore, a few surveys can be seen in the literature for the different abnormal activities recognition; but none of them have addressed different abnormal activities in a review. In this paper, we present the state-of-the-art which demonstrates the overall progress of suspicious activity recognition from the surveillance videos in the last decade. We include a brief introduction of the suspicious human activity recognition with its issues and challenges. This paper consists of six abnormal activities such as abandoned object detection, theft detection, fall detection, accidents and illegal parking detection on road, violence activity detection, and fire detection. In general, we have discussed all the steps those have been followed to recognize the human activity from the surveillance videos in the literature; such as foreground object extraction, object detection based on tracking or non-tracking methods, feature extraction, classification; activity analysis and recognition. The objective of this paper is to provide the literature review of six different suspicious activity recognition systems with its general framework to the researchers of this field.