Activity based surveillance video content modelling

This paper tackles the problem of surveillance video content modelling. Given a set of surveillance videos, the aims of our work are twofold: firstly a continuous video is segmented according to the activities captured in the video; secondly a model is constructed for the video content, based on which an unseen activity pattern can be recognised and any unusual activities can be detected. To segment a video based on activity, we propose a semantically meaningful video content representation method and two segmentation algorithms, one being offline offering high accuracy in segmentation, and the other being online enabling real-time performance. Our video content representation method is based on automatically detected visual events (i.e. ‘what is happening in the scene’). This is in contrast to most previous approaches which represent video content at the signal level using image features such as colour, motion and texture. Our segmentation algorithms are based on detecting breakpoints on a high-dimensional video content trajectory. This differs from most previous approaches which are based on shot change detection and shot grouping. Having segmented continuous surveillance videos based on activity, the activity patterns contained in the video segments are grouped into activity classes and a composite video content model is constructed which is capable of generalising from a small training set to accommodate variations in unseen activity patterns. A run-time accumulative unusual activity measure is introduced to detect unusual behaviour while usual activity patterns are recognised based on an online likelihood ratio test (LRT) method. This ensures robust and reliable activity recognition and unusual activity detection at the shortest possible time once sufficient visual evidence has become available. Comparative experiments have been carried out using over 10 h of challenging outdoor surveillance video footages to evaluate the proposed segmentation algorithms and modelling approach.     

On verification of nested workflows with extra constraints: From theory to practice

Workflows are used to formally describe processes of various types such as business and manufacturing processes. One of the critical tasks of workflow management is automated discovery of possible flaws in the workflow – workflow verification. In this paper, we formalize the problem of workflow verification as the problem of verifying that there exists a feasible process for each task in the workflow. This problem is tractable for nested workflows that are the workflows with a hierarchical structure similar to hierarchical task networks in planning. However, we show that if extra synchronization, precedence, or causal constraints are added to the nested structure, the workflow verification problem becomes NP-complete. We present a workflow verification algorithm for nested workflows with extra constraints that is based on constraint satisfaction techniques and exploits an incremental temporal reasoning algorithm. We then experimentally demonstrate efficiency of the proposed techniques on randomly generated workflows with various structures and sizes. The paper is concluded by notes on exploiting the presented techniques in the application FlowOpt for modeling, optimizing, visualizing, and analyzing production workflows.     

Early fire detection using non-linear multitemporal prediction of thermal imagery

This paper presents a sub-pixel thermal anomaly detection method based on predicting background pixel intensities using a non-linear function of a plurality of past images of the inspected scene. At present, the multitemporal approach to thermal anomaly detection is in its early development stage. In case of space-borne surveillance the multitemporal detection is complicated by both spatial and temporal variability of background surface properties, weather influences, viewing geometries, sensor noise, residual misregistration, and other factors. We use the problem of fire detection and the MODIS data to demonstrate that advanced multitemporal detection methods can potentially outperform the operationally used optimized contextual algorithms both under morning and evening conditions.     

面向遮挡与透视影响的道路凸包梯形检测算法

以安防监控视频下受遮挡与透视影响的道路作为研究对象,针对行人、行车对道路边界线的遮挡造成的消失点难以检测和实例分割欠分割难题,提出了一种改进的实例分割道路检测方法。首先使用实例分割对道路区域进行提取,然后通过凸包算法补偿行人、行车对道路边界线的遮挡,最后将其拟合成符合场景结构的梯形模型,从而优化对道路的检测。实验结果表明,该方法解决了道路受遮挡与透视影响而导致检测不准确的问题,可以满足安防监控视频下道路检测的需求,具有实际意义。     

An appearance-based approach for consistent labeling of humans and objects in video

In this paper, we present an approach for consistently labeling people and for detecting human–object interactions using mono-camera surveillance video. The approach is based on a robust appearance-based correlogram model combined with histogram information to model color distributions of people and objects in the scene. The models are dynamically built from non-stationary objects, which are the outputs of background subtraction, and are used to identify objects on a frame-by-frame basis. We are able to detect when people merge into groups and to segment them even during partial occlusion. We can also detect when a person deposits or removes an object. The models persist when a person or object leaves the scene and are used to identify them when they reappear. Experiments show that the models are able to accommodate perspective foreshortening that occurs with overhead camera angles, as well as partial occlusion. The results show that this is an effective approach that is able to provide important information to algorithms performing higher-level analysis, such as activity recognition, where human–object interactions play an important role.     

An Effective Cloud Workflow Scheduling Approach Combining PSO and Idle Time Slot-Aware Rules

Workflow scheduling is a key issue and remains a challenging problem in cloud computing.Faced with the large number of virtual machine(VM)types offered by cloud providers,cloud users need to choose the most appropriate VM type for each task.Multiple task scheduling sequences exist in a workflow application.Different task scheduling sequences have a significant impact on the scheduling performance.It is not easy to determine the most appropriate set of VM types for tasks and the best task scheduling sequence.Besides,the idle time slots on VM instances should be used fully to increase resources'utilization and save the execution cost of a workflow.This paper considers these three aspects simultaneously and proposes a cloud workflow scheduling approach which combines particle swarm optimization(PSO)and idle time slot-aware rules,to minimize the execution cost of a workflow application under a deadline constraint.A new particle encoding is devised to represent the VM type required by each task and the scheduling sequence of tasks.An idle time slot-aware decoding procedure is proposed to decode a particle into a scheduling solution.To handle tasks'invalid priorities caused by the randomness of PSO,a repair method is used to repair those priorities to produce valid task scheduling sequences.The proposed approach is compared with state-of-the-art cloud workflow scheduling algorithms.Experiments show that the proposed approach outperforms the comparative algorithms in terms of both of the execution cost and the success rate in meeting the deadline.     

Shadow detection for moving objects based on texture analysis

This paper presents a new approach for shadow detection of moving objects in visual surveillance environment, improving localization, segmentation, tracking and classification of detected objects. An automatic segmentation procedure based on adaptive background difference is performed in order to detect potential shadow points so that, for all moving pixels, the approach evaluates the compatibility of photometric properties with shadow characteristics. The shadow detection approach is improved by evaluating the similarity between little textured patches, since shadow regions present same textural characteristics in each frame and in the corresponding adaptive background model. In this work we suggest a new approach to describe textural information in terms of redundant systems of functions. The algorithm is designed to be unaffected by scene type, background type or light conditions. Experimental results validate the algorithm's performance on a benchmark suite of indoor and outdoor video sequences.     

3D measures exploitation for a monocular semi-supervised fall detection system

Falls have been reported as the leading cause of injury-related visits to emergency departments and the primary etiology of accidental deaths in elderly. Thus, the development of robust home surveillance systems is of great importance. In this article, such a system is presented, which tries to address the fall detection problem through visual cues. The proposed methodology utilizes a fast, real-time background subtraction algorithm, based on motion information in the scene and pixels intensity, capable to operate properly in dynamically changing visual conditions, in order to detect the foreground object. At the same time, it exploits 3D space’s measures, through automatic camera calibration, to increase the robustness of fall detection algorithm which is based on semi-supervised learning approach. The above system uses a single monocular camera and is characterized by minimal computational cost and memory requirements that make it suitable for real-time large scale implementations.     

Space-Time Domain Expansion Approach to VLSI and Its Application to Hierarchical Scene Matching

VLSI technology has recently received increasing attention due to its high performance and high reliability. Designing a VLSI structure systematically for a given task becomes a very important problem to many computer engineers. In this paper, we present a method to transform a recursive computation task into a VLSI structure systematically. The main advantages of this approach are its simplicity and completeness. Several examples, such as vector inner product, matrix multiplication, convolution, comparison operations in relational database and fast Fourier transformation (FFT), are given to demonstrate the transformation procedure. Finally, we apply the proposed method to hierarchical scene matching. Scene matching refers to the process of locating or matching a region of an image with a corresponding region of another view of the same image taken from a different viewing angle or at a different time. We first present a constant threshold estimation for hierarchical scene matching. The VLSI implementation of the hierarchical scene matching is then described in detail.     

Hierarchical abnormal event detection by real time and semi-real time multi-tasking video surveillance system

In this paper, we describe how to detect abnormal human activities taking place in an outdoor surveillance environment. Human tracks are provided in real time by the baseline video surveillance system. Given trajectory information, the event analysis module will attempt to determine whether or not a suspicious activity is currently being observed. However, due to real-time processing constrains, there might be false alarms generated by video image noise or non-human objects. It requires further intensive examination to filter out false event detections which can be processed in an off-line fashion. We propose a hierarchical abnormal event detection system that takes care of real time and semi-real time as multi-tasking. In low level task, a trajectory-based method processes trajectory data and detects abnormal events in real time. In high level task, an intensive video analysis algorithm checks whether the detected abnormal event is triggered by actual humans or not.     

Multilayer background modeling under occlusions

A multilayer background modeling technique is presented for video surveillance. Rather than simply classifying all features in a scene as either dynamically moving foreground or long-lasting, stationary background, a temporal model is used to place each scene object in time relative to each other. Foreground objects that become stationary are registered as layers on top of the background layer. In this process of layer formation, the algorithm deals with ”fake objects” created by moved background, and noise created by dynamic background and moving foreground objects. Objects that leave the scene are removed based on the occlusion reasoning among layers. The technique allows us to understand and visualize a scene with multiple objects entering, leaving, and occluding each other at different points in time. This scene understanding leads to a richer representation of temporal scene events than traditional foreground/background segmentation. The technique builds on a low-cost background modeling technique that makes it suitable for embedded, real-time platforms.     

Detection of swimmer using dense optical flow motion map and intensity information

A vision-based system that can locate individual swimmers and recognize the activities is applicable for swimming gait analysis, drowning event detection, etc. The system relies on accurate detection of swimmer’s body parts such as head and upper limbs. The swimmer detection problem can be regarded as background subtraction. Swimmer detection in the aquatic environment is very difficult due to a dynamic background with water ripples, splashes, specular reflections, etc. This paper presents a swimmer detection method which utilizes both local motion and intensity information estimated from the image sequence. Local motion information is obtained by computing dense optical flow and periodogram. We adopt a heuristic approach to generate a motion map characterizing the local motions (random/stationary, ripple or swimming) of image pixels over a short duration. Intensity information is modeled as a mixture of Gaussians. Finally, using the motion map and the Gaussian models, swimmers are detected in each video frame. We test the method on video sequences captured at daytime, and nighttime, and of different swimming styles (breaststroke, freestyle, backstroke). Our method can detect swimmers much better than that using intensity information alone. In addition, we compare our method with existing algorithms—codebook model and self-organizing artificial neural networks. The methods are tested on publicly available video sequence and our swimming video sequence. We show through the quantitative measures the superiority of our method.     

Abnormal visual event detection based on multi-instance learning and autoregressive integrated moving average model in edge-based Smart City surveillance

The abnormal visual event detection is an important subject in Smart City surveillance where a lot of data can be processed locally in edge computing environment. Real-time and detection effectiveness are critical in such an edge environment. In this paper, we propose an abnormal event detection approach based on multi-instance learning and autoregressive integrated moving average model for video surveillance of crowded scenes in urban public places, focusing on real-time and detection effectiveness. We propose an unsupervised method for abnormal event detection by combining multi-instance visual feature selection and the autoregressive integrated moving average model. In the proposed method, each video clip is modeled as a visual feature bag containing several subvideo clips, each of which is regarded as an instance. The time-transform characteristics of the optical flow characteristics within each subvideo clip are considered as a visual feature instance, and time-series modeling is carried out for multiple visual feature instances related to all subvideo clips in a surveillance video clip. The abnormal events in each surveillance video clip are detected using the multi-instance fusion method. This approach is verified on publically available urban surveillance video datasets and compared with state-of-the-art alternatives. Experimental results demonstrate that the proposed method has better abnormal event detection performance for crowded scene of urban public places with an edge environment.     

基于混合GA算法的工作流作业调度队列优化

工作流作业的调度效率是评价工作流管理系统整体表现的重要指标。众所周知,工作流作业的调度问题是一个NP-hard问题,而异构的计算环境使得问题更加棘手。分层基因算法LGA将启发式算法与GA算法相结合,利用GA算法来优化经过正向分层之后的工作流作业调度队列,显著地减少了工作流作业的执行时间。该算法根据作业的分层优先级来产生作业队列,把队列中的同层作业从整体上看作是一位基因来处理,有效地对算法的进化方向进行规划,并通过对杂交和变异流程的改进,增强算法的搜索深度和广度。实验表明,相比于其他混合GA算法,经LGA算法优化之后的工作流作业调度队列,所需的执行时间更少。     

基于OGSA网格的分层式网格任务调度器设计

文章根据网格任务调度的需求、网格任务调度的特点,在充分分析一般网格任务调度的过程等的基础上,另外考虑到了网格计算环境的一些特点,比如虚拟化、分层次及自治的本质特征,以及在工作流任务协同需求下网格任务的资源依赖、粗粒度、重复执行等特性的前提下,改进设计了一种网格工作流任务主从式分层调度模型,并给出了调度策略和调度算法实现。该调度器模型在实际的网格工作流任务协同系统中得到了较好的应用效果。     

Moving people tracking with detection by latent semantic analysis for visual surveillance applications

The latent semantic analysis (LSA) has been widely used in the fields of computer vision and pattern recognition. Most of the existing works based on LSA focus on behavior recognition and motion classification. In the applications of visual surveillance, accurate tracking of the moving people in surveillance scenes, is regarded as one of the preliminary requirement for other tasks such as object recognition or segmentation. However, accurate tracking is extremely hard under challenging surveillance scenes where similarity among multiple objects or occlusion among multiple objects occurs. Usual temporal Markov chain based tracking algorithms suffer from the ‘tracking error accumulation problem’. The accumulated errors can finally make the tracking to drift from the target. To handle the problem of tracking drift, some authors have proposed the idea of using detection along with tracking as an effective solution. However, many of the critical issues still remain unsettled in these detection based tracking algorithms. In this paper, we propose a novel moving people tracking with detection based on (probabilistic) LSA. By employing a novel ‘twin-pipeline’ training framework to find the latent semantic topics of ‘moving people’, the proposed detection can effectively detect the interest points on moving people in different indoor and outdoor environments with camera motion. Since the detected interest points on different body parts can be used to locate the position of moving people more accurately, by combining the detection with incremental subspace learning based tracking, the proposed algorithms resolves the problem of tracking drift during each target appearance update process. In addition, due to the time independent processing mechanism of detection, the proposed method is also able to handle the error accumulation problem. The detection can calibrate the tracking errors during updating of each state of the tracking algorithm. Extensive, experiments on various surveillance environments using different benchmark datasets have proved the accuracy and robustness of the proposed tracking algorithm. Further, the experimental comparison results clearly show that the proposed tracking algorithm outperforms the well known tracking algorithms such as ISL, AMS and WSL algorithms. Furthermore, the speed performance of the proposed method is also satisfactory for realistic surveillance applications.     

A novel framework for intelligent surveillance system based on abnormal human activity detection in academic environments

Abnormal activity detection plays a crucial role in surveillance applications, and a surveillance system that can perform robustly in an academic environment has become an urgent need. In this paper, we propose a novel framework for an automatic real-time video-based surveillance system which can simultaneously perform the tracking, semantic scene learning, and abnormality detection in an academic environment. To develop our system, we have divided the work into three phases: preprocessing phase, abnormal human activity detection phase, and content-based image retrieval phase. For motion object detection, we used the temporal-differencing algorithm and then located the motions region using the Gaussian function. Furthermore, the shape model based on OMEGA equation was used as a filter for the detected objects (i.e., human and non-human). For object activities analysis, we evaluated and analyzed the human activities of the detected objects. We classified the human activities into two groups: normal activities and abnormal activities based on the support vector machine. The machine then provides an automatic warning in case of abnormal human activities. It also embeds a method to retrieve the detected object from the database for object recognition and identification using content-based image retrieval. Finally, a software-based simulation using MATLAB was performed and the results of the conducted experiments showed an excellent surveillance system that can simultaneously perform the tracking, semantic scene learning, and abnormality detection in an academic environment with no human intervention.

     

A Real-Time Algorithm for Small Group Detection in Medium Density Crowds

In this paper, we focus on the task of small group detection in crowded scenarios. Small groups are widely considered as one of the basic elements in crowds, so it is a major challenge to distinguish group members from the individuals in the crowd. It is also a basic problem in video surveillance and scene understanding. We propose a solution for this task, which could run in real time and could work in both low and medium density crowded scenes. In particular, we build a social force based collision avoidance model on each individual for goal direction prediction, and employ the predicted goal directions instead of traditional positions and velocities in collective motion detection to find group members. We evaluate our approach over three datasets including tens of challenging crowded scenarios. The experimental results demonstrate that our proposed approach is not only highly accurate but also improves the practical property performance compared to other state-of-the-art methods.     

基于自适应背景模型运动目标检测

随着城市化速度的加快,机动车日益普及,人们在享受机动车所带来的巨大便利的同时,也面临着交通拥挤的困扰。随着计算机硬件技术和计算机视觉技术的发展,基于计算机视觉的交通监控系统成为可能。从一个交通视频序列中识别出运动物体是许多交通监控系统应用系统的重要任务,针对该问题,提出了一种建立在对视频序列中的整个背景情景的统计描述基础上的运动目标的检测的有效方法,该方法能够适应变化的背景,具有较强的鲁棒性和较好的实时性。