Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space

This paper proposes a novel technique for clustering and classification of object trajectory-based video motion clips using spatiotemporal function approximations. Assuming the clusters of trajectory points are distributed normally in the coefficient feature space, we propose a Mahalanobis classifier for the detection of anomalous trajectories. Motion trajectories are considered as time series and modelled using orthogonal basis function representations. We have compared three different function approximations – least squares polynomials, Chebyshev polynomials and Fourier series obtained by Discrete Fourier Transform (DFT). Trajectory clustering is then carried out in the chosen coefficient feature space to discover patterns of similar object motions. The coefficients of the basis functions are used as input feature vectors to a Self- Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Our proposed techniques are validated on three different datasets – Australian sign language, hand-labelled object trajectories from video surveillance footage and real-time tracking data obtained in the laboratory. Applications to event detection and motion data mining for multimedia video surveillance systems are envisaged.     

A simplified multi-class support vector machine with reduced dual optimization

Support vector machine (SVM) was initially designed for binary classification. To extend SVM to the multi-class scenario, a number of classification models were proposed such as the one by Crammer and Singer (2001). However, the number of variables in Crammer and Singer’s dual problem is the product of the number of samples (l) by the number of classes (k), which produces a large computational complexity. This paper presents a simplified multi-class SVM (SimMSVM) that reduces the size of the resulting dual problem from l × k to l by introducing a relaxed classification error bound. The experimental results demonstrate that the proposed SimMSVM approach can greatly speed-up the training process, while maintaining a competitive classification accuracy.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

智能视频监控系统结构分析

视频智能监控系统是一套运用于公共场合,能够实现实时记录与数据分析。该系统可以实现普通视频监控系统中的视频数据记录功能,并且解决了人为视频监控中无法实时判断特征信息的功能缺陷。     

Activity classification and anomaly detection using m-mediods based modelling of motion patterns

Techniques for video object motion analysis, behaviour recognition and event detection are becoming increasingly important with the rapid increase in demand for and deployment of video surveillance systems. Motion trajectories provide rich spatiotemporal information about an object's activity. This paper presents a novel technique for classification of motion activity and anomaly detection using object motion trajectory. In the proposed motion learning system, trajectories are treated as time series and modelled using modified DFT-based coefficient feature space representation. A modelling technique, referred to as m-mediods, is proposed that models the class containing n members with m mediods. Once the m-mediods based model for all the classes have been learnt, the classification of new trajectories and anomaly detection can be performed by checking the closeness of said trajectory to the models of known classes. A mechanism based on agglomerative approach is proposed for anomaly detection. Four anomaly detection algorithms using m-mediods based representation of classes are proposed. These includes: (i)global merged anomaly detection (GMAD), (ii) localized merged anomaly detection (LMAD), (iii) global un-merged anomaly detection (GUAD), and (iv) localized un-merged anomaly detection (LUAD). Our proposed techniques are validated using variety of simulated and complex real life trajectory datasets.     

Real-time stereo matching using memory-efficient Belief Propagation for high-definition 3D telepresence systems

New generations of telecommunications systems will include high-definition 3D video that provides a telepresence feeling. These systems require high-quality depth maps to be generated in a very short time (very low latency, typically about 40 ms). Classical Belief Propagation algorithms (BP) generate high-quality depth maps but they require huge memory bandwidths that limit low-latency implementations of stereo-vision systems with high-definition images.This paper proposes a real-time (latency inferior to 40 ms) high-definition (1280 × 720) stereo matching algorithm using Belief Propagation with good immersive feeling (80 disparity levels). There are two main contributions. The first is an improved BP algorithm with pixel classification that outperforms classical BP while reducing the number of memory accesses. The second is an adaptive message compression technique with a low performance penalty that greatly reduces the memory traffic. The combination of these techniques outperforms classical BP by about 6.0% while reducing the memory traffic by more than 90%.     

基于SVM的多类别轨迹的分类方法

轨迹分析是解决视觉监控系统中异常检测问题的重要途径.文章将对轨迹进行采样得到的坐标点集作为特征向量,利用SVM训练分类器,并采用一对一算法实现多类别轨迹的分类.实验结果表明,该方法能够满足SVM中核函数对于输入数据的要求,并实现对多类别轨迹的有效分类.     

Unsupervised and simultaneous training of multiple object detectors from unlabeled surveillance video

Object detection is an essential component in automated vision-based surveillance systems. In general, object detectors are constructed using training examples obtained from large annotated data sets. The inevitable limitations of typical training data sets make such supervised methods unsuitable for building generic surveillance systems applicable to a wide variety of scenes and camera setups. In our previous work we proposed an unsupervised method for learning and detecting the dominant object class in a general dynamic scene observed by a static camera. In this paper, we investigate the possibilities to expand the applicability of this method to the problem of multiple dominant object classes. We propose an idea on how to approach this expansion, and perform an evaluation of this idea using two representative surveillance video sequences.     

A study on various methods used for video summarization and moving object detection for video surveillance applications

With the advancement in digital video technology, video surveillance has been playing its vital role for ensuring safety and security. The surveillance systems are deployed in wide range of applications to invigilate stuffs and to analyse the activities in the environment. From the single or multi surveillance camera, a huge amount of data is generated, stored and processed for security purpose. Due to time constraints, it is a very tedious process for an analyst to go through the full content. This limitation has been overcome by the use of video summarization. The video summarization is intended to afford comprehensible analysis of video by removing duplications and extracting key frames from the video. To make an easily interpreted outline, the various available video summarization methods will try to shot the summary of the main occurrences, scenes, or objects in a frame. Depending on the applications, it is required to summarize the happenings in the scene and detect the objects (static/dynamic) which is recorded in the video. Hence this paper provides the various methods used for video summarization and a comparative study of different techniques. It also presents different object detection, object classification and object tracking algorithms available in the literature.     

Suspicious human activity recognition: a review

Suspicious human activity recognition from surveillance video is an active research area of image processing and computer vision. Through the visual surveillance, human activities can be monitored in sensitive and public areas such as bus stations, railway stations, airports, banks, shopping malls, school and colleges, parking lots, roads, etc. to prevent terrorism, theft, accidents and illegal parking, vandalism, fighting, chain snatching, crime and other suspicious activities. It is very difficult to watch public places continuously, therefore an intelligent video surveillance is required that can monitor the human activities in real-time and categorize them as usual and unusual activities; and can generate an alert. Recent decade witnessed a good number of publications in the field of visual surveillance to recognize the abnormal activities. Furthermore, a few surveys can be seen in the literature for the different abnormal activities recognition; but none of them have addressed different abnormal activities in a review. In this paper, we present the state-of-the-art which demonstrates the overall progress of suspicious activity recognition from the surveillance videos in the last decade. We include a brief introduction of the suspicious human activity recognition with its issues and challenges. This paper consists of six abnormal activities such as abandoned object detection, theft detection, fall detection, accidents and illegal parking detection on road, violence activity detection, and fire detection. In general, we have discussed all the steps those have been followed to recognize the human activity from the surveillance videos in the literature; such as foreground object extraction, object detection based on tracking or non-tracking methods, feature extraction, classification; activity analysis and recognition. The objective of this paper is to provide the literature review of six different suspicious activity recognition systems with its general framework to the researchers of this field.     

Directional Prediction CamShift algorithm based on Adaptive Search Pattern for moving object tracking

Moving object tracking is a fundamental task on smart video surveillance systems, because it provides a focus of attention for further investigation. Continuously Adaptive MeanShift (CamShift) algorithm is an adaptation of the MeanShift algorithm for moving objects tracking significantly, and it has been attracting increasing interests in recent years. In this work, a new CamShift approach, Directional Prediction CamShift (DP-CamShift) algorithm, is proposed to improve the tracking accuracy rate. According to the characteristic of the center-based motion vector distribution for the real-world video sequence, this work employs an Adaptive Search Pattern (ASP) to refine the central area search. The proposed approach is more robust because it adapts the optimal search pattern methods for the most adequate direction of the moving target. Since the fast Motion Estimation (ME) method has its own moving direction feature, we can adaptively use the most proper fast ME method to the certain moving object to have the best performance. Furthermore for estimation in large motion situations, the strategy of the DP-CamShift can preserve good performance. For the test video sequences with frame size of 320 × 240, the experimental results indicate that the proposed algorithm can have an accuracy rate of 99 % and achieve 23 frames per second (FPS) processing speed.     

Background modeling using Object-based Selective Updating and Correntropy adaptation

Background modeling is widely used in visual surveillance systems aiming to facilitate analysis of real-world video scenes. The goal is to discriminate between pixels from foreground objects and those ones from the background. However, real-world scenarios tend to have time and spatial non-stationary variations, being difficult to reveal the foreground and background entities from video data. Here, we propose a novel adaptive background modeling, termed Object-based Selective Updating with Correntropy (OSUC), to support video-based surveillance systems. Our approach that is developed within an adaptive learning framework unveils existing spatio-temporal pixel relationships, making use of a single Gaussian for the model representation stage. Moreover, we introduce a background updating scheme composed of an updating rule that is based on the stochastic gradient algorithm and Correntropy cost function. As a result, this scheme can extract the temporal statistical pixel distribution, at the same time, dealing with non-stationary pixel value fluctuations that affect the background model. Here, an automatic tuning strategy of the cost function bandwidth parameter is carried out that can handle both Gaussian and non-Gaussian noise environments. Besides, to include pixel spatial relationships in the background modeling processing, we introduce an object-based selective learning rate strategy for enhancing the background modeling accuracy. Particularly, an object motion analysis stage is presented to detect and track foreground entities based on pixel intensities and motion direction attained via optical flow computation. Testing is provided on well-known datasets for discriminating between foreground and background that include stationary and non-stationary behaviors. Achieved results show that the OSUC outperforms, in most of the considered cases, the-state-of-the-art approaches with an affordable computational cost. Therefore, the proposed approach is suitable for supporting real-world video-based surveillance systems.     

Object-based analysis and change detection of major wetland cover types and their classification uncertainty during the low water period at Poyang Lake, China

Productive wetland systems at land-water interfaces that provide unique ecosystem services are challenging to study because of water dynamics, complex surface cover and constrained field access. We applied object-based image analysis and supervised classification to four 32-m Beijing-1 microsatellite images to examine broad-scale surface cover composition and its change during November 2007-March 2008 low water season at Poyang Lake, the largest freshwater lake-wetland system in China (> 4000 km²). We proposed a novel method for semi-automated selection of training objects in this heterogeneous landscape using extreme values of spectral indices (SIs) estimated from satellite data. Dynamics of the major wetland cover types (Water, Mudflat, Vegetation and Sand) were investigated both as transitions among primary classes based on maximum membership value, and as changes in memberships to all classes even under no change in a primary class. Fuzzy classification accuracy was evaluated as match frequencies between classification outcome and a) the best reference candidate class (MAX function) and b) any acceptable reference class (RIGHT function). MAX-based accuracy was relatively high for Vegetation (≥ 90%), Water (≥ 82%), Mudflat (≥ 76%) and the smallest-area Sand (≥ 75%) in all scenes; these scores improved with the RIGHT function to 87-100%. Classification uncertainty assessed as the proportion of fuzzy object area within a class at a given fuzzy threshold value was the highest for all classes in November 2007, and consistently higher for Mudflat than for other classes in all scenes. Vegetation was the dominant class in all scenes, occupying 41.2-49.3% of the study area. Object memberships to Vegetation mostly declined from November 2007 to February 2008 and increased substantially only in February-March 2008, possibly reflecting growing season conditions and grazing. Spatial extent of Water both declined and increased during the study period, reflecting precipitation and hydrological events. The “fuzziest” Mudflat class was involved in major detected transitions among classes and declined in classification accuracy by March 2008, representing a key target for finer-scale research. Future work should introduce Vegetation sub-classes reflecting differences in phenology and alternative methods to discriminate Mudflat from other classes. Results can be used to guide field sampling and top-down landscape analyses in this wetland.     

Maximum margin multiple-instance feature weighting

Feature weighting is of considerable importance in machine learning due to its effectiveness to highlight relevant components and suppress irrelevant ones. In this paper, we focus on the feature weighting problem in a specific machine learning area: multiple-instance learning, and propose maximum margin multiple-instance feature weighting (M3IFW) to seek large classification margins in the weighted feature space. The designed M3IFW algorithm can be applied to both standard binary-class multiple-instance learning and the corresponding multi-class learning, and we abbreviate them to B-M3IFW (binary-class M3IFW) and M-M3IFW (multi-class M3IFW), respectively. Both B-M3IFW and M-M3IFW contain three kinds of unknown variables, i.e., positive prototypes, classification margins, and weighting coefficients. We utilize the coordinate ascent algorithm to update the three kinds of unknown variables, respectively and iteratively, and then perform classifications in the weighted feature space. Experiments conducted on synthetic and real-world datasets empirically demonstrate the effectiveness of M3IFW in improving classification accuracies.     

An efficient orientation distance–based discriminative feature extraction method for multi-classification

Feature extraction is an important step before actual learning. Although many feature extraction methods have been proposed for clustering, classification and regression, very limited work has been done on multi-class classification problems. This paper proposes a novel feature extraction method, called orientation distance–based discriminative (ODD) feature extraction, particularly designed for multi-class classification problems. Our proposed method works in two steps. In the first step, we extend the Fisher Discriminant idea to determine an appropriate kernel function and map the input data with all classes into a feature space where the classes of the data are well separated. In the second step, we put forward two variants of ODD features, i.e., one-vs-all-based ODD and one-vs-one-based ODD features. We first construct hyper-plane (SVM) based on one-vs-all scheme or one-vs-one scheme in the feature space; we then extract one-vs-all-based or one-vs-one-based ODD features between a sample and each hyper-plane. These newly extracted ODD features are treated as the representative features and are thereafter used in the subsequent classification phase. Extensive experiments have been conducted to investigate the performance of one-vs-all-based and one-vs-one-based ODD features for multi-class classification. The statistical results show that the classification accuracy based on ODD features outperforms that of the state-of-the-art feature extraction methods.     

Motion-based behaviour learning, profiling and classification in the presence of anomalies

Techniques for understanding video object motion activity are becoming increasingly important with the widespread adoption of CCTV surveillance systems. Motion trajectories provide rich spatiotemporal information about an object's activity. This paper presents a novel technique for clustering and classification of motion. In the proposed motion learning system, trajectories are treated as time series and modelled using modified DFT (discrete fourier transform)-based coefficient feature space representation. A framework (iterative HSACT-LVQ (hierarchical semi-agglomerative clustering-learning vector quantization)) is proposed for learning of patterns in the presence of significant number of anomalies in training data. A novel modelling technique, referred to as m-Mediods, is also proposed that models the class containing n members with m Mediods. Once the m-Mediods-based model for all the classes have been learnt, the classification of new trajectories and anomaly detection can be performed by checking the closeness of said trajectory to the models of known classes. A mechanism based on agglomerative approach is proposed for anomaly detection. Our proposed techniques are validated using variety of simulated and complex real life trajectory data sets.     

An adaptive feature fusion framework for multi-class classification based on SVM

An adaptive feature fusion framework is proposed for multi-class classification based on SVM. In a similar manner of one-versus-all (OVA), one of the multi-class SVM schemes, the proposed approach decomposes a multi-class classification into several binary classifications. The main difference lies in that each classifier is created with the most suitable feature vectors to discriminate one class from all the other classes. The feature vectors of the unknown samples are selected by each classifier adaptively such that recognition is fulfilled accordingly. In addition, novel evaluation criterions are defined to deal with the frequent small-number sample problems. A writer recognition experiment is carried out to accomplish this framework with three kinds of feature vectors: texture, structure and morphological features. Finally, the performance of the proposed approach is illustrated as compared with the OVA by applying the same feature vectors for all classes.     

Multi-view fast object detection by using extended haar filters in uncontrolled environments

In this paper, we propose multi-view object detection methodology by using specific extended class of haar-like filters, which apparently detects the object with high accuracy in the unconstraint environments. There are several object detection techniques, which work well in restricted environments, where illumination is constant and the view angle of the object is restricted. The proposed object detection methodology successfully detects faces, cars, logo objects at any size and pose with high accuracy in real world conditions. To cope with angle variation, we propose a multiple trained cascades by using the proposed filters, which performs even better detection by spanning a different range of orientation in each cascade. We tested the proposed approach by still images by using image databases and conducted some evaluations by using video images from an IP camera placed in outdoor. We tested the method for detecting face, logo, and vehicle in different environments. The experimental results show that the proposed method yields higher classification performance than Viola and Jones’s detector, which uses a single feature for each weak classifier. Given the less number of features, our detector detects any face, object, or vehicle in 15 fps when using 4 megapixel images with 95% accuracy on an Intel i7 2.8 GHz machine.