Tucker visual search-based hybrid tracking model and Fractional Kohonen Self-Organizing Map for anomaly localization and detection in surveillance videos

Anomaly detection (AD) in video is a challenging task employed in the intelligent video surveillance applications. This paper presents a technique for localizing and detecting anomalies in surveillance videos by proposing hybrid tracking model and Fractional Kohonen Self-Organizing Map (FKSOM). At first, the objects in the initial frames are detected by extracting the background and comparing with the succeeding frames. Then, a tracking model is developed to track the objects in the frame. Further, the features, such as object shape, speed, energy, correlation, and homogeneity, are extracted in the feature extraction process. Finally, the proposed FKSOM algorithm performs AD by identifying anomalous and normal events in the frame. The performance of the proposed technique is evaluated using the metrics, such as Multiple Object Tracking Precision (MOTP), accuracy, sensitivity, and specificity, where it obtains MOTP of 0.9895 with an average accuracy of 0.9339, the sensitivity of 0.9288 and specificity of 1.     

Bridging semantic gap between high-level and low-level features in content-based video retrieval using multi-stage ESN–SVM classifier

Content-based video retrieval system aims at assisting a user to retrieve targeted video sequence in a large database. Most of the search engines use textual annotations to retrieve videos. These types of engines offer a low-level abstraction while the user seeks high-level semantics. Bridging this type of semantic gap in video retrieval remains an important challenge. In this paper, colour, texture and shapes are considered to be low-level features and motion is a high-level feature. Colour histograms convert the RGB colour space into YcbCr and extract hue and saturation values from frames. After colour extraction, filter mask is applied and gradient value is computed. Gradient and threshold values are compared to draw the edge map. Edges are smoothed for sharpening to remove the unnecessary connected components. These diverse shapes are then extracted and stored in shape feature vectors. Finally, an SVM classifier is used for classification of low-level features. For high-level features, depth images are extracted for motion feature identification and classification is done via echo state neural networks (ESN). ESN are a supervised learning technique and follow the principle of recurrent neural networks. ESN are well known for time series classification and also proved their effective performance in gesture detection. By combining the existing algorithms, a high-performance multimedia event detection system is constructed. The effectiveness and efficiency of proposed event detection mechanism is validated using MSR 3D action pair dataset. Experimental results show that the detection accuracy of proposed combination is better than those of other algorithms     

Efficient object detection and tracking in video sequences

One of the most important problems in computer vision is the computation of the two-dimensional projective transformation (homography) that maps features of planar objects in different images and videos. This computation is required by many applications such as image mosaicking, image registration, and augmented reality. The real-time performance imposes constraints on the methods used. In this paper, we address the real-time detection and tracking of planar objects in a video sequence where the object of interest is given by a reference image template. Most existing approaches for homography estimation are based on two steps: feature extraction (first step) followed by a combinatorial optimization method (second step) to match features between the reference template and the scene frame. This paper has two main contributions. First, we detect both planar and nonplanar objects via efficient object feature classification in the input images, which is applied prior to performing the matching step. Second, for the tracking part (planar objects), we propose a fast method for the computation of the homography that is based on the transferred object features and their associated local raw brightness. The advantage of the proposed schemes is a fast matching as well as fast and robust object registration that is given by either a homography or three-dimensional pose.     

Kullback-Leibler similarity measures for effective content based video retrieval

Abstract

Recent advancements in the multimedia technologies allow the capture and storage of video data with relatively inexpensive computers. As the necessity to query these data competently becomes significant, the amount of broadly accessible video data grows. As a result, content-based retrieval of video data turns out to be a demanding and vital problem. In this paper, an effective content-based video retrieval system is proposed. The raw video data are segmented into shots and the object feature, movement feature and the occlusion feature are extracted from these shots and the feature library is utilised for the storage process of these features. Subsequently, the Kullback–Leibler distance is computed among the features of the feature library and the features of the query clip which is extracted in the similar manner. Hence, with the aid of the Kullback–Leibler distance, the similar videos are extracted from the collection of videos based on the given query video clip in an effective manner.     

Automatic skin segmentation and tracking in sign language recognition

Skin segmentation and tracking play an important role in sign language recognition. A framework for segmenting and tracking skin objects from signing videos is described. It mainly consists of two parts: a skin colour model and a skin object tracking system. The skin colour model is first built based on the combination of support vector machine active learning and region segmentation. Then, the obtained skin colour model is integrated with the motion and position information to perform segmentation and tracking. The tracking system is able to predict occlusions among any of the skin objects using a Kalman filter (KF). Moreover, the skin colour model can be updated with the help of tracking to handle illumination variation. Experimental evaluations using real-world gesture videos and comparison with other existing algorithms demonstrate the effectiveness of the proposed work.     

A content-based image retrieval system based on object-moment feature

The last decade has witnessed great interest in research on content-based image retrieval (CBIR). In 2009, Lin et al. proposed a smart CBIR system based on colour and texture feature. Their system has a high detection rate except the cases where image objects have similar shapes. To enhance the detection rate a shape-based image feature called object-moment is proposed in this paper. Object-moment uses the moment of force to compute the object edge feature by calculating the distance from each edge pixel to the axis, and adding them up as a feature. Besides, we integrate the colour features (NSOM, CSOM) and the texture features (CCM, DBPSP) to enhance image detection rate and simplify computation of image retrieval. A series of analyses and comparisons are performed in our experiments to demonstrate that our proposed method improves the retrieval accuracy significantly.     

Human Behavior Classification Using Geometrical Features of Skeleton and Support Vector Machines

Classification of human actions under video surveillance is gaining a lot of attention from computer vision researchers. In this paper, we have presented methodology to recognize human behavior in thin crowd which may be very helpful in surveillance. Research have mostly focused the problem of human detection in thin crowd, overall behavior of the crowd and actions of individuals in video sequences. Vision based Human behavior modeling is a complex task as it involves human detection, tracking, classifying normal and abnormal behavior. The proposed methodology takes input video and applies Gaussian based segmentation technique followed by post processing through presenting hole filling algorithm i.e., fill hole inside objects algorithm. Human detection is performed by presenting human detection algorithm and then geometrical features from human skeleton are extracted using feature extraction algorithm. The classification task is achieved using binary and multi class support vector machines. The proposed technique is validated through accuracy, precision, recall and F-measure metrics.     

Bandwidth-Efficient Transmission Method for User View-Oriented Video Services

The trend in video viewing has been evolving beyond simply providing a multiview option. Recently, a function that allows selection and viewing of a clip from a multiview service that captures a specific range or object has been added. In particular, the freeview service is an extended concept of multi-view and provides a freer viewpoint. However, since numerous videos and additional data are required for its construction, all of the clips constituting the content cannot be simultaneously provided. Only certain clips are selected and provided to the user. If the video is not the preferred video, change request is made, and a delay occurs during retransmission from the server. Delays due to frequent rerequests degrade the overall quality of service. For free-view services, selectively transmitting the video according to the user’s desired viewpoint and region of interest within the limited network of available videos is important. In this study, we propose a method of screening and providing the correct video based on objects in the contents. Based on the method of recognizing the object in each clip, we designed a method of setting its priority based on information about the object’s location for each viewpoint. During the transmission and receiving process using this information, the selected video can be rapidly recognized and changed. Herein, we present a service system configuration method and propose video selection examples for free-view services.     

Zero-stopping constraint-based hybrid tracking model for dynamic and high-dense crowd videos

Owing to the importance of video surveillance in the public area, tracking finds significant applications using computer vision algorithms to observe the activity of human. In tracking, multi-object tracking is an active research to analyse and detect the activity of anomalies in the crowded scenes. Accordingly, different multi-object tracking algorithms are proposed in the literature to track the human behaviour of the crowded scenes. In this paper, we have presented a zero-stopping criteria-based hybrid tracking algorithm for high-dense crowd videos. Here, head objects are detected using the proposed objective function which considers both colour and texture property of videos. Then, tracking based on motion is performed using the proposed HSIM measure which includes structural similarity (SSIM) and the proposed similarity function. Along with, the data prediction model, exponential weighted moving average (EWMA), is also utilised to track the spatial location of human objects. These two tracking models are then hybridised to obtain the final tracked output. The experimentation is performed with three marathon sequences and the performance is evaluated with particle filtering-based algorithm using tracking number, tracking distance and optimal subpattern assignment metric (OSPA).     

监控视频智能检索关键技术综述

近年来随着监控系统的规模化及网络化,如何在海量的监控视频中快速查找所需要的信息、准确定位嫌疑目标的位置和运动轨迹,成为法庭科学研究的热点。本文就监控视频智能检索影响因素、运动目标的检测和提取技术以及运动目标跟踪方法等方面进行了论述。     

Development of childhood fall motion database and browser based on behavior measurements

This paper describes the development of a fall motion database and a browser designed to facilitate investigations into fall-related injury risk. First, child-related daily activities were collected at a “sensor home”, which is a model of a normal living environment equipped with an embedded video-surveillance system and within which child test subjects were equipped with wearable acceleration-gyro sensors. As of this report, measurements have been conducted for 19 children (months age: mean = 23.8, standard deviation = 10.5), and data has been obtained on 105 fall incidents. During our research, falls were detected from the accumulated sensor data using a detection algorithm developed by the authors, and then video clips of detected falls were extracted from the recorded video streams automatically. The extracted video clips were then used for fall motion analysis. A computer vision (CV) algorithm, which was developed to automate fall motion analysis, facilitates accumulation of fall motion data into the abovementioned database, and the associated database browser allows users to perform conditional searches of fall data by inputting search conditions, such as child attributes and specific fall situations. Before this study, there was no database which contains child's actual fall motion data, and it has the potential to facilitate injury risk reduction related to falls in daily living environments.     

基于组合图像特征与分层节点搜索的回环检测方法

目的 文中通过提出一种新的回环解决方案,平衡回环检测系统的高准确率与高运行效率。方法 提出一种利用组合图像特征与分层节点搜索的新方法。首先,计算一种原始图像的下采样二值化全局特征和经过改进的ORB(oriented FAST and rotated BRIEF)局部特征,将其存入图像特征数据库。其次,引入一种分层节点搜索算法,在数据库中搜索与当前图像特征最相似的全局特征作为回环候选。最后,利用改进的ORB特征进行局部特征匹配,验证候选图像,确定回环检测结果。结果 使用该算法在3个不同的数据集上进行验证,测试中每次回环检测的平均处理时间仅需19 ms。结论 实验结果表明,该算法在运行效率、准确率、召回率等方面均达到了领域内的先进水平。     

基于假设理论的序列图像中多目标检测和跟踪

本文提出一种基于目标检测的多维假设多目标检测和跟踪方法,此算法对于序列图像的照明变化和遮蔽现象具有很高的鲁棒性.首先,对序列图像进行背景抑制、时域滤波和杂波剔除的预处理,得到单帧的初始目标检测结果.对于初始检测中存在的漏检和误检现象,采用基于假设理论的跟踪概率模型优化初始检测轨迹;将目标的跟踪信息反馈于目标检测模块,形成一闭环自适应跟踪系统,达到多目标的最优检测和跟踪.实验结果表明了所提出的方法在多目标检测和跟踪中的可行性和有效性.     

多尺度空间基于集中度判定的二维稳像算法

二轴运动平台由于震动、非一致性摩擦等因素造成视频序列出现抖动,同时外部噪声,光照变化等对图像特征提取、匹配等存在严重影响导致运动向量估计错误,本文提出一种在多尺度空间基于集中度判定的二维稳像算法.通过在多尺度空间下提取视频序列的不变特征,采用深度优先最邻近搜索算法,寻找匹配点对,然后计算匹配点对的集中度,通过最优集中化...     

MRMR Based Feature Vector Design for Efficient Citrus Disease Detection

In recent times, the images and videos have emerged as one of the most important information source depicting the real time scenarios. Digital images nowadays serve as input for many applications and replacing the manual methods due to their capabilities of 3D scene representation in 2D plane. The capabilities of digital images along with utilization of machine learning methodologies are showing promising accuracies in many applications of prediction and pattern recognition. One of the application fields pertains to detection of diseases occurring in the plants, which are destroying the widespread fields. Traditionally the disease detection process was done by a domain expert using manual examination and laboratory tests. This is a tedious and time consuming process and does not suffice the accuracy levels. This creates a room for the research in developing automation based methods where the images captured through sensors and cameras will be used for detection of disease and control its spreading. The digital images captured from the field's forms the dataset which trains the machine learning models to predict the nature of the disease. The accuracy of these models is greatly affected by the amount of noise and ailments present in the input images, appropriate segmentation methodology, feature vector development and the choice of machine learning algorithm. To ensure the high rated performance of the designed system the research is moving in a direction to fine tune each and every stage separately considering their dependencies on subsequent stages. Therefore the most optimum solution can be obtained by considering the image processing methodologies for improving the quality of image and then applying statistical methods for feature extraction and selection. The training vector thus developed is capable of presenting the relationship between the feature values and the target class. In this article, a highly accurate system model for detecting the diseases occurring in citrus fruits using a hybrid feature development approach is proposed. The overall improvement in terms of accuracy is measured and depicted.     

Moving object detection and tracking in videos through turbulent medium

This paper addresses the problem of identifying and tracking moving objects in a video sequence having a time-varying background. This is a fundamental task in many computer vision applications, though a very challenging one because of turbulence that causes blurring and spatiotemporal movements of the background images. Our proposed approach involves two major steps. First, a moving object detection algorithm that deals with the detection of real motions by separating the turbulence-induced motions using a two-level thresholding technique is used. In the second step, a feature-based generalized regression neural network is applied to track the detected objects throughout the frames in the video sequence. The proposed approach uses the centroid and area features of the moving objects and creates the reference regions instantly by selecting the objects within a circle. Simulation experiments are carried out on several turbulence-degraded video sequences and comparisons with an earlier method confirms that the proposed approach provides a more effective tracking of the targets.     

Multiple Pedestrian Detection and Tracking in Night Vision Surveillance Systems

Pedestrian detection and tracking are vital elements of today’s surveillance systems, which make daily life safe for humans. Thus, human detection and visualization have become essential inventions in the field of computer vision. Hence, developing a surveillance system with multiple object recognition and tracking, especially in low light and night-time, is still challenging. Therefore, we propose a novel system based on machine learning and image processing to provide an efficient surveillance system for pedestrian detection and tracking at night. In particular, we propose a system that tackles a two-fold problem by detecting multiple pedestrians in infrared (IR) images using machine learning and tracking them using particle filters. Moreover, a random forest classifier is adopted for image segmentation to identify pedestrians in an image. The result of detection is investigated by particle filter to solve pedestrian tracking. Through the extensive experiment, our system shows 93% segmentation accuracy using a random forest algorithm that demonstrates high accuracy for background and roof classes. Moreover, the system achieved a detection accuracy of 90% using multiple template matching techniques and 81% accuracy for pedestrian tracking. Furthermore, our system can identify that the detected object is a human. Hence, our system provided the best results compared to the state-of-art systems, which proves the effectiveness of the techniques used for image segmentation, classification, and tracking. The presented method is applicable for human detection/tracking, crowd analysis, and monitoring pedestrians in IR video surveillance.     

Video retrieval for shot cluster and classification based on key feature set

The proposed content-based video retrieval method in this study uses feature extraction, shot boundary detection, shot noise filter, shot cluster, the key feature set of shot extraction and shot classification. Shot boundary detection is conducted using a weight selection genetic algorithm. In order to verify the video retrieval method in this paper, three video databases were used in the experiment. These video databases are used to verify the shot boundary detection, the shot classification and video retrieval of this study. A series of comparisons and analyses were then conducted using the above video databases. The experiment results showed that the shot boundary detection method exhibits better performance than the other compared methods. Moreover, the video retrieval experiment results showed that the proposed method is able to precisely catch a shot in the search video, and do so with reduced video retrieval time.