Adaptive weighted fusion with new spatial and temporal fingerprints for improved video copy detection

In this paper, we propose a new and novel modality fusion method designed for combining spatial and temporal fingerprint information to improve video copy detection performance. Most of the previously developed methods have been limited to use only pre-specified weights to combine spatial and temporal modality information. Hence, previous approaches may not adaptively adjust the significance of the temporal fingerprints that depends on the difference between the temporal variances of compared videos, leading to performance degradation in video copy detection. To overcome the aforementioned limitation, the proposed method has been devised to extract two types of fingerprint information: (1) spatial fingerprint that consists of the signs of DCT coefficients in local areas in a keyframe and (2) temporal fingerprint that computes the temporal variances in local areas in consecutive keyframes. In addition, the so-called temporal strength measurement technique is developed to quantitatively represent the amount of the temporal variances; it can be adaptively used to consider the significance of compared temporal fingerprints. The experimental results show that the proposed modality fusion method outperforms other state-of-the-arts fusion methods and popular spatio-temporal fingerprints in terms of video copy detection. Furthermore, the proposed method can save 39.0%, 25.1%, and 46.1% time complexities needed to perform video fingerprint matching without a significant loss of detection accuracy for our synthetic dataset, TRECVID 2009 CCD Task, and MUSCLE-VCD 2007, respectively. This result indicates that our proposed method can be readily incorporated into the real-life video copy detection systems.     

A perceptual scheme for fully automatic video shot boundary detection

In this paper, we propose a novel and robust modus operandi for fast and accurate shot boundary detection where the whole design philosophy is based on human perceptual rules and the well-known “Information Seeking Mantra”. By adopting a top–down approach, redundant video processing is avoided and furthermore elegant shot boundary detection accuracy is obtained under significantly low computational costs. Objects within shots are detected via local image features and used for revealing visual discontinuities among shots. The proposed method can be used for detecting all types of gradual transitions as well as abrupt changes. Another important feature is that the proposed method is fully generic, which can be applied to any video content without requiring any training or tuning in advance. Furthermore, it allows a user interaction to direct the SBD process to the user's “Region of Interest” or to stop it once satisfactory results are obtained. Experimental results demonstrate that the proposed algorithm achieves superior computational times compared to the state-of-art methods without sacrificing performance.     

CNN spatiotemporal features and fusion for surveillance video forgery detection

Surveillance cameras are widely used to provide protection and security; also their videos are used as strong evidences in the courts. Through the availability of video editing tools, it has become easy to distort these evidences. Sometimes, to hide the traces of forgery, some post-processing operations are performed after editing. Hence, the authenticity and integrity of surveillance videos have become urgent to scientifically validate. In this paper, we propose inter-frame forgeries (frame deletion, frame insertion, and frame duplication) detection system using 2D convolution neural network (2D-CNN) of spatiotemporal information and fusion for deep automatically feature extraction; Gaussian RBF multi-class support vector machine (RBF-MSVM) is used for classification process. The experimental results show that the efficiency of the proposed system for detecting all inter-frame forgeries, even when the forged videos have undergone additional post-processing operations such as Gaussian noise, Gaussian blurring, brightness modifications and compression.     

Rotation and flipping robust region binary patterns for video copy detection

Many video fingerprints have been proposed to handle the video transformations problems when the original contents are copied and redistributed. However, most of them did not take into account flipping and rotation transformations. In this paper, we propose a novel video fingerprint based on region binary patterns, aiming to realize robust and fast video copy detection against video transformations including rotation and flipping. We extract two complementary region binary patterns from several rings in keyframes. These two kinds of binary patterns are converted into a new type of patterns for the proposed video fingerprint which is robust against rotation and flipping. The experimental results demonstrated that the proposed video fingerprint is effective for video copy detection particularly in the case of rotation and flipping. Furthermore, our experimental results proved that the proposed method allows for high storage efficiency and low computation complexity, which is suitable for practical video copy system.     

A video semantic detection method based on locality-sensitive discriminant sparse representation and weighted KNN

Video semantic detection has been one research hotspot in the field of human-computer interaction. In video features-oriented sparse representation, the features from the same category video could not achieve similar coding results. To address this, the Locality-Sensitive Discriminant Sparse Representation (LSDSR) is developed, in order that the video samples belonging to the same video category are encoded as similar sparse codes which make them have better category discrimination. In the LSDSR, a discriminative loss function based on sparse coefficients is imposed on the locality-sensitive sparse representation, which makes the optimized dictionary for sparse representation be discriminative. The LSDSR for video features enhances the power of semantic discrimination to optimize the dictionary and build the better discriminant sparse model. More so, to further improve the accuracy of video semantic detection after sparse representation, a weighted K-Nearest Neighbor (KNN) classification method with the loss function that integrates reconstruction error and discrimination for the sparse representation is adopted to detect video semantic concepts. The proposed methods are evaluated on the related video databases in comparison with existing sparse representation methods. The experimental results show that the proposed methods significantly enhance the power of discrimination of video features, and consequently improve the accuracy of video semantic concept detection.     

Visual attention guided video copy detection based on feature points matching with geometric-constraint measurement

In this paper, to efficiently detect video copies, focus of interests in videos is first localized based on 3D spatiotemporal visual attention modeling. Salient feature points are then detected in visual attention regions. Prior to evaluate similarity between source and target video sequences using feature points, geometric constraint measurement is employed for conducting bi-directional point matching in order to remove noisy feature points and simultaneously maintain robust feature point pairs. Consequently, video matching is transformed to frame-based time-series linear search problem. Our proposed approach achieves promising high detection rate under distinct video copy attacks and thus shows its feasibility in real-world applications.     

An HMM based analysis framework for semantic video events

Semantic video analysis plays an important role in the field of machine intelligence and pattern recognition. In this paper, based on the Hidden Markov Model （HMM）, a semantic recognition framework on compressed videos is proposed to analyze the video events according to six low-level features. After the detailed analysis of video events, the pattern of global motion and five features in foreground-the principal parts of videos, are employed as the observations of the Hidden Markov Model to classify events in videos. The applications of the proposed framework in some video event detections demonstrate the promising success of the proposed framework on semantic video analysis.     

Compressed domain video saliency detection using global and local spatiotemporal features

A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.     

视频监控系统中的行人及其面部侦测研究

采用集成H．264硬件编解码视频处理单元Hi3512来设计视频监控系统。并探讨行人目标的自动侦测问题。在对视频图像进行形态学分析的基础上,利用背景差方法实现运动目标区域的粗提取,通过阴影去除算法实现运动目标的精确定位,再利用连续均值量化变换（SuccessiveMeanQuantizafionTransform,SMQT）算法实现运动区域灰度图像的增强处理,然后利用SNoW（SparseNetworkofWinnows）分类算法实现行人及其人脸部位的侦测。实验结果表明,所采用方法能够自动检测出监控区域的行人目标及其面部信息,可有效地应用于无人值守视频监控场合。     

Retinex based visual identicalness detection for videos corrupted by imaging noise

Detecting the visually identical regions among successive frames for noisy videos, called visual identicalness detection (VID) in this paper, is a fundamental tool in video applications for lower power consumption and higher efficiency. In this paper, instead of performing VID on the original video signal or on the de-noised video signal, a Retinex based VID approach is proposed to perform VID on the Retinex signal to eliminate the noise influence introduced by imaging system. Several Retinex output generation approaches are compared, within which the proposed Cohen–Daubechies–Feauveau wavelet based approach is demonstrated to have better efficiency in detection and higher adaptability to the video content and noise severity. Compared with approaches performing detection in the de-noised images, the proposed algorithm presents up to 4.78 times higher detection rate for the videos with moving objects and up to 30.79 times higher detection rate for the videos with static scenes, respectively, at the same error rate. Also, an application of this technique is provided by integrating it into an H.264/AVC video encoder. Compared with compressing the de-noised videos using the existing fast algorithm, an average of 1.7 dB performance improvement is achieved with up to 5.47 times higher encoding speed. Relative to the reference encoder, up to 32.47 times higher encoding speed is achieved without sacrificing the subjective quality.     

Encoder adaptable difference detection for low power video compression in surveillance system

As a state-of-the-art video compression technique, H.264/AVC has been deployed in many surveillance cameras to improve the compression efficiency. However, it induces very high coding complexity, and thus high power consumption. In this paper, a difference detection algorithm is proposed to reduce the computational complexity and power consumption in surveillance video compression by automatically distributing the video data to different modules of the video encoder according to their content similarity features. Without any requirement in changing the encoder hardware, the proposed algorithm provides high adaptability to be integrated into the existing H.264 video encoders. An average of over 82% of overall encoding complexity can be reduced regardless of whether or not the H.264 encoder itself has employed fast algorithms. No loss is observed in both subjective and objective video quality.     

Fall detection using body geometry and human pose estimation in video sequences

According to the World Health Organization, falling is a significant health problem that causes thousands of deaths every year. Fall detection and fall prediction tasks enable accurate medical assistance to vulnerable populations whenever required, allowing local authorities to predict daily health care resources and to reduce fall damages accordingly. We present in this paper, a fall detection approach that explores human body geometry available at different frames of the video sequence. Especially, pose estimation, the angle and the distance between the vector formed by the head-centroid of the identified facial image and the center hip of the body, and the vector aligned with the horizontal axis of the center hip, are employed to construct new distinctive image features. A two-class Support Vector Machine (SVM) classifier and a Temporal Convolution Network (TCN) are trained on the newly constructed feature images. At the same time, a Long-Short-Term Memory (LSTM) network is trained on the calculated angle and distance sequences to classify fall and non-fall activities. We perform experiments on the Le2i FD dataset and the UR FD dataset, where we also propose a cross-dataset evaluation. The results demonstrate the effectiveness and efficiency of the developed approach.     

Feature-based detection and correction of occlusions and split of video objects

This paper proposes a novel algorithm for the real-time detection and correction of occlusion and split in object tracking for surveillance applications. The paper assumes a feature-based model for tracking and is based on the identification of sudden variations of spatio-temporal features of objects to detect occlusions and splits. The detection is followed by a validation stage that uses past tracking information to prevent false detection of occlusion or split. Special care is taken in case of heavy occlusion, when there is a large superposition of objects. For the detection of splits, in addition to the analysis of spatio-temporal changes in objects’ features, our algorithm analyzes the temporal behavior of split objects to discriminate between errors in segmentation and real separation of objects, such as in a deposit event. Both objective and subjective experimental results show the ability of the proposed algorithm to detect and correct, both, split and occlusion of objects. The proposed algorithm is suitable in video surveillance applications due to its good performance in multiple, heavy, and total occlusions, its ability to differentiate between real object separation and faulty object split, its handling of simultaneous occlusion and split events, and its low computational complexity. The algorithm was integrated into an on-line video surveillance system and tested under several conditions with promising results. This work was partially supported by the National Science and Engineering Research Council of Canada (NSERC).     

结合视觉显著性引导与分类器融合的遥感目标检测

利用有限计算资源对大视场遥感图像进行快速目标检测有着重要的现实意义。借鉴注意机制在人类视觉系统中的选择性感知特点,结合自底向上的视觉显著性引导及自顶向下的显著区域解译,提出一种新的大视场遥感目标检测模型。设计其整体架构分为注意初期、注视阶段及注意后期3个递进的层级,通过引入一种自适应形态学的显著图生成策略快速搜寻整个视场中的显著区域,并在其引导下利用分类器融合技术从特征属性相似的显著物中区分出任务目标。以大视场遥感图像舰船检测验证模型,性能及对比实验结果表明该模型是可行的,同时实现了计算资源有层次、有重点地合理分配。     

高密度特征的网络入侵检测算法研究

为了提高网络入侵检测的正确率,针对特征优化和训练样本选择问题,提出一种高密度的网络入侵特征检测算法。首先提取网络状态特征,然后将特征编码成为粒子的位置向量,通过粒子之间信息共享找到最优特征子集,删除冗余和无效特征,降低特征维数,最后采用模糊均值聚类算法选择最优训练样本,并通过支持向量机建立网络入侵检测器。在Matlab 2012平台上采用标准网络入侵数据库对算法性能进行测试,实验结果表明,相对于其它网络入侵检测算法,本文算法提高了网络入侵检测的正确率和检测效率,获得更加理想的网络入侵检测结果。