Nonchronological video synopsis and indexing

The amount of captured video is growing with the increased numbers of video cameras, especially the increase of millions of surveillance cameras that operate 24 hours a day. Since video browsing and retrieval is time consuming, most captured video is never watched or examined. Video synopsis is an effective tool for browsing and indexing of such a video. It provides a short video representation, while preserving the essential activities of the original video. The activity in the video is condensed into a shorter period by simultaneously showing multiple activities, even when they originally occurred at different times. The synopsis video is also an index into the original video by pointing to the original time of each activity. Video Synopsis can be applied to create a synopsis of an endless video streams, as generated by webcams and by surveillance cameras. It can address queries like "Show in one minute the synopsis of this camera broadcast during the past day'. This process includes two major phases: (i) An online conversion of the endless video stream into a database of objects and activities (rather than frames). (ii) A response phase, generating the video synopsis as a response to the user's query.     

基于IP Camera的网络视频监控系统的设计

设计了一种基于IP Camera的网络视频监控系统,系统通过各种不同的接入服务,提供了C/S、Web、ClickOnce等多种访问方式,实现了多种平台下对视频终端的监控.系统采用类似于DNS的目录服务,将所有物理节点、逻辑节点均统一编码,并存储到本地配置数据库或文件中,实现了视频终端、客户监控端、视频数据文件等信息的管理和访问.系统在有线和无线两种不同的传输方式下,根据不同的网络传输特点,分别采用了TCP和UDP两种不同的通信方式和通信机制向外部提供视频数据服务,提高了视频流畅性.     

Smart seed selection-based effective black box fuzzing for IIoT protocol

The Journal of Supercomputing - Connections of cyber-physical system (CPS) components are gradually increasing owing to the introduction of the Industrial Internet of Things (IIoT). IIoT...     

Live camera recording robust video fingerprinting

The present paper advances a robust video fingerprinting system for tracking the visual content subjected to live recording. The methodological novelty of the system relies in creating synergies between architectural modules, designed so as to offer: (1) local visual feature representations, invariant with respect to scale, orientation and affine transformations; (2) scalable global feature representations invariant with respect to photometric transformations and (3) time-variant jitter synchronization. The system is tested on a reference database of 14 h of cinematographic content and on a query dataset of 28 h of video related to two use cases: (a) computer-generated distortions (Gaussian filtering, sharpening, rotations with 2°, conversion to grayscale, contrast changes, brightness changes, geometric random bending) and (b) live camera recording. The former use case resulted in ideal rate of false alarm, probability of missed detection of 0.02 and F1 score of 0.97. However, the applicative novelty is given by solving the latter use case: experimental values of the false alarm rate lower than 0.01, probability of missed detection of 0.04 and F1 score equal to 0.94 were obtained for content live recorded from theaters’ and PC screens; these results demonstrate the robustness of the advanced method against live camera recording.     

Automated camera sabotage detection for enhancing video surveillance systems

Surveillance cameras are vital source of information in crime investigations. A surveillance video must be recorded with correct field of view and be of good quality, otherwise, it may not be suitable for investigation or analysis purposes. Perpetrators may tamper the recorded video or the physical device itself, in order to conceal their dubious activities. Generally, surveillance systems are unmanned due to limitations of manual monitoring. Automatic detection of camera tamper events is crucial for timely operator intervention. We propose a new method for detecting video camera tampering events like occlusion, defocus and displacement. The features used are edge information, frame count, foreground objects’ coverage area and its static nature. Effectiveness of our method is tested through experimentation on public datasets. The results obtained are encouraging with high detection and low false alarm rates. The proposed method automatically detects routine problems with cameras like dirt on camera lens, fog and smoke.

     

Preserving interactions among moving objects in surveillance video synopsis

Video synopsis is an effective solution for fast browsing and retrieval of long surveillance videos. It aims to shorten long video sequences into its equivalent compact video representation by rearranging the video events in the temporal domain and/or spatial domain. Conventional video synopsis methods focus on reducing the collisions between tubes and maintaining their chronological order, which may alter the original interactions between tubes due to improper tube rearrangement. In this paper, we present an approach to preserve the relationships among tubes (tracks of moving objects) of the original video in the synopsis video. First, a recursive tube-grouping algorithm is proposed to determine the behavior interactions among tubes in a video and group the related tubes together to form tube sets. Second, to preserve the discovered relationships, a spatio-temporal cube voting algorithm is proposed. This cube voting method optimally rearranges the tube sets in the synopsis video, minimizing false collisions between tubes. Third, a method to estimate the duration of the synopsis video is proposed based on an entropy measure of tube collisions. The extensive experimental results demonstrate that the proposed video synopsis framework condenses videos by preserving the original tube interactions and reducing false tube collisions.

     

一种改进的摄像头视频实时拼接方法

针对普通摄像头水平视角较小的问题,通过同时采集具有一定重叠区域的摄像头视频帧图像,基于尺度不变特征变换（ScaleInvariantFeatureTransform,SIFT）特征点,用二分哈希搜索算法（DichotomyBasedHash,DBH）进行匹配,用随机采样一致（RandomSampleConsensus,RANSAC）算法消除误匹配,得到帧图像拼接映射关系。实验结果表明,该方法能有效地实现摄像头视频实时拼接,克服了既有方法在重叠区域小于20％时失效的不足,在重叠区域为10％左右时仍能取得有效的拼接。     

面向视频传感器网络的视频压缩方法设计

视频传感器网络在监控过程中会产生大量数据,为此提出一种改进的基于对象的视频压缩方法.首先,运用改进的混合高斯模型进行背景建模;其次,提出宏块分类方法,对视频帧图像进行宏块分类;然后,利用宏块分类结果,实现前景和背景的分离;最后,分别对前景和背景进行压缩,形成各自的编码流.实验结果表明,该方法具有比MPEG-4更强的压缩能力,满足视频传感器网络的视频压缩需求.     

Aligning windows of live video from an imprecise pan-tilt-zoom camera into a remote panoramic display for remote nature observation

A pan-tilt-zoom (PTZ) robotic camera can provide a detailed live video of selected areas of interest within a large potential viewing field. The selective coverage is ideal for nature observation applications where power and bandwidth are often limited. To provide the spatial context for human observers, it is desirable to insert the live video into a large spherical panoramic display representing the entire viewing field. Accurate alignment of the video stream within the panoramic display is difficult due to imprecise pan-tilt values and rapid changes in camera configurations. Common image alignment algorithms are computationally expensive for real time applications. We are interested in designing algorithms that fit low power computation platform and hence can be implemented inside the PTZ camera in the future. We present a sampling-based constant-time image alignment algorithm based on spherical projection and projection-invariant selective sampling that accurately registers paired images at 100 frames per second on a simulated embedded platform. The alignment accuracy actually is better than existing methods when high rotational difference is involved. Experiments suggest that the new alignment algorithm is faster than existing algorithms by 1,471.6 times when aligning a six-mega-pixel image pair.     

Surround video: a multihead camera approach

We describe algorithms for creating, storing and viewing high-resolution immersive surround videos. Given a set of unit cameras designed to be almost aligned at a common nodal point, we first present a versatile process for stitching seamlessly synchronized streams of videos into a single surround video corresponding to the video of the multihead camera. We devise a general registration process onto raymaps based on minimizing a tailored objective function. We review and introduce new raymaps with good sampling properties. We then give implementation details on the surround video viewer and present experimental results on both real-world acquired and computer-graphics rendered full surround videos. We conclude by mentioning potential applications and discuss ongoing related activities. Video supplements: http://www.csl.sony.co.jp/person/nielsen     

Extracting representative motion flows for effective video retrieval

In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.     

Exploiting local spatio-temporal characteristics for effective video understanding

Multimedia Tools and Applications - The explosive growth in online video streaming presents challenges for video understanding with high accuracy and low computation complexity. Recent methods have...     

Efficient compression and network adaptive video coding for distributed video surveillance

The availability of cheap network based video cameras and the prevalence of wireless networks has lead to a major thrust towards deployment of large scale Distributed Video Surveillance (DVS) systems. This has opened up an important area of research to deal with the issues involved in DVS system for efficient collection and transmission of large scale video streams from the cameras at the guarded sites, to the end users in possibly constrained network conditions. In this paper, we propose a framework based on content-based video classification and scalable compression scheme to provide a robust bandwidth efficient video transmission for DVS. The scheme builds on a Discrete Wavelet Transform (DWT) based Color-Set Partitioning for Hierarchical Trees (CSPIHT) coding to obtain a scalable bitstream. Wavelet domain segmentation and compression assists in development of a DVS architecture. The architecture includes a novel module for dynamic allocation of Network bandwidth based on the current available resources and constraints. Different frame constituents are optimally coded based on their relative significance, perceptual quality, and available estimate of network bandwidth. Experimental result over different video sequences and simulations for Network conditions demonstrate the efficient performance of the approach.     

An improved surveillance video synopsis framework: a HSATLBO optimization approach

Video surveillance cameras capture huge amount of data 24 hours a day. However, most of these videos contain redundant data which make the process difficult for browsing and analysis. A significant amount of research findings have been made in summarization of recorded video, but such schemes do not have much impact on video surveillance applications. On the contrary, video synopsis is a smart technology that preserves all the activities of every single object and projects them concurrently in a condensed time. The energy minimization module in video synopsis framework plays a vital role, which in turn minimizes the activity loss, number of collision and temporal consistency cost. In most of the reported schemes, Simulated Annealing (SA) algorithm is employed to solve the energy minimization problem. However, it suffers from slow convergence rate resulting in a high computational load to the system. In order to mitigate this issue, this article presents an improved energy minimization scheme using hybridization of SA and Teaching Learning based Optimization (TLBO) algorithms. The suggested framework for static surveillance video synopsis generation consists of four computational modules, namely, Object detection and segmentation, Tube formation, Optimization, and finally Stitching and the central focus is on the optimization module. Thus, the present work deals with an improved hybrid energy minimization problem to achieve global optimal solution with reduced computational time. The motivation behind hybridization (HSATLBO) is that TLBO algorithm has the ability to search rigorously, ensuring to reach the optimum solution with less computation. On the contrary, SA reaches the global optimum solution, but it may get disarrayed and miss some critical search points. Exhaustive experiments are carried out and results compared with that of benchmark schemes in terms of minimizing the activity, collision and temporal consistency costs. All the experiments are conducted on five widely used videos taken from standard surveillance video data set (PETS 2001, MIT Surveillance Dataset, ChangeDetection.Net, PETS 2006 and UMN Dataset) as well as one real generated surveillance video from the IIIT Bhubaneswar Surveillance Dataset. To make a fair comparison, additionally, performance of the proposed hybrid scheme to solve video synopsis optimization problem is also compared with that of the other benchmark functions. Experimental evaluation and analysis confirm that the proposed scheme outperforms other state-of-the-art approaches. Finally, the suggested scheme can be easily and reliably deployed in the off-line video synopsis generation.

     

Shooting a smooth video with a shaky camera

Abstract. The image sequence in a video taken by a moving camera may suffer from irregular perturbations because of irregularities in the motion of the person or vehicle carrying the camera. We show how to use information in the image sequence to correct the effects of these irregularities so that the sequence is smoothed, i.e., is approximately the same as the sequence that would have been obtained if the motion of the camera had been smooth. Our method is based on the fact that the irregular motion is almost entirely rotational, and that the rotational image motion can be detected and corrected if a distant object, such as the horizon, is visible. Received: 14 February 2001 / Accepted: 11 February 2002 Correspondence to: A. Rosenfeld     

Optimized camera handover scheme in free-viewpoint video streaming

Free-viewpoint video (FVV) is a promising approach that allows users to control their viewpoint and generate virtual views from any desired perspective. The individual user viewpoints are synthesized from two or more camera streams and correspondent depth sequences. In case of continuous viewpoint changes, the camera inputs of the view-synthesis process must be changed in a seamless way, to avoid the starvation of the viewpoint synthesizer algorithm. Starvation occurs when the desired user viewpoint cannot be synthesized with the currently streamed camera views, and thus, the FVV playout interrupts. In this paper, we propose three different camera handover schemes (TCC, MA, and SA) based on viewpoint prediction to minimize the probability of playout stalls and find the trade-off between the image quality and the camera handover frequency. Our simulation results show that the introduced camera switching methods can reduce the handover frequency with more than 40%, and hence, the viewpoint synthesis starvation and the playout interruption can be minimized. By providing seamless viewpoint changes, the quality of experience can be significantly improved, making the new FVV service more attractive in the future.     

Robust video communication for ubiquitous network access

Ubiquitous network access implies that video can be streamed to portable devices whether they are moving outdoors or docked at home. Unfortunately, broadband wireless channels and their wired alternatives present a hostile environment for video communication, which manifests itself in error bursts. This paper presents a robust application layer, channel-coding scheme suitable for data-partitioned, compressed video. Data partitioning prioritizes the more important data within a compressed bitstream. In the scheme, the more important compressed data are protected prior to communication over an access network. In particular, window-growth rateless codes are used. This form of rateless code can be incrementally scaled to reflect the importance of the data being protected. The paper gives details of the scheme for achieving this in the context of an H.264/AVC codec’s picture types and structures. The paper considers how best to apply the scheme to H.264/AVC’s data-partitioning modes in a practical manner. Simulations of error-prone channels show that the proposed unequal protection scheme achieves several dBs of improvement in video quality, when compared with equal protection. The simulations modeled both wireless and wired access networks.     

User interface for video observation over the internet

This paper presents the design and application of a system for live video transmission and remote camera control over the World Wide Web. Extensive testing of the Internet Video Server (IVS) prompted us to improve its user interface. The GlobalView extension of IVS was developed which enables the generation of panoramic images of the environment and a more intuitive control of the camera. The live video frame is superimposed on a 360^°static panoramic picture. By interactively moving a rectangular frame in the panoramic picture, the user locally selects the new direction of the camera. Once the view is selected the users prompts the selection and the command is issued over the Internet to the remotely-controlled camera. The static panoramic image is constantly updated in areas where new live video information gets available. Two methods are described for static panoramic image generation: one uses geometric transformation and the other is the brute-force scanning approach. We discuss how visual summaries of activities on an observed location can be generated and custom queries made with a similar intuitive user interface.