Intended human object detection for automatically protecting privacy in mobile video surveillance

With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons’ capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.     

Effective multimedia surveillance using a human-centric approach

Large-scale multimedia surveillance installations usually consist of a number of spatially distributed video cameras that are installed in a premise and are connected to a central control station, where human operators (e.g., security personnel) remotely monitor the scene images captured by the cameras. In the majority of these systems the ratio of human operators to the number of camera views is very low. This potentially raises the problem that some important events may be missed. Studies have shown that a human operator can effectively monitor only four camera views. Moreover, the visual attention of human operator drops below the acceptable level while performing the task of visual monitoring. Therefore, there is a need for the selection of the four most relevant camera views at a given time instant. This paper proposes a human-centric approach to solve the problem of dynamically selecting and scheduling the four best camera views. In the proposed approach we use a feedback camera to observe the human monitoring the surveillance camera feeds. Using this information, the system computes the operator’s attention to the camera views to automatically determine the importance of events being captured by the respective cameras. This real-time non-invasive relevance feedback is then augmented with the automatic detection of events to compute the four best feeds. The experiments show the effectiveness of the proposed approach by improving the identification of important events occurring in the environment.     

Video fire detection – Review

This is a review article describing the recent developments in Video based Fire Detection (VFD). Video surveillance cameras and computer vision methods are widely used in many security applications. It is also possible to use security cameras and special purpose infrared surveillance cameras for fire detection. This requires intelligent video processing techniques for detection and analysis of uncontrolled fire behavior. VFD may help reduce the detection time compared to the currently available sensors in both indoors and outdoors because cameras can monitor “volumes” and do not have transport delay that the traditional “point” sensors suffer from. It is possible to cover an area of 100 km² using a single pan-tilt-zoom camera placed on a hilltop for wildfire detection. Another benefit of the VFD systems is that they can provide crucial information about the size and growth of the fire, direction of smoke propagation.     

3D head tracking for fall detection using a single calibrated camera

The head trajectory is an interesting source of information for behavior recognition and can be very useful for video surveillance applications, especially for fall detection. Consequently, much work has been done to track the head in the 2D image plane using a single camera or in a 3D world using multiple cameras. Tracking the head in real-time with a single camera could be very useful for fall detection. Thus, in this article, an original method to extract the 3D head trajectory of a person in a room is proposed using only one calibrated camera. The head is represented as a 3D ellipsoid, which is tracked with a hierarchical particle filter based on color histograms and shape information. Experiments demonstrated that this method can run in quasi-real-time, providing reasonable 3D errors for a monocular system. Results on fall detection using the head 3D vertical velocity or height obtained from the 3D trajectory are also presented.     

Human fall detection using normalized shape aspect ratio

In video surveillance, automatic human fall detection is important to protect vulnerable groups such as the elderly. When the camera layout varies, the shape aspect ratio (SAR) of a human body may change substantially. In order to rectify these changes, in this paper, we propose an automatic human fall detection method using the normalized shape aspect ratio (NSAR). A calibration process and bicubic interpolation are implemented to generate the NSAR table for each camera. Compared with some representative fall detection methods using the SAR, the proposed method integrates the NSAR with the moving speed and direction information to robustly detect human fall, as well as being able to detect falls toward eight different directions for multiple humans. Moreover, while most of the existing fall detection methods were designed only for indoor environment, experimental results demonstrate that this newly proposed method can effectively detect human fall in both indoor and outdoor environments.

     

Principal axis-based correspondence between multiple cameras for people tracking

Visual surveillance using multiple cameras has attracted increasing interest in recent years. Correspondence between multiple cameras is one of the most important and basic problems which visual surveillance using multiple cameras brings. In this paper, we propose a simple and robust method, based on principal axes of people, to match people across multiple cameras. The correspondence likelihood reflecting the similarity of pairs of principal axes of people is constructed according to the relationship between "ground-points" of people detected in each camera view and the intersections of principal axes detected in different camera views and transformed to the same view. Our method has the following desirable properties; 1) camera calibration is not needed; 2) accurate motion detection and segmentation are less critical due to the robustness of the principal axis-based feature to noise; 3) based on the fused data derived from correspondence results, positions of people in each camera view can be accurately located even when the people are partially occluded in all views. The experimental results on several real video sequences from outdoor environments have demonstrated the effectiveness, efficiency, and robustness of our method.     

Surveillance camera scheduling: a virtual vision approach

We present a surveillance system, comprising wide field-of-view (FOV) passive cameras and pan/tilt/zoom (PTZ) active cameras, which automatically captures high-resolution videos of pedestrians as they move through a designated area. A wide-FOV static camera can track multiple pedestrians, while any PTZ active camera can capture high-quality videos of one pedestrian at a time. We formulate the multi-camera control strategy as an online scheduling problem and propose a solution that combines the information gathered by the wide-FOV cameras with weighted round-robin scheduling to guide the available PTZ cameras, such that each pedestrian is observed by at least one PTZ camera while in the designated area. A centerpiece of our work is the development and testing of experimental surveillance systems within a visually and behaviorally realistic virtual environment simulator. The simulator is valuable as our research would be more or less infeasible in the real world given the impediments to deploying and experimenting with appropriately complex camera sensor networks in large public spaces. In particular, we demonstrate our surveillance system in a virtual train station environment populated by autonomous, lifelike virtual pedestrians, wherein easily reconfigurable virtual cameras generate synthetic video feeds. The video streams emulate those generated by real surveillance cameras monitoring richly populated public spaces.A preliminary version of this paper appeared as [1].     

Vehicle matching in smart camera networks using image projection profiles at multiple instances

Tracking vehicles using a network of cameras with non-overlapping views is a challenging problem of great importance in traffic surveillance. One of the main challenges is accurate vehicle matching across the cameras. Even if the cameras have similar views on vehicles, vehicle matching remains a difficult task due to changes of their appearance between observations, and inaccurate detections and occlusions, which often occur in real scenarios. To be executed on smart cameras the matching has also to be efficient in terms of needed data and computations. To address these challenges we present a low complexity method for vehicle matching robust against appearance changes and inaccuracies in vehicle detection. We efficiently represent vehicle appearances using signature vectors composed of Radon transform like projections of the vehicle images and compare them in a coarse-to-fine fashion using a simple combination of 1-D correlations. To deal with appearance changes we include multiple observations in each vehicle appearance model. These observations are automatically collected along the vehicle trajectory. The proposed signature vectors can be calculated in low-complexity smart cameras, by a simple scan-line algorithm of the camera software itself, and transmitted to the other smart cameras or to the central server. Extensive experiments based on real traffic surveillance videos recorded in a tunnel validate our approach.     

An advanced video-based system for monitoring active volcanoes

Video surveillance systems are consolidated techniques for monitoring eruptive phenomena in volcanic areas. Along with these systems, which use standard video cameras, people working in this field sometimes make use of infrared cameras providing useful information about the thermal evolution of eruptions. Real-time analysis of the acquired frames is required, along with image storing, to analyze and classify the activity of volcanoes. Human effort and large storing capabilities are hence required to perform monitoring tasks.In this paper we present a new strategy aimed at improving the performance of video surveillance systems in terms of human-independent image processing and storing optimization. The proposed methodology is based on real-time thermo-graphic analysis of the area considered. The analysis is performed by processing images acquired with an IR camera and extracting information about meaningful volcanic events.Two software tools were developed. The first provides information about the activity being monitored and automatically adapts the image storing rate. The second tool automatically produces useful information about the eruptive activity encompassed by a selected frame sequence.The software developed includes a suitable user interface allowing for convenient management of the acquired images and easy access to information about the volcanic activity monitored.     

User-centric privacy awareness in video surveillance

Especially in urban environments, video cameras have become omnipresent. Supporters of video surveillance argue that it is an excellent tool for many applications including crime prevention and law enforcement. While this is certainly true, it must be questioned if sufficient efforts are made to protect the privacy of monitored people. Privacy concerns are often set aside when compared to public safety and security. One reaction to this situation is emerging: community-based efforts where citizens register and map surveillance cameras in their environment. Our study is inspired by this idea and proposes a user-specific and location-aware privacy awareness system. Using conventional smartphones, users not only can contribute to the camera maps, but also use community-collected data to be alerted of potential privacy violations. In our model, we define different levels of privacy awareness. For the highest level, we present a mechanism that allows users to directly interact with specially designed, trustworthy cameras. These cameras provide direct feedback about the tasks that are executed by the camera and how privacy-sensitive data is handled. A hardware security chip that is integrated into the camera is used to ensure authenticity, integrity and freshness of the provided camera status information.     

一种基于特征匹配的目标入侵检测方法

视频监控中常用云台摄像机监控视场较大的区域.对于云台摄像机跟随拍摄的情况,提出了一种基于特征匹配的目标入侵检测方法.通过提取的尺度不变特征变换(SIFT)特征点对,将当前图像和全景图像进行匹配,从而得到当前图像和全景图像投影关系,再将当前图像的坐标系变换到全景图像下,最后运用差分法,找到入侵目标.实验结果表明,即使当前图像与全景图像存在尺度、缩放、形变等差异,通过本方法也可正确地检测出入侵目标.     

A survey of approaches and trends in person re-identification

Person re-identification is a fundamental task in automated video surveillance and has been an area of intense research in the past few years. Given an image/video of a person taken from one camera, re-identification is the process of identifying the person from images/videos taken from a different camera. Re-identification is indispensable in establishing consistent labeling across multiple cameras or even within the same camera to re-establish disconnected or lost tracks. Apart from surveillance it has applications in robotics, multimedia and forensics. Person re-identification is a difficult problem because of the visual ambiguity and spatiotemporal uncertainty in a person's appearance across different cameras. These difficulties are often compounded by low resolution images or poor quality video feeds with large amounts of unrelated information in them that does not aid re-identification. The spatial or temporal conditions to constrain the problem are hard to capture. However, the problem has received significant attention from the computer vision research community due to its wide applicability and utility. In this paper, we explore the problem of person re-identification and discuss the current solutions. Open issues and challenges of the problem are highlighted with a discussion on potential directions for further research.     

Distributed interactive video arrays for event capture and enhanced situational awareness

Video surveillance activity has dramatically increased over the past few years. Earlier work dealt mostly with single stationary cameras, but the recent trend is toward active multicamera systems. Such systems offer several advantages over single camera systems - multiple overlapping views for obtaining 3D information and handling occlusions, multiple nonoverlapping cameras for covering wide areas, and active pan-tilt-zoom (PTZ) cameras for observing object details. To address these issues, we have developed a multicamera video surveillance approach, called distributed interactive video array. The DIVA framework provides multiple levels of semantically meaningful information ("situational" awareness) to match the needs of multiple remote observers. We have designed DIVA-based systems that can track and identify vehicles and people, monitor perimeters and bridges, and analyze activities. A new video surveillance approach employing a large-scale cluster of video sensors demonstrates the promise of multicamera arrays for homeland security.     

A multi-camera vision system for fall detection and alarm generation

Abstract: In-house video surveillance can represent an excellent support for people with some difficulties (e.g. elderly or disabled people) living alone and with a limited autonomy. New hardware technologies and in particular digital cameras are now affordable and they have recently gained credit as tools for (semi-)automatically assuring people's safety. In this paper a multi-camera vision system for detecting and tracking people and recognizing dangerous behaviours and events such as a fall is presented. In such a situation a suitable alarm can be sent, e.g. by means of an SMS. A novel technique of warping people's silhouette is proposed to exchange visual information between partially overlapped cameras whenever a camera handover occurs. Finally, a multi-client and multi-threaded transcoding video server delivers live video streams to operators/remote users in order to check the validity of a received alarm. Semantic and event-based transcoding algorithms are used to optimize the bandwidth usage. A two-room setup has been created in our laboratory to test the performance of the overall system and some of the results obtained are reported.     

Video image target monitoring based on RNN-LSTM

Traditional image object classification and detection algorithms and strategies cannot meet the problem of video image acquisition and processing. Deep learning deliberately simulates the hierarchical structure of human brain, and establishes the mapping from low-level signals to high-level semantics, so as to achieve hierarchical feature representation of data. Deep learning technology has powerful visual information processing ability, which has become the forefront technology and domestic and international research hotspots to deal with this challenge. In order to solve the problem of target space location in video surveillance system, time-consuming and other problems, in this paper, we propose the algorithm based on RNN-LSTM deep learning. At the same time, according to the principle of OpenGL perspective imaging and photogrammetry consistency, we use 3D scene simulation imaging technology, relying on the corresponding relationship between video images and simulation images we locate the target object. In the 3D virtual scene, we set up the virtual camera to simulate the imaging processing of the actual camera, and the pixel coordinates in the video image of the surveillance target are substituted into the simulation image, next, the spatial coordinates of the target are inverted by the inverse process of the virtual imaging. The experimental results show that the detection of target objects has high accuracy, which has an important reference value for outdoor target localization through video surveillance images.

     

Nonchronological video synopsis and indexing

The amount of captured video is growing with the increased numbers of video cameras, especially the increase of millions of surveillance cameras that operate 24 hours a day. Since video browsing and retrieval is time consuming, most captured video is never watched or examined. Video synopsis is an effective tool for browsing and indexing of such a video. It provides a short video representation, while preserving the essential activities of the original video. The activity in the video is condensed into a shorter period by simultaneously showing multiple activities, even when they originally occurred at different times. The synopsis video is also an index into the original video by pointing to the original time of each activity. Video Synopsis can be applied to create a synopsis of an endless video streams, as generated by webcams and by surveillance cameras. It can address queries like "Show in one minute the synopsis of this camera broadcast during the past day'. This process includes two major phases: (i) An online conversion of the endless video stream into a database of objects and activities (rather than frames). (ii) A response phase, generating the video synopsis as a response to the user's query.     

An improved statistical approach for moving object detection in thermal video frames

In a video surveillance system, background modeling is assumed to be a fundamental technique for moving object detection. The surveillance system based on thermal video overcomes many challenges, such as background variations, varying light intensity, external illumination source, and so on. This paper presents a new method for background modeling and background subtraction. The method utilizes the combined approach of Fisher's Linear Discriminant and Relative Entropy for pixel based classification and detection of moving objects in thermal video frames. The experimental results show the higher average value of various performance indicators like Accuracy, ROC, and F-measure. In contrast, the percentage of false classification and total error is minimum and also has lesser execution time. The method outperforms when compared with the other existing methods.

     

MeshVision: an adaptive wireless mesh network video surveillance system

The major surveillance camera manufacturers have begun incorporating wireless networking functionality into their products to enable wireless access. However, the video feeds from such cameras can only be accessed within the transmission range of the cameras. These cameras must be connected to backbone infrastructure in order to access them from more than one hop away. This network infrastructure is both time-consuming and expensive to install, making it impractical in many rapid deployment situations (e.g., to provide temporary surveillance at a crime scene). To overcome this problem, we propose the MeshVision system that incorporates wireless mesh network functionality directly into the cameras. Video streams can be pulled from any camera within a network of MeshVision cameras, irrespective of how many hops away that camera is. To manage the trade-off between video stream quality and the number of video streams that could be concurrently accessed over the network, MeshVision uses a bandwidth adaptation mechanism. This mechanism monitors the wireless network looking for drops in link quality or signs of congestion and adjusts the quality of existing video streams in order to reduce that congestion. A significant benefit of the approach is that it is of low cost, requiring only a software upgrade of the cameras.     

采用截面流量统计的高密度人群行人计数

利用摄像头实现行人计数在智能视频监控领域有着重要的价值,但是行人互相遮挡、噪声、摄像机透视效果和图像背景等问题影响了人群计数的准确性.针对高密度人群场景的行人计数准确率的问题,提出了基于截面流量统计的行人计数方法,该方法基于梯度运动历史图像检测前景,并用有效运动图像改进了基于特征提取的行人计数方法,结合运动速度提取方法实现了行人计数.实验结果表明,提出的计数方法在高密度人群场景中具有较高的准确率和实时性,是一种针对高密度人群有效的行人计数方法.