Anonymous subject identification and privacy information management in video surveillance

The widespread deployment of surveillance cameras has raised serious privacy concerns, and many privacy-enhancing schemes have been recently proposed to automatically redact images of selected individuals in the surveillance video for protection. Of equal importance are the privacy and efficiency of techniques to first, identify those individuals for privacy protection and second, provide access to original surveillance video contents for security analysis. In this paper, we propose an anonymous subject identification and privacy data management system to be used in privacy-aware video surveillance. The anonymous subject identification system uses iris patterns to identify individuals for privacy protection. Anonymity of the iris-matching process is guaranteed through the use of a garbled-circuit (GC)-based iris matching protocol. A novel GC complexity reduction scheme is proposed by simplifying the iris masking process in the protocol. A user-centric privacy information management system is also proposed that allows subjects to anonymously access their privacy information via their iris patterns. The system is composed of two encrypted-domain protocols: The privacy information encryption protocol encrypts the original video records using the iris pattern acquired during the subject identification phase; the privacy information retrieval protocol allows the video records to be anonymously retrieved through a GC-based iris pattern matching process. Experimental results on a public iris biometric database demonstrate the validity of our framework.     

Understanding trust in privacy-aware video surveillance systems

Recent advances in pervasive video surveillance systems pave the way for a comprehensive surveillance of every aspect of our lives, hence, leading us to a state of dataveillance. Computerized and interconnected systems of cameras could be used to profile, track and monitor individuals for the sake of security. Notwithstanding, these systems clearly interfere with the fundamental right of the individuals to privacy. Most literature on privacy in video surveillance systems concentrates on the goal of detecting faces and other regions of interest and in proposing different methods to protect them. However, the trustworthiness of those systems and, by extension, of the privacy they provide are mostly neglected. In this article, we define the concept of trustworthy privacy-aware video surveillance system. Moreover, we assess the techniques proposed in the literature according to their suitability for such a video surveillance system. Finally, we describe the properties that a deployment of a trustworthy video surveillance system must fulfill.     

A general framework for managing and processing live video data with privacy protection

Though a large body of existing work on video surveillance focuses on image and video processing techniques, few address the usability of such systems, and in particular privacy issues. This study fuses concepts from stream processing and content-based image retrieval to construct a privacy-preserving framework for rapid development and deployment of video surveillance applications. Privacy policies, instantiated to as privacy filters, may be applied both granularly and hierarchically. Privacy filters are granular as they are applicable to specific objects appearing in the video streams. They are hierarchal because they can be specified at specific objects in the framework (e.g., users, cameras) and are combined such that the disseminated video stream adheres to the most stringent aspect specified in the cascade of all privacy filters relevant to a video stream or query. To support this privacy framework, we extend our Live Video Database Model with an informatics-based approach to object recognition and tracking and add an intrinsic privacy model that provides a level of privacy protection not previously available for real-time streaming video data. The proposed framework also provides a formal approach to implement and enforce privacy policies that are verifiable, an important step towards privacy certification of video surveillance systems through a standardized privacy specification language.     

基于宽容训练和隐私保护的快速监控视频检索模型

监控视频关键帧检索和属性查找在交通、安防、教育等领域具有众多应用场景,应用深度学习模型处理海量视频数据在一定程度上缓解了人力消耗,但是存在隐私泄露、计算资源消耗大、时间长等特点.基于上述场景,提出了一个面向大规模监控视频的安全、快速的视频检索模型.具体地,根据云端算力大、监控摄像头算力规模小的特点,在云端部署重量级模型,并使用所提出的宽容训练策略对其进行定制化知识蒸馏,将蒸馏后的轻量级模型部署在监控摄像头内,同时使用局部加密算法对图像敏感部分进行加密,结合云端TEE技术和用户授权机制,在极低资源消耗的情况下实现隐私保护.通过合理控制蒸馏策略的“容忍度”,能够较好地平衡摄像头视频输入阶段和云端检索阶段的耗时,在保证极高准确率的前提下,保证极低的检索时延.相比于传统检索方法,该模型具有安全高效、可伸缩、低延时的特点.实验结果显示,在多个公开数据集上,该模型相比于传统检索方法提供9×-133×的加速.     

Effective distributed service architecture for ubiquitous video surveillance

Video surveillance systems are playing an important role to protect lives and assets of individuals, enterprises and governments. Due to the prevalence of wired and wireless access to Internet, it would be a trend to integrate present isolated video surveillance systems by applying distributed computing environment and to further gestate diversified multimedia intelligent surveillance (MIS) applications in ubiquity. In this paper, we propose a distributed and secure architecture for ubiquitous video surveillance (UVS) services over Internet and error-prone wireless networks with scalability, ubiquity and privacy. As cloud computing, users consume UVS related resources as a service and do not need to own the physical infrastructure, platform, or software. To protect the service privacy, preserve the service scalability and provide reliable UVS video streaming for end users, we apply the AES security mechanism, multicast overlay network and forward error correction (FEC), respectively. Different value-added services can be created and added to this architecture without introducing much traffic load and degrading service quality. Besides, we construct an experimental test-bed for UVS system with three kinds of services to detect fire and fall-incident features and record the captured video at the same time. Experimental results showed that the proposed distributed service architecture is effective and numbers of services on different multicast islands were successfully connected without influencing the playback quality. The average sending rate and the receiving rates of these services are quite similar, and the surveillance video is smoothly played.     

Privacy enabled video surveillance using a two state Markov tracking algorithm

Nowadays video surveillance systems are widely deployed in many public places. However, the widespread use of video surveillance violates the privacy rights of the people. Many authors have addressed the privacy issues from various points of view. In this paper we propose a novel, on-demand selectively revocable, privacy preserving mechanism. The surveillance video can be tuned to view with complete privacy or by revoking the privacy of any subset of pedestrians while ensuring complete privacy to the remaining pedestrians. We achieve this by tracking the pedestrians using a novel Markov chain algorithm with two hidden states, detecting the head contour of the tracked pedestrians and obscuring their faces using an encryption mechanism. The detected pedestrian face/head is obscured by encrypting with a unique key derived from a master key for the privacy preservation purpose. The performance evaluations on many challenging surveillance scenarios show that the proposed mechanism can effectively and robustly track as well as identify multiple pedestrians and obscure/unobscure their faces/head in real time.     

Intended human object detection for automatically protecting privacy in mobile video surveillance

With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons’ capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.     

Chaos-cryptography based privacy preservation technique for video surveillance

A multimedia surveillance system aims to provide security and safety of people in a monitored space. However, due to the nature of surveillance, privacy-sensitive information such as face, gait, and other physical parameters based on the captured media from multiple sensors, can be revealed without the permission of the people who appear in the surveillance video. This is a major concern in recent days. Therefore, it is desirable to have such mechanism that can hide privacy-sensitive information as much as possible, yet supporting effective surveillance tasks. In this article, we propose a chaos cryptography based data scrambling approach that can be applied on selected regions of interest (ROIs) in video camera footage, which contains privacy-sensitive data. Our approach also supports multiple levels of abstraction of data hiding depending on the role of the authorized user. In order to evaluate the suitability of this approach, we applied our algorithm on some video camera footage and observed that our approach is computationally efficient and, hence, it can be applied for real-time video surveillance tasks in preserving privacy sensitive information.     

User-centric privacy awareness in video surveillance

Especially in urban environments, video cameras have become omnipresent. Supporters of video surveillance argue that it is an excellent tool for many applications including crime prevention and law enforcement. While this is certainly true, it must be questioned if sufficient efforts are made to protect the privacy of monitored people. Privacy concerns are often set aside when compared to public safety and security. One reaction to this situation is emerging: community-based efforts where citizens register and map surveillance cameras in their environment. Our study is inspired by this idea and proposes a user-specific and location-aware privacy awareness system. Using conventional smartphones, users not only can contribute to the camera maps, but also use community-collected data to be alerted of potential privacy violations. In our model, we define different levels of privacy awareness. For the highest level, we present a mechanism that allows users to directly interact with specially designed, trustworthy cameras. These cameras provide direct feedback about the tasks that are executed by the camera and how privacy-sensitive data is handled. A hardware security chip that is integrated into the camera is used to ensure authenticity, integrity and freshness of the provided camera status information.     

Dealing with digital traces: Understanding protective behaviors on mobile devices

With increasingly digitization, more and more information is collected from individuals and organizations, leading to several privacy concerns. These risks are further heightened in the mobile realm as data collection can occur continuously and ubiquitously. When individuals use their own devices in work settings, these issues become concerns for organization as well. The question then is how to ensure individuals perform proper information protection behaviors on mobile devices. In this research, we develop a model of mobile information protection based on an integration of the Theory of Planned Behavior and the information privacy literature to explore the antecedents of the attitude of individuals towards sharing information on their mobile devices, their intentions to use protective settings, and their actual practices. The model is tested with data from 228 iPhone users. The results indicate that mobile information protection intention leads to actual privacy settings practice, and that attitude towards information sharing and mobile privacy protection self-efficacy affect this intention. Determinants of attitude towards information sharing include mobile privacy concern and trust of the mobile platform. Finally, prior invasion experience is related to privacy concern. These findings provide insights into factors that can be targeted to enhance individuals’ protective actions to limit the amount of digital information they share via their smartphones.     

W3-privacy: understanding what, when, and where inference channels in multi-camera surveillance video

Huge amounts of video are being recorded every day by surveillance systems. Since video is capable of recording and preserving an enormous amount of information which can be used in many applications, it is worth examining the degree of privacy loss that might occur due to public access to the recorded video. A fundamental requirement of privacy solutions is an understanding and analysis of the inference channels than can lead to a breach of privacy. Though inference channels and privacy risks are well studied in traditional data sharing applications (e.g., hospitals sharing patient records for data analysis), privacy assessments of video data have been limited to the direct identifiers such as people’s faces in the video. Other important inference channels such as location (Where), time (When), and activities (What) are generally overlooked. In this paper we propose a privacy loss model that highlights and incorporates identity leakage through multiple inference channels that exist in a video due to what, when, and where information. We model the identity leakage and the sensitive information separately and combine them to calculate the privacy loss. The proposed identity leakage model is able to consolidate the identity leakage through multiple events and multiple cameras. The experimental results are provided to demonstrate the proposed privacy analysis framework.     

Error-resilient surveillance video transmission based on a practical joint source-channel distortion computing model

In recent years, IP (Internet Protocol)-based video surveillance has widely been useful for post-event analysis and assisting the work of privacy protection and public safety. To support high-quality IP video surveillance, error-resilience techniques are important for surveillance system design, because video has more stringent requirements than general video transmission for packet loss, latency, and jitter. The optimal FEC (forward error correction) code rate decision is a crucial procedure to determine the optimal source and channel coding rates to minimize the overall picture distortion when transporting video packets over packet loss channels. The conventional FEC code rate decision schemes using an analytical source-coding distortion model and a channel-induced distortion model are usually complex and typically employ the process of model parameter training, which involves potentially high computational complexity and implementation cost. To avoid the complex modeling procedure, we propose a simple but accurate joint source-channel distortion model to estimate the channel-loss threshold set for optimal FEC code rate decision. Since the proposed model is expressed as a simple closed form and has a small number of scene-dependent model parameters, a video sender of the surveillance system using the model can be easily implemented. For training the scene-dependent model parameters in real time, we propose a practical test-run procedure. This method accelerates the test-run while maintaining its accuracy for training the scene-dependent model parameters. Using the proposed simple model and practical test-run method, the video sender can find the optimal code rate for on-the-fly joint source-channel coding whenever there is a change in the packet-loss condition in the channel. Simulations show that the proposed method can accurately estimate the channel loss threshold set, resulting in an optimal FEC code rate with low computational complexity.     

AIoT-enabled smart surveillance for personal data digitalization: Contextual personalization-privacy paradox in smart home

Artificial intelligence of things technology provides smart surveillance capability for personal data digitalization. It will invade individuals’ information, physical, and social spaces and raise contextual privacy concerns while providing personalized services, which has not been explored in previous research. We theorize three types of smart surveillance and identify three subdimensions of contextual personalization and privacy concerns. Grounded in surveillance theory and personalization-privacy paradox, we examined the different trade-offs of contextual personalization and privacy concerns underlying the three types of smart surveillance on users’ behavioral intention in smart home context. The results also indicated that transparency can lessen the trade-off effects.     

Scenario-based query processing for video-surveillance archives

Automated video surveillance has emerged as a trendy application domain in recent years, and accessing the semantic content of surveillance video has become a challenging research area. The results of a considerable amount of research dealing with automated access to video surveillance have appeared in the literature; however, significant semantic gaps in event models and content-based access to surveillance video remain. In this paper, we propose a scenario-based query-processing system for video surveillance archives. In our system, a scenario is specified as a sequence of event predicates that can be enriched with object-based low-level features and directional predicates. We introduce an inverted tracking scheme, which effectively tracks the moving objects and enables view-based addressing of the scene. Our query-processing system also supports inverse querying and view-based querying, for after-the-fact activity analysis. We propose a specific surveillance query language to express the supported query types in a scenario-based manner. We also present a visual query-specification interface devised to facilitate the query-specification process. We have conducted performance experiments to show that our query-processing technique has a high expressive power and satisfactory retrieval accuracy in video surveillance.     

Will robots know us better than we know ourselves?

This paper aims to highlight some conceptual aspects on the impact of robotics on our concept of privacy. In those areas where robotics applications will invade the privacy of individuals as computers or mobile phones do today, the current idea of privacy will no longer suffice to ensure the right level of people’s protection. If we think to answer or stop the forthcoming controversies only relying on self-regulation of private parties, we will escape the real challenge: the next generation of robots does not affect solely persons and their individual rights, but the entire structure of society. This article assumes the robotics–privacy relationship as a clear illustration of how the technology–society nexus should be regulated in the future. We need approaches that are contextual–normativeand that should be politically addressed to the creation of a critical culture of technology.     

An Efficient Clustering Algorithm for <Emphasis Type="Italic">k</Emphasis>-Anonymisation

K-anonymisation is an approach to protecting individuals from being identified from data.Good k-anonymisations should retain data utility and preserve privacy,but few methods have considered these two conflicting requirements together. In this paper,we extend our previous work on a clustering-based method for balancing data utility and privacy protection, and propose a set of heuristics to improve its effectiveness.We introduce new clustering criteria that treat utility and privacy on equal terms and propose sampling-based techniques to optimally set up its parameters.Extensive experiments show that the extended method achieves good accuracy in query answering and is able to prevent linking attacks effectively.     

StreamTune: dynamic resource scheduling approach for workload skew in video data center

Video surveillance applications need video data center to provide elastic virtual machine (VM) provisioning. However, the workloads of the VMs are hardly to be predicted for online video surveillance service. The unknown arrival workloads easily lead to workload skew among VMs. In this paper, we study how to balance the workload skew on online video surveillance system. First, we design the system framework for online surveillance service which consists of video capturing and analysis tasks. Second, we propose StreamTune, an online resource scheduling approach for workload balancing, to deal with irregular video analysis workload with the minimum number of VMs. We aim at timely balancing the workload skew on video analyzers without depending on any workload prediction method. Furthermore, we evaluate the performance of the proposed approach using a traffic surveillance application. The experimental results show that our approach is well adaptive to the variation of workload and achieves workload balance with less VMs.     

Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras

Despite the fact that personal privacy has become a major concern, surveillance technology is now becoming ubiquitous in modern society. This is mainly due to the increasing number of crimes as well as the essential necessity to provide secure and safer environment. Recent research studies have confirmed now the possibility of recognizing people by the way they walk i.e. gait. The aim of this research study is to investigate the use of gait for people detection as well as identification across different cameras. We present a new approach for people tracking and identification between different non-intersecting un-calibrated stationary cameras based on gait analysis. A vision-based markerless extraction method is being deployed for the derivation of gait kinematics as well as anthropometric measurements in order to produce a gait signature. The novelty of our approach is motivated by the recent research in biometrics and forensic analysis using gait. The experimental results affirmed the robustness of our approach to successfully detect walking people as well as its potency to extract gait features for different camera viewpoints achieving an identity recognition rate of 73.6 % processed for 2270 video sequences. Furthermore, experimental results confirmed the potential of the proposed method for identity tracking in real surveillance systems to recognize walking individuals across different views with an average recognition rate of 92.5 % for cross-camera matching for two different non-overlapping views.     

Preserving privacy by de-identifying face images

In the context of sharing video surveillance data, a significant threat to privacy is face recognition software, which can automatically identify known people, such as from a database of drivers' license photos, and thereby track people regardless of suspicion. This paper introduces an algorithm to protect the privacy of individuals in video surveillance data by deidentifying faces such that many facial characteristics remain but the face cannot be reliably recognized. A trivial solution to deidentifying faces involves blacking out each face. This thwarts any possible face recognition, but because all facial details are obscured, the result is of limited use. Many ad hoc attempts, such as covering eyes, fail to thwart face recognition because of the robustness of face recognition methods. This work presents a new privacy-enabling algorithm, named k-Same, that guarantees face recognition software cannot reliably recognize deidentified faces, even though many facial details are preserved. The algorithm determines similarity between faces based on a distance metric and creates new faces by averaging image components, which may be the original image pixels (k-Same-Pixel) or eigenvectors (k-Same-Eigen). Results are presented on a standard collection of real face images with varying k.     

Internet of surveillance: a cloud supported large-scale wireless surveillance system

Large-scale video surveillance systems are among the necessities for securing our life these days. The high bandwidth demand and the large storage requirements are the main challenges in such systems. To face these challenges, the system can be deployed as a multi-tier framework that utilizes different technologies. In such a framework, technologies proposed under the umbrella of the Internet of Things (IoT) can play a significant rule in facing the challenges. In video surveillance, the cameras can be considered as “the things” that are streaming videos to a central processing and storage server (the cloud) through the Internet. Wireless technologies can be used to connect wireless cameras to the surveillance system more conveniently than wired cameras. Unfortunately, wireless communication in general tend to have limited bandwidth that needs careful management to achieve scalability. In this paper, we design and evaluate a reliable IoT-based wireless video surveillance system that provides an optimal bandwidth distribution and allocation to minimize the overall surveillance video distortion. We evaluate our system using NS-3 simulation. The results show that the proposed framework fully utilizes the available cloud bandwidth budget and achieves high scalability.