Calibrating Distributed Camera Networks

Recent developments in wireless sensor networks have made feasible distributed camera networks, in which cameras and processing nodes may be spread over a wide geographical area, with no centralized processor and limited ability to communicate a large amount of information over long distances. This paper overviews distributed algorithms for the calibration of such camera networks- that is, the automatic estimation of each camera's position, orientation, and focal length. In particular, we discuss a decentralized method for obtaining the vision graph for a distributed camera network, in which each edge of the graph represents two cameras that image a sufficiently large part of the same environment. We next describe a distributed algorithm in which each camera performs a local, robust nonlinear optimization over the camera parameters and scene points of its vision graph neighbors in order to obtain an initial calibration estimate. We then show how a distributed inference algorithm based on belief propagation can refine the initial estimate to be both accurate and globally consistent.      

一种适用于分布式摄像机网络的SCIWCF算法

本文使用容积卡尔曼滤波器来处理分布式摄像机网络中的目标跟踪问题.平方根容积信息滤波（Square-Root Cubature Information Filter,SCIF）是容积卡尔曼滤波的一种扩展,其具有有效性和可靠性等方面优势,有利于对多源信息进行信息融合.然而当该算法应用于像摄像机网络这种大规模网络时,如果采用一般的集中式处理,中心节点可能会承受较大的计算压力.针对这个问题,本文首先将平方根容积信息滤波器进行了扩展,提出分布式平方根容积信息滤波器,使其能适应大规模网络.另外在摄像机网络中,由于摄像机装置在一个较大的区域内,由于摄像机观测区域有限,目标可能会出现在观察的盲区,这样就会存在某些摄像机的测量数据无效.针对这个问题,本文提出了平方根容积信息加权一致性滤波器（Square-Root Cubature Information Weighted Consensus Filter,SCIWCF）对状态信息和信息矩阵加权,减小这些无效信息在一致性算法的作用,从而提高整体的滤波性能.仿真实验结果表明,本文提出的算法能够在摄像机网络中对目标进行有效跟踪,在估计精度和滤波器稳定性等方面要优于传统的信息滤波.     

A network of co-operative cameras for visual surveillance

A network of co-operative cameras for the visual surveillance of parking lots is presented. Such a network employs multiple subnets able to manage static and active cameras in a hierarchical framework. The system is able to track multiple targets simultaneously and in real-time throughout the controlled areas. The positions of detected objects, computed from different sensors, are fused considering a dynamic reliability factor for each sensor reading. Close-up recordings of suspicious events are obtained by tasking the active camera systems (ACSs). The co-operation is performed through a multicast communication system studied to transmit useful data both intra and inter networks. In particular, information about the position of the object to track, sent by a static camera system (SCS), is used by an ACS to operate an initial repositioning. The ACS compensates background changes owing to the camera motion, detects mobile objects in the scene and autonomously tracks the object of interest. Tracking results are presented in the context of a video surveillance application for a parking lot.     

一种基于参数重估计和分层匹配的PTZ相机参数修正算法

由于多种因素的影响,PTZ(pan-ti lt-zoom)相机经过长时间运行后其参数会偏离真实值,因 此有必要对相机参数进行修正。针对现有算法参数修正精度低、适应能力弱等问题,提 出了一 种改进的基于参数重估计和分层匹配的参数修正算法。算法通过引入参数重估计策略,避免 了参数 修正过程中的误差累积,提高了参数修正的精度;通过设计分层匹配和特征传播步骤,增强 了算法 对不同尺度图像的适应性。实际场景中的多组实验结果表明,本文算法可以准确修正PTZ相 机参数,并且较现有方法更具优越性。     

Multi-camera systems

In this article, we present some simple and effective techniques for accurately calibrating a multi-camera acquisition system. The proposed methods were proven to be capable of accurate results even when using very simple calibration target sets and low-cost imaging devices, such as standard TV-resolution cameras connected to commercial frame-grabbers. In fact, the performance of our calibration approach yielded results that were about the same as that of other traditional calibration methods based on large 3D target sets. The proposed calibration strategy is based on a multi-view multi-camera approach. This was based on the analysis of a number of views of a simple calibration target-set placed in different (unknown) positions. Furthermore, the method is based on a self-calibration approach, which can refine the a priori knowledge of the world coordinates of the targets (even when such information is very poor) while estimating the parameters of the camera model. Finally we proposed a method, to make the calibration technique adaptive through the analysis of natural scene features, allowing the camera parameters to hold accurate throughout the acquisition session in the presence of parameter drift     

Smart Camera Networks in Virtual Reality

This paper presents our research towards smart camera networks capable of carrying out advanced surveillance tasks with little or no human supervision. A unique centerpiece of our work is the combination of computer graphics, artificial life, and computer vision simulation technologies to develop such networks and experiment with them. Specifically, we demonstrate a smart camera network comprising static and active simulated video surveillance cameras that provides extensive coverage of a large virtual public space, a train station populated by autonomously self-animating virtual pedestrians. The realistically simulated network of smart cameras performs persistent visual surveillance of individual pedestrians with minimal intervention. Our innovative camera control strategy naturally addresses camera aggregation and handoff, is robust against camera and communication failures, and requires no camera calibration, detailed world model, or central controller.      

Distributed target localization using a progressive certainty map in visual sensor networks

Collaboration in visual sensor networks (VSNs) is essential not only to compensate for the processing, sensing, energy, and bandwidth limitations of each sensor node but also to improve the accuracy and robustness of the network. In this paper, we study target localization in VSNs, a challenging computer vision problem because of two unique features of cameras, including the extremely higher data rate and the directional sensing characteristics with limited field of view. Traditionally, the problem is solved by localizing the targets at the intersections of the back-projected 2D cones of each target. However, the existence of visual occlusion among targets would generate many false alarms. In this work, instead of resolving the uncertainty about target existence at the intersections, we identify and study the non-occupied areas in the cone and generate the so-called certainty map of non-existence of targets. As a result, after fusing inputs from a set of sensor nodes, the unresolved regions on the certainty map would be the location of targets. This paper focuses on the design of a light-weight, energy-efficient, and robust solution where not only each camera node transmits a very limited amount of data but that a limited number of camera nodes is involved. We propose a dynamic itinerary for certainty map integration where the entire map is progressively clarified from sensor to sensor. When the confidence of the certainty map is satisfied, targets are localized at the remaining unresolved regions in the certainty map. Based on results obtained from both simulation and real experiments, the proposed progressive method shows effectiveness in detection accuracy as well as energy and bandwidth efficiency.     

Aerial video surveillance and exploitation

There is growing interest in performing aerial surveillance using video cameras. Compared to traditional framing cameras, video cameras provide the capability to observe ongoing activity within a scene and to automatically control the camera to track the activity. However, the high data rates and relatively small field of view of video cameras present new technical challenges that must be overcome before such cameras can be widely used. In this paper, we present a framework and details of the key components for real-time, automatic exploitation of aerial video for surveillance applications. The framework involves separating an aerial video into the natural components corresponding to the scene. Three major components of the scene are the static background geometry, moving objects, and appearance of the static and dynamic components of the scene. In order to delineate videos into these scene components, we have developed real time, image-processing techniques for 2-D/3-D frame-to-frame alignment, change detection, camera control, and tracking of independently moving objects in cluttered scenes. The geo-location of video and tracked objects is estimated by registration of the video to controlled reference imagery, elevation maps, and site models. Finally static, dynamic and reprojected mosaics may be constructed for compression, enhanced visualization, and mapping applications     

采用地平面约束的双目PTZ主从跟踪方法

在视频监控领域,包含PTZ（pan-tilt-zoom）相机的双目主从系统可以同时获取跟踪目标的全景信息和高分辨率信息,因此得到了广泛研究与应用。针对智能视频监控的需求,提出了一种基于地平面约束的双目PTZ主从跟踪方法。该方法分为离线标定和在线实时跟踪两个阶段。离线阶段,利用两相机不同视角间的目标匹配关系计算地平面所诱导的单应矩阵,提出了一种从两相机同步视频流中自动估计单应矩阵的方法,该方法与传统方法相比具有不需要标定物和人工干预的优点,然后采用匹配特征点的方法估计相机的主点和等效焦距。在线实时跟踪时,通过单应变换建立主从相机之间的坐标关联,并利用离线阶段标定的主点和等效焦距估计从相机控制参数,从而实现主从跟踪。与其他算法相比,该方法可以应用于宽基线的情形,能够适应目标深度的变化,满足了实时性的要求。室内、外场景的多组实验验证了所提方法的有效性。     

Homography-based block motion estimation for video coding of PTZ cameras

Due to the constrained movement of pan-tilt-zoom (PTZ) cameras, two frames in the video sequences captured by such cameras can be geometrically related by a relationship (homography). This geometric relationship is helpful for reducing the spatial redundancy in video coding. In this paper, by exploiting the homography between two frames with optical flow tracking algorithm, we propose a novel homography-based search (HBS) algorithm for block motion estimation in coding the sequences captured by PTZ cameras. In addition, adaptive thresholds are adopted in our method to classify different kinds of blocks. Compared with other traditional fast algorithms, the proposed HBS algorithm is proved to be more efficient for the sequences captured by PTZ cameras. And compared to our previous work in ICME (Cui et al., 2011), which only deals with pan-tilt (PT) camera and calculates the homography with mechanical devices, in this extended work we compute the homography by using information on images instead.     

MRANet: Multi-atrous residual attention Network for stereo image super-resolution

In recent years, stereo cameras have been widely used in various fields. Due to the limited resolution of real equipments, stereo image super-resolution (SR) is a very important and hot topic. Recent studies have shown that deep network structures can directly affect feature expression and extraction and thus influence the final results. In this paper, we propose a multi-atrous residual attention stereo super-resolution network (MRANet) with parallax extraction and strong discriminative ability. Specifically, we propose a multi-scale atrous residual attention (MARA) block to obtain receptive fields of different scales through a multi-scale atrous convolution and then combine them with attention mechanisms to extract more diverse and meaningful information. Moreover, we propose a stereo feature fusion unit for stereo parallax extraction and single viewpoint feature refinement and integration. Experiments on benchmark datasets show that MRANet achieves state-of-the-art performance in terms of quantitative metrics and visual quality compared with several SR methods.     

An Introduction to Distributed Smart Cameras

Distributed smart cameras (DSCs) are real-time distributed embedded systems that perform computer vision using multiple cameras. This new approach has emerged thanks to a confluence of simultaneous advances in four key disciplines: computer vision, image sensors, embedded computing, and sensor networks. Processing images in a network of distributed smart cameras introduces several complications. However, we believe that the problems DSCs solve are much more important than the challenges of designing and building a distributed video system. We argue that distributed smart cameras represent key components for future embedded computer vision systems and that smart cameras will become an enabling technology for many new applications. We summarize smart camera technology and applications, discuss current trends, and identify important research challenges.      

双目协同多分辨率主动跟踪方法

PTZ（pan-tilt-zoom）相机由于其具有可变视角和可变分辨率能力,在视频监控领域得到了广泛的应用。该文针对智能监控的需求,提出了一种基于双目PTZ相机的多分辨率主动跟踪方法。该方法分为离线标定和在线协同跟踪两部分。离线标定部分,提出了一种基于图像特征匹配的单目自标定和基于目标运动信息的双目自标定方法,该方法操作简单,无需标定物,在最大程度上减小了对人工干预的依赖,在此基础上推导了系统所具有的两个重要性质;在线协同跟踪部分,设计了一种分段静止的协同跟踪策略。通过实际监控场景下的视频实验,验证方法的有效性和可行性。实验结果表明,该方法可以在复杂环境下有效的主动跟踪目标,在智能监控领域具有较广泛的应用前景。     

Scene categorization based on local–global feature fusion and multi-scale multi-spatial resolution encoding

With the bag-of-contextual-visual-word (BOCVW) models, we propose a scene categorization method based on local–global feature fusion and multi-scale multi-spatial resolution encoding. First, the performances of the BOCVW models belonging to different features are mutually reinforced by fusing other types of features within local regions. Then, the spatial configuration information is explored using a multi-scale multi-spatial resolution encoding approach. Furthermore, these encoded BOCVW models are globally fused using an improved maximum-margin optimization strategy, which considers the margin between input vectors of different categories and the diameter of the smallest ball containing feature vectors simultaneously. The proposed method has been evaluated on three scene categorization datasets consisting of scene categories 8, 15, and 67, respectively. And its effectiveness has been verified by these experimental results.     

An Ant-based Multipath Routing Algorithm for QoS Aware Mobile Ad-hoc Networks

In the wireless ad-hoc network management, Quality of Service (QoS) is an important issue. Along with the QoS ensuring, another desirable property is the network reliability. In data communications, multi-path routing strategy can cope with the problem of traffic overloads while balancing the network resource consumption. In this paper, we propose a new multipath routing algorithm for QoS-sensitive multimedia services. Based on the ant colony optimization technique, the proposed algorithm can establish effective multi-paths to enhance the network reliability. According to the load balancing strategy, data packets are adaptively distributed through the established paths while maintaining an acceptable level of QoS requirement. The most important feature of the proposed approach is its adaptability to current traffic conditions. Simulation results indicate the superior performance of the proposed algorithm, while other schemes cannot offer such an attractive performance balance.     

无线传感器网络自适应功率控制策略

无线传感器网络功率控制技术对于网络的拓扑连通、能量效率、网络容量、吞吐量、实时性等性能均有显著影响,是其实用化的重要支撑技术。该文提出了一种适用于无线传感器网络的自适应功率控制策略APCS(Adaptive Power Control Strategy),该策略是只需要局部信息的分布式算法,通过调整路径损耗指数和功率控制参数可以获得性能极佳的目标拓扑,并能满足实时性和容错能力要求较高的应用场景。另外,该算法还采用了动态功率调整以保持网络的连通性,延长网络的生命周期。仿真结果证实了所提方法的有效性。     

基于Voronoi的无线传感器网络覆盖控制优化策略

针对无线传感器网络运行状态中存在覆盖空洞的问题,提出了一种基于Voronoi有效覆盖区域的空洞侦测修复策略。该策略以满足一定网络区域覆盖质量为前提,在空洞区域内合理增加工作节点以提高网络覆盖率为优化目标,采用几何图形向量方法对节点感知范围和Voronoi多边形的位置特性进行理论分析,力求较准确地计算出空洞面积,找寻最佳空洞修复位置,部署较少的工作节点保证整个网络的连通性。仿真结果表明,该策略能有效地减少网络总节点个数和感知重叠区域,控制网络中冗余节点的存在,同时其收敛速度较快,能够获得比现有算法更高的目标区域空洞修复率,实现网络覆盖控制优化.     

基于ADSP-BF561的视频跟踪系统的研究与实现

设计了基于ADSP-BF561的视频跟踪系统,通过控制摄像机姿态实现对运动目标的自动识别和跟踪。系统采用三帧间差分法与图像分割相结合进行目标识别,利用卡尔曼滤波算法进行目标定位,辅以云台运动控制算法完成目标跟踪。经室内测试,该系统能对运动目标进行有效识别与跟踪,且具有一定的实时性及稳定性。     

Geometry-based distributed scene representation with omnidirectional vision sensors

This paper addresses the problem of efficient representation of scenes captured by distributed omnidirectional vision sensors. We propose a novel geometric model to describe the correlation between different views of a 3-D scene. We first approximate the camera images by sparse expansions over a dictionary of geometric atoms. Since the most important visual features are likely to be equivalently dominant in images from multiple cameras, we model the correlation between corresponding features in different views by local geometric transforms. For the particular case of omnidirectional images, we define the multiview transforms between corresponding features based on shape and epipolar geometry constraints. We apply this geometric framework in the design of a distributed coding scheme with side information, which builds an efficient representation of the scene without communication between cameras. The Wyner-Ziv encoder partitions the dictionary into cosets of dissimilar atoms with respect to shape and position in the image. The joint decoder then determines pairwise correspondences between atoms in the reference image and atoms in the cosets of the Wyner-Ziv image in order to identify the most likely atoms to decode under epipolar geometry constraints. Experiments demonstrate that the proposed method leads to reliable estimation of the geometric transforms between views. In particular, the distributed coding scheme offers similar rate-distortion performance as joint encoding at low bit rate and outperforms methods based on independent decoding of the different images.     

Mobile target tracking algorithm for wireless camera sensor networks with adjustable monitoring direction of nodes

Wireless camera sensor networks (WCSNs) possess a powerful physical environment monitoring capability. Camera nodes with adjustable monitoring directions further improve their flexibility. This study focuses on tracking multiple mobile targets to investigate the node scheduling and target location evaluation strategy of WCSNs on the basis of rotating nodes. By referring to existing research, this study improves the camera node monitoring and rotation model and proposes three network performance evaluation indicators. The proposed algorithm schedules nodes and their monitoring directions by using the unutilized energy of the nodes and the number of monitored targets. It also predicts the moving trends of the targets and selects active nodes by using the locations and linear speeds of the targets. Experimental results show that the proposed algorithm has a high target tracking accuracy. Compared with traditional target tracking algorithms, the proposed algorithm can effectively reduce the number of active nodes, balance the energy consumption between nodes, and prolong network lifetime.