Homography-based block motion estimation for video coding of PTZ cameras

Due to the constrained movement of pan-tilt-zoom (PTZ) cameras, two frames in the video sequences captured by such cameras can be geometrically related by a relationship (homography). This geometric relationship is helpful for reducing the spatial redundancy in video coding. In this paper, by exploiting the homography between two frames with optical flow tracking algorithm, we propose a novel homography-based search (HBS) algorithm for block motion estimation in coding the sequences captured by PTZ cameras. In addition, adaptive thresholds are adopted in our method to classify different kinds of blocks. Compared with other traditional fast algorithms, the proposed HBS algorithm is proved to be more efficient for the sequences captured by PTZ cameras. And compared to our previous work in ICME (Cui et al., 2011), which only deals with pan-tilt (PT) camera and calculates the homography with mechanical devices, in this extended work we compute the homography by using information on images instead.     

Geometry-based distributed scene representation with omnidirectional vision sensors

This paper addresses the problem of efficient representation of scenes captured by distributed omnidirectional vision sensors. We propose a novel geometric model to describe the correlation between different views of a 3-D scene. We first approximate the camera images by sparse expansions over a dictionary of geometric atoms. Since the most important visual features are likely to be equivalently dominant in images from multiple cameras, we model the correlation between corresponding features in different views by local geometric transforms. For the particular case of omnidirectional images, we define the multiview transforms between corresponding features based on shape and epipolar geometry constraints. We apply this geometric framework in the design of a distributed coding scheme with side information, which builds an efficient representation of the scene without communication between cameras. The Wyner-Ziv encoder partitions the dictionary into cosets of dissimilar atoms with respect to shape and position in the image. The joint decoder then determines pairwise correspondences between atoms in the reference image and atoms in the cosets of the Wyner-Ziv image in order to identify the most likely atoms to decode under epipolar geometry constraints. Experiments demonstrate that the proposed method leads to reliable estimation of the geometric transforms between views. In particular, the distributed coding scheme offers similar rate-distortion performance as joint encoding at low bit rate and outperforms methods based on independent decoding of the different images.     

Bioinspired Artificial Eyes: Optic Components,Digital Cameras,and Visual Prostheses

The diverse vision systems found in nature can provide interesting design inspiration for imaging devices, ranging from optical subcomponents to digital cameras and visual prostheses, with more desirable optical characteristics compared to conventional imagers. The advantages of natural vision systems include high visual acuity, wide field of view, wavelength‐free imaging, improved aberration correction and depth of field, and high motion sensitivity. Recent advances in soft materials, ultrathin electronics, and deformable optoelectronics have facilitated the realization of novel processes and device designs that mimic biological vision systems. This review highlights recent progress and continued efforts in the research and development of bioinspired artificial eyes. At first, the configuration of two representative eyes found in nature: a single‐chambered eye and a compound eye, is explained. Then, advances in bioinspired optic components and image sensors are discussed in terms of materials, optical/mechanical designs, and integration schemes. Subsequently, novel visual prostheses as representative application examples of bioinspired artificial eyes are described.     

A Novel Character Segmentation Method for Text Images Captured by Cameras

Due to the rapid development of mobile devices equipped with cameras, instant translation of any text seen in any context is possible. Mobile devices can serve as a translation tool by recognizing the texts presented in the captured scenes. Images captured by cameras will embed more external or unwanted effects which need not to be considered in traditional optical character recognition (OCR). In this paper, we segment a text image captured by mobile devices into individual single characters to facilitate OCR kernel processing. Before proceeding with character segmentation, text detection and text line construction need to be performed in advance. A novel character segmentation method which integrates touched character filters is employed on text images captured by cameras. In addition, periphery features are extracted from the segmented images of touched characters and fed as inputs to support vector machines to calculate the confident values. In our experiment, the accuracy rate of the proposed character segmentation system is 94.90%, which demonstrates the effectiveness of the proposed method.     

光场相机建模与畸变校正改进方法

光场相机作为一种新型的成像系统,可以直接从一次曝光的图像中得到三维信息。为了能够更充分有效地利用光场数据包含的角度和位置信息,完成更加精准的场景深度计算,从而提升光场相机的三维重建的精度,需要实现精确的几何建模,并精确标定其模型参数。该方法从薄透镜模型和小孔成像模型出发,将主透镜建模为薄透镜模型,将微透镜建模为小孔成像模型,结合光场相机双平面模型,将每个提取到的特征点与其在三维空间中的射线建立联系,详细解释了内参矩阵中每个参数的物理意义,以及标定过程中初值确定的过程,并在镜头径向畸变模型的基础上进一步应用了相机镜头的切向畸变模型以及基于射线重投影误差的非线性优化方法,改进了光场相机的标定方法。实验显示,该方法的RMS射线重投影误差为0.332 mm,与经典的Dansereau标定方法相比,进行非线性优化后得到的射线重投影误差精度提升了8%。该方法详细分析的场景点与特定像素索引的推导过程对光场相机的标定具有重要的研究意义,为光场相机光学模型的建立与初始化标定奠定了基础。     

Component forensics

Visual sensor technologies have experienced tremendous growth in recent decades, and digital devices are becoming ubiquitous. Digital images taken by various imaging devices have been used in a growing number of applications, from military and reconnaissance to medical diagnosis and consumer photography. Consequently, a series of new forensic issues arise amidst such rapid advancement and widespread adoption of imaging technologies. For example, one can readily ask what kinds of hardware and software components as well as their parameters have been employed inside these devices? Given a digital image, which imaging sensor or which brand of sensor was used to acquire the image? How was the image acquired? Was it captured using a digital camera, cell phone camera, image scanner, or was it created artificially using an imageediting software? Has the image undergone any manipulation after capture? Is it authentic, or has it been tampered in any way? Does it contain any hidden information or steganographic data? Many of these forensic questions are related to tracing the origin of the digital image to its creation process. Evidence obtained from such analysis would provide useful forensic information to law enforcement, security, and intelligence agencies. Knowledge of image acquisition techniques can also help answer further forensic questions regarding the nature of additional processing that the image might have undergone after capture.     

基于多视点伪序列的光场图像压缩

近年来，作为一种能够提供更富有沉浸感的多媒体媒质，光场图像（Light Field Image，LFI）引起广泛的关注。针对光场图像数据量巨大的问题，本文提出了一种基于多视点伪序列的光场图像高效压缩方案。在编码端，所提方法首先将光场相机捕获得到的原始光场图像根据相机的微透镜阵列分解成子孔径图像。接着根据子孔径图像存在较强视点内和视点间相关性，选取部分子孔径图像进行多视点伪序列构建，基于MV-HEVC设计适用于多视点伪序列的预测编码结构进行编码。在解码端，所提方法基于已解码多视点伪序列通过视频帧插值方法重建出未编码传输的子孔径视图，从而重建出全部光场图像。实验结果表明本文所提算法优于现有基于视差引导稀疏编码的光场图像压缩方法，BD-rate平均节约18.5%，BD-PSNR平均提高1.28dB。      

基于光场相机波前传感器的气动光学效应校正研究

受气动光学效应的影响,来自目标的光波波前会产生动态扰动,导致成像模糊化。常用的校正方法是在测得波前的前提下进行解卷积处理,达到还原图像的效果。传统的波前传感器只能有效测量中心视场,由于存在非等晕问题,导致所能还原的图像区域过小。光场相机波前传感器作为一种新型波前传感器,具有视场大、动态范围大的优点,可以同时探测模糊图像不同区域的点扩散函数,从而一次性还原整幅图像。文章利用Matlab模拟了光场相机的大视场波前探测特性,对气动光学效应引起的模糊图像进行清晰化处理,并与夏克-哈特曼传感器的模拟结果进行了比较。结果表明,光场相机波前传感器可以对气动光学效应造成的波前扰动进行有效的大视场波前探测,一次探测能够清晰化整个视场的图像,且视场范围是传统波前传感器的数倍以上。     

工业相机自热引起像点漂移模型与补偿方法

In order to reduce the influence of temperature on the image point coordinates of industrial cameras in visual measurement, the image point drift caused by the self-heating of the camera was studied, and a compensation method for the thermal image point drift of industrial cameras was proposed. The finite element simulation analysis of the industrial camera model through Ansys Workbench shows that the self-heating of the industrial camera will cause the imaging optical path change and sensor expansion change, quantitatively the influence of the optical path change and sensor expansion on the image point coordinates was analyzed, and the image point drift compensation model was established. A large number of experiments have shown that the image point drift error compensated by the model is reduced from 0.4-0.6 pixel to 0.1-0.2 pixel, which is equivalent to the image point drift suppression effect achieved by hardware thermal control. However, compared with the thermal control device, the method of using the model for compensation has obvious advantages of simple structure and low cost. The temperature compensation model proposed in this research provides a theoretical basis for reducing the image point drift error caused by the self-heating of the camera in the visual measurement.     

基于Kinect深度图像的人体识别分析

介绍了深度图像在模式识别中的研究现状及其在人体识别中的应用。针对目前普通相机拍摄的图像识别在光照、姿态、遮挡等因素影响下性能下降的问题,以微软推出的Kinect设备为平台,通过分析Kinect相机获取的深度图的特征,提出以综合点特征和梯度特征的局域梯度特征的方式来对人体部位区分判定,并以手肘为例作了简要论证。     

Super-resolution reconstruction in a computational compound-eye imaging system

From consumer electronics to biomedical applications, device miniaturization has shown to be highly desirable. This often includes reducing the size of some optical systems. However, diffraction effects impose a constraint on image quality when we simply scale down the imaging parameters. Over the past few years, compound-eye imaging system has emerged as a promising architecture in the development of compact visual systems. Because multiple low-resolution (LR) sub-images are captured, post-processing algorithms for the reconstruction of a high-resolution (HR) final image from the LR images play a critical role in affecting the image quality. In this paper, we describe and investigate the performance of a compound-eye system recently reported in the literature. We discuss both the physical construction and the mathematical model of the imaging components, followed by an application of our super-resolution algorithm in reconstructing the image. We then explore several variations of the imaging system, such as the incorporation of a phase mask in extending the depth of field, which are not possible with a traditional camera. Simulations with a versatile virtual camera system that we have built verify the feasibility of these additions, and we also report the tolerance of the compound-eye system to variations in physical parameters, such as optical aberrations, that are inevitable in actual systems.     

光场相机成像质量评价方法研究

光场相机应用一种新的成像技术,利用光学手段获取四维光场信息,包括目标辐射的二维空间分布信息和辐射传播的二维方向信息。与传统相机相比,光场相机在实际应用中可以获得大的景深范围。由此成像质量评价是光场相机研究中一项十分关键的工作。结合光场成像的特点,对光场相机成像模型进行了分析,完成了实际系统中的光场追迹过程,并对点扩散模型进行了计算,仿真实验结果表明该评价方法有效。     

Demosaicing Method for Digital Cameras with White‐RGB Color Filter Array

Demosaicing, or color filter array (CFA) interpolation, estimates missing color channels of raw mosaiced images from a CFA to reproduce full‐color images. It is an essential process for single‐sensor digital cameras with CFAs. In this paper, a new demosaicing method for digital cameras with Bayer‐like W‐RGB CFAs is proposed. To preserve the edge structure when reproducing full‐color images, we propose an edge direction–adaptive method using color difference estimation between different channels, which can be applied to practical digital camera use. To evaluate the performance of the proposed method in terms of CPSNR, FSIM, and S‐CIELAB color distance measures, we perform simulations on sets of mosaiced images captured by an actual prototype digital camera with a Bayer‐like W‐RGB CFA. The simulation results show that the proposed method demosaics better than a conventional one by approximately +22.4% CPSNR, +0.9% FSIM, and +36.7% S‐CIELAB distance.     

Photo-mosaicing of images of pipe inner surface

This paper describes the algorithm for the construction of continuous visually consistent images of the inner surface of a pipe from a sequence of images acquired by a wide-angle camera that traveled inside the pipe. The algorithm is designed to be a proof of concept and performs well on simulated data (rendered images) even when camera poses (attitude and location) have errors as much as 5%. Photo-mosaics are suitable for traditional (visual) inspection or automatic processing for the detection of manufacturing faults, corroded areas, and cracks. It is demonstrated that the quality of the resulting mosaic depends how the camera is oriented with respect to the pipe axis and that the traditional orientation with an almost collinear camera optical axis and the pipe axis is not the optimal choice. The proposed system is useful for inspection of pipelines that cannot accommodate traditional devices (e.g., pipeline inspection gauges or crawlers), for example, small-scale boilers and gas systems.     

Visual node module: An open‐source extension to the ns‐3 network simulator

A wireless visual sensor network is a collective network of directional and battery‐operated sensor nodes equipped with cameras. The field of view of these nodes depends on the camera opening angle, its direction, and its depth of view. Therefore, coverage and object detection in this type of networks are more challenging compared with the traditional wireless sensor networks. Thus, many researchers propose algorithms and solutions in this field that need tests and simulations. In this paper, we focus on network simulator 3 (ns‐3), which is an open‐source and discrete‐event tool suitable for wireless network simulation targeted primarily for research and educational use. The lack of models that can simulate visual sensor nodes in this simulator motivated us to design and develop a new visual node module as an extension of the ns‐3 core libraries and also to adapt the NetAnim tool to present these nodes graphically. This module will help researchers to simulate, test, and visualize their solutions in wireless visual sensor networks field. In this paper, we present the design and implementation of the proposed module. Furthermore, we show how it can be used in ns‐3 to simulate different scenarios of object detection and visualize the results in NetAnim tool.     

基于三维集成成像相机阵列获取的元素图像校正

利用相机阵列获取三维信息实现三维集成成像与显示时,为消除相机阵列空间位置偏差对元素图像阵列的影响,提高再现三维图像的质量,以相机阵列记录系统为基础提出了一种元素图像阵列校正方法。通过特征点位置坐标以及相机位置平移误差和旋转误差的计算,分析了相机阵列位置平移误差和旋转误差与元素图像间的关系,以及校正算法的精度。利用光学实验对该算法进行了验证,结果表明,此方法可有效消除相机阵列位置偏差对元素图像阵列的影响,并且校正后再现三维图像质量明显优于误差图像,峰值信噪比提高了33.6%,实现了基于三维集成成像相机阵列获取的元素图像校正,满足了集成成像的显示要求。     

Object Detection, Tracking and Recognition for Multiple Smart Cameras

Video cameras are among the most commonly used sensors in a large number of applications, ranging from surveillance to smart rooms for videoconferencing. There is a need to develop algorithms for tasks such as detection, tracking, and recognition of objects, specifically using distributed networks of cameras. The projective nature of imaging sensors provides ample challenges for data association across cameras. We first discuss the nature of these challenges in the context of visual sensor networks. Then, we show how real-world constraints can be favorably exploited in order to tackle these challenges. Examples of real-world constraints are a) the presence of a world plane, b) the presence of a three-dimiensional scene model, c) consistency of motion across cameras, and d) color and texture properties. In this regard, the main focus of this paper is towards highlighting the efficient use of the geometric constraints induced by the imaging devices to derive distributed algorithms for target detection, tracking, and recognition. Our discussions are supported by several examples drawn from real applications. Lastly, we also describe several potential research problems that remain to be addressed.      

一体化EMCCD微光相机的设计

电子倍增电荷耦合器件(E1ectron Multiplying Charge Coupled Device,EMCCD)是一种新型高灵敏度图像传感器。近年来,EMCCD相机在微光探测领域的应用越来越广泛。为了在微光相机中应用新型EMCCD器件,设计了一种探测能力强、数据更新快、具有一体化光纤接口的微光成像系统。主要研究了EMCCD相机的设计方法,说明了EMCCD的工作原理,论述了基于TC253SPD—BO的EMCCD微光相机的设计方案。用成像实验和信噪比测试实验验证了所设计的一体化微光相机的性能。结果表明,该相机不仅可以实现20km以上的数据传输和30f/s的拍摄帧频,而且还可实现弱光条件下的探测功能,并具有较高的系统信噪比。     

Low-Resolution and Low-Quality Face Super-Resolution in Monitoring Scene via Support-Driven Sparse Coding

Monitoring cameras are now widely used to monitor everything from a room in a house to an entire warehouse. However, in real monitoring scenarios, a variety of factors, such as underexposure, optical blurring, defocusing, have an impact on the quality of images, which leads to low-quality and low-resolution (LR) of the individual of interest. Reconstruction of a high-resolution (HR) face image with detailed facial features, from a LR observation based on a set of HR and LR training image pairs, plays an important role in computer vision and face image analysis applications. To super-resolve an HR face given a LR face image, the key issue is how to effectively encode the LR image patch. However, due to stability and accuracy issues, the coding approaches proposed so far are far from satisfactory. In this paper, we present a novel sparse coding method via exploiting the support information on the coding coefficients. According to the distances between the input patch and bases in the dictionary, we first assign different weights to the coding coefficients and then obtain the coding coefficients by solving a weighted sparse problem. Experiments on commonly used databases and some face images on the real monitoring conditions demonstrate that our method outperforms state-of-the-art.     

Efficient Hardware-Based Image Compression Schemes for Wireless Sensor Networks: A Survey

Multidimensional sensors, such as digital camera sensors in the visual sensor networks VSNs generate a huge amount of information compared with the scalar sensors in the wireless sensor networks WSNs. Processing and transmitting such data from low power sensor nodes is a challenging issue through their limited computational and restricted bandwidth requirements in a hardware constrained environment. Source coding can be used to reduce the size of vision data collected by the sensor nodes before sending it to its destination. With image compression, a more efficient method of processing and transmission can be obtained by removing the redundant information from the captured image raw data. In this paper, a survey of the main types of the conventional state of the art image compression standards such as JPEG and JPEG2000 is provided. A literature review of their advantages and shortcomings of the application of these algorithms in the VSN hardware environment is specified. Moreover, the main factors influencing the design of compression algorithms in the context of VSN are presented. The selected compression algorithm may have some hardware-oriented properties such as; simplicity in coding, low memory need, low computational load, and high-compression rate. In this survey paper, an energy efficient hardware based image compression is highly requested to counter the severe hardware constraints in the WSNs.