Intelligent Biometric Group Hand Tracking (IBGHT) database for visual hand tracking research and development

With the increase of innovations in vision-based hand gesture interaction system, new techniques and algorithms are being developed by researchers. However, less attention has been paid on the scope of dismantling hand tracking problems. There is also limited publicly available database developed as benchmark data to standardize the research on hand tracking area. For this purpose, we develop a versatile hand gesture tracking database. This database consists of 60 video sequences containing a total of 15,554 RGB color images. The tracking sequences are captured in different situations ranging from an easy indoor scene to extremely high challenging outdoor scenes. Complete with annotated ground truth data, this database is made available on the web for the sake of assisting other researchers in the related fields to test and evaluate their algorithms based on standard benchmark data.     

Integrating tracking with fine object segmentation

We present a novel method for on-line, joint object tracking and segmentation in a monocular video captured by a possibly moving camera. Our goal is to integrate tracking and fine segmentation of a single, previously unseen, potentially non-rigid object of unconstrained appearance, given its segmentation in the first frame of an image sequence as the only prior information. To this end, we tightly couple an existing kernel-based object tracking method with Random Walker-based image segmentation. Bayesian inference mediates between tracking and segmentation, enabling effective data fusion of pixel-wise spatial and color visual cues. The fine segmentation of an object at a certain frame provides tracking with reliable initialization for the next frame, closing the loop between the two building blocks of the proposed framework. The effectiveness of the proposed methodology is evaluated experimentally by comparing it to a large collection of state of the art tracking and video-based object segmentation methods on the basis of a data set consisting of several challenging image sequences for which ground truth data is available.     

Region of interest extraction and virtual camera control based on panoramic video capturing

We present a system for automatically extracting the region of interest (ROI) and controlling virtual cameras' control based on panoramic video. It targets applications such as classroom lectures and video conferencing. For capturing panoramic video, we use the FlyCam system that produces high resolution, wide-angle video by stitching video images from multiple stationary cameras. To generate conventional video, a region of interest can be cropped from the panoramic video. We propose methods for ROI detection, tracking, and virtual camera control that work in both the uncompressed and compressed domains. The ROI is located from motion and color information in the uncompressed domain and macroblock information in the compressed domain, and tracked using a Kalman filter. This results in virtual camera control that simulates human controlled video recording. The system has no physical camera motion and the virtual camera parameters are readily available for video indexing.     

基于Contourlet变换的多波段SAR图像伪彩色融合

针对多波段SAR图像融合问题, 提出了一种基于contourlet变换的伪彩色图像融合方法. 该方法首先利用contourlet变换多尺度, 多方向以及各向异性的良好特性对不同波段SAR图像进行多尺度分解, 采用边缘信息量测因子策略融合方向高频子带, 加权平均策略融合低通子带. 然后对灰度融合图像利用混合高频提升滤波方法得到RGB彩色空间的三个颜色通道, 并在RGB空间量化显示,把人眼难以分辨的灰度信息转化为可分辨的色彩, 保持SAR图像的空间分辨率的同时增强了光谱分辨率. 采用Ku和L波段SAR图像进行融合实验, 并用客观评价因子对融合质量进行评价, 结果验证了该方法的有效性.     

点云下地平面检测的RGB-D相机外参自动标定

目的 RGB-D相机的外参数可以被用来将相机坐标系下的点云转换到世界坐标系的点云,可以应用在3维场景重建、3维测量、机器人、目标检测等领域。 一般的标定方法利用标定物（比如棋盘）对RGB-D彩色相机的外参标定,但并未利用深度信息,故很难简化标定过程,因此,若充分利用深度信息,则极大地简化外参标定的流程。基于彩色图的标定方法,其标定的对象是深度传感器,然而,RGB-D相机大部分则应用基于深度传感器,而基于深度信息的标定方法则可以直接标定深度传感器的姿势。方法 首先将深度图转化为相机坐标系下的3维点云,利用MELSAC方法自动检测3维点云中的平面,根据地平面与世界坐标系的约束关系,遍历并筛选平面,直至得到地平面,利用地平面与相机坐标系的空间关系,最终计算出相机的外参数,即相机坐标系内的点与世界坐标系内的点的转换矩阵。结果 实验以棋盘的外参标定方法为基准,处理从PrimeSense相机所采集的RGB-D视频流,结果表明,外参标定平均侧倾角误差为-1.14°,平均俯仰角误差为4.57°,平均相机高度误差为3.96 cm。结论 该方法通过自动检测地平面,准确估计出相机的外参数,具有很强的自动化,此外,算法具有较高地并行性,进行并行优化后,具有实时性,可应用于自动估计机器人姿势。     

基于颜色自适应字典的双相机光谱系统重建算法

通过将快照编码孔径光谱成像和普通RGB彩色成像结合,双相机光谱成像系统能够高效地获取场景的光谱信息,具有广阔的应用前景.如何高质量地从压缩采样中重建高光谱图像是该系统需要解决的重要问题.根据高光谱图像与彩色图像在空间结构和光谱响应上的相关性,本文了提出一种基于颜色自适应字典的重建算法,用以提高双相机光谱成像系统的重建质量.首先,利用RGB观测分别训练三通道非负字典.然后,以彩色相机的光谱响应曲线为指导,为每一个谱带选择光谱相关性最大的字典.最后,完成高光谱图像的稀疏重建.高光谱数据库和遥感数据库的仿真结果均表明,本文提出的算法能够大幅度提升双相机光谱成像系统的重建质量.     

Adaptive skin color modeling using the skin locus for selecting training pixels

Techniques for color-based tracking of faces or hands often assume a static skin model yet skin color, as measured by a camera, can change when lighting changes. Therefore, for robust skin pixel detection, an adaptive skin color model must be employed. We demonstrate a chromaticity-based constraint to select training pixels in a scene for updating a dynamic skin color model under changing illumination conditions. The method makes use of the ‘skin locus’ of a camera, that is, the area in chromaticity space where skin chromaticity under various lighting and camera calibration conditions is observed. Skin color models derived from the technique are compared with that derived by a common spatial constraint and is shown to be more consistent with manually extracted ground truth skin model per frame even as localization errors increase. The technique is applied to color-based face tracking in indoor and outdoor videos and is shown to succeed more often than other color model adaptation techniques.     

Free Form Incident Light Fields

This paper presents methods for photo‐realistic rendering using strongly spatially variant illumination captured from real scenes. The illumination is captured along arbitrary paths in space using a high dynamic range, HDR, video camera system with position tracking. Light samples are rearranged into 4‐D incident light fields (ILF) suitable for direct use as illumination in renderings. Analysis of the captured data allows for estimation of the shape, position and spatial and angular properties of light sources in the scene. The estimated light sources can be extracted from the large 4D data set and handled separately to render scenes more efficiently and with higher quality. The ILF lighting can also be edited for detailed artistic control.     

基于阵列相机的多光谱成像系统光谱重建算法

针对传统三通道RGB相机在光源光谱已知条件下不能完全恢复物体表面光谱反射率的缺点,本文构造一套多光谱成像阵列相机系统。该阵列相机采用12个大恒DH-HV1300FM型相机,且11个镜头装有波长不同的滤光片。本文结合阵列相机多通道数的优势,提出一种MSIS-GOC（Multi-spectral Imaging System based on Group of Camera）算法,能够可靠并有效地重建场景的光谱反射率。仿真实验结果分析验证了该系统的有效性。     

Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration

3D gaze tracking from a single RGB camera is very challenging due to the lack of information in determining the accurate gaze target from a monocular RGB sequence. The eyes tend to occupy only a small portion of the video, and even small errors in estimated eye orientations can lead to very large errors in the triangulated gaze target. We overcome these difficulties with a novel lightweight eyeball calibration scheme that determines the user-specific visual axis, eyeball size and position in the head. Unlike the previous calibration techniques, we do not need the ground truth positions of the gaze points. In the online stage, gaze is tracked by a new gaze fitting algorithm, and refined by a 3D gaze regression method to correct for bias errors. Our regression is pre-trained on several individuals and works well for novel users. After the lightweight one-time user calibration, our method operates in real time. Experiments show that our technique achieves state-of-the-art accuracy in gaze angle estimation, and we demonstrate applications of 3D gaze target tracking and gaze retargeting to an animated 3D character.     

An indoor localization dataset and data collection framework with high precision position annotation

We introduce a novel technique and an associated high resolution dataset that aim to precisely evaluate wireless signal based indoor positioning algorithms. The technique implements an augmented reality (AR) based positioning system that is used to annotate the wireless signal parameter data samples with high precision position data. We track the position of a practical and low cost navigable setup of cameras and a Bluetooth Low Energy (BLE) beacon in an area decorated with AR markers. We maximize the performance of the AR-based localization by using a redundant number of markers. Video streams captured by the cameras are subjected to a series of marker recognition, subset selection and filtering operations to yield highly precise pose estimations. Our results show that we can reduce the positional error of the AR localization system to a rate under 0.05 meters. The position data are then used to annotate the BLE data that are captured simultaneously by the sensors stationed in the environment, hence, constructing a wireless signal dataset with the ground truth, which allows a wireless signal based localization system to be evaluated accurately.     

Universal demosaicking for imaging pipelines with an RGB color filter array

A universal demosaicking solution appropriate for imaging pipelines employing a red-green-blue (RGB) color filter array (CFA) is introduced. The proposed solution can demosaick a raw, grayscale, CFA sensor image captured by a digital camera equipped with any RGB-CFA currently in use. The solution utilizes a spectral model, an edge-sensing mechanism, and a postprocessor to preserve the coloration and sharpness of the captured image. The framework readably unifies existing demosaicking solutions which differ in design characteristics, performance, and computational efficiency. Simulation studies indicate that the universal demosaicking framework allows for cost-effective camera image processing and produces visually pleasing full-color digital images.     

多标记室内小型无人机定位与姿态估计方法

随着无人机（Unmanned Aerial Vehicle,UAV）小型化、轻便化的发展,因其价格低廉,以及在娱乐和服务领域的广泛使用的特点,使得如何实现一个便捷且易实现的自主飞行跟踪系统成为关注点。由于无人机在室内GPS信号弱,使得跟踪与姿态获取成为进一步室内无人机自主控制的重点与难点。与动辄几十万美元搭建的无人机跟踪系统相比,采用低成本单目摄像机的无人机跟踪系统具有更高的科研价值和更广泛的应用前景。针对目前流行的基于增强现实（Augmented Reality,AR）技术的ArUco标记算法和颜色空间域标记算法,设计了一种多标记的无人机跟踪系统。在无人机目标跟踪过程中比较两种方法,验证了两种方法非接触式深度传感器无人机跟踪和姿态估计的效果,并比较了两种方法对空间亮度与空间颜色复杂度的鲁棒性,以及不同跟踪距离下视频中无人机检出率与跟踪精度。实验结果表明,基于深度摄像机获得的无人机位置和姿态数据,无人机可以进行自主的PID控制飞行,且AR标记在复杂环境下无人机的检出率、跟踪实时性、姿态估计精度以及鲁棒性都优于颜色标记,为之后室内无人机在非接触式传感的控制、路径规划、自主规避等进一步实验研究提供了无人机的位置和姿态数据。     

Personalized production of basketball videos from multi-sensored data under limited display resolution

Integration of information from multiple cameras is essential in television production or intelligent surveillance systems. We propose an autonomous system for personalized production of basketball videos from multi-sensored data under limited display resolution. The problem consists in selecting the right view to display among the multiple video streams captured by the investigated camera network. A view is defined by the camera index and the parameters of the image cropped within the selected camera. We propose criteria for optimal planning of viewpoint coverage and camera selection. Perceptual comfort is discussed as well as efficient integration of contextual information, which is implemented by smoothing generated viewpoint/camera sequences to alleviate flickering visual artifacts and discontinuous story-telling artifacts. We design and implement the estimation process and verify it by experiments, which shows that our method efficiently reduces those artifacts.     

Reinterpretable Imager: Towards Variable Post‐Capture Space,Angle and Time Resolution in Photography

We describe a novel multiplexing approach to achieve tradeoffs in space, angle and time resolution in photography. We explore the problem of mapping useful subsets of time‐varying 4D lightfields in a single snapshot. Our design is based on using a dynamic mask in the aperture and a static mask close to the sensor. The key idea is to exploit scene‐specific redundancy along spatial, angular and temporal dimensions and to provide a programmable or variable resolution tradeoff among these dimensions. This allows a user to reinterpret the single captured photo as either a high spatial resolution image, a refocusable image stack or a video for different parts of the scene in post‐processing. A lightfield camera or a video camera forces a‐priori choice in space‐angle‐time resolution. We demonstrate a single prototype which provides flexible post‐capture abilities not possible using either a single‐shot lightfield camera or a multi‐frame video camera. We show several novel results including digital refocusing on objects moving in depth and capturing multiple facial expressions in a single photo.     

数码印染业中的光谱颜色量化

印染业中,用测色仪器测量颜色及传统颜色测量方法存在着测色精度低、测量速度慢、测量效果依赖于设备等不足,基于此问题,通过对物体颜色光谱特性的研究,提出一种数学模型结合扫描仪或数码相机等数字图像输入设备的光谱颜色定量化方法,首先经扫描仪／照相机获取物体的颜色信息,在此基础上,利用多项式模型预测各个像素RGB值对应的XY-ZLMS值,再通过光谱重构算法,将XYZLMS值转化为光谱反射比,最终用光谱反射比来对颜色进行表征量化。实验过程选用自制色卡为样本,对整个颜色量化过程进行数据分析和仿真,结果证明：该方法克服了测色仪器测量及传统颜色测量方法上的不足之处,达到了颜色测量的高精度,快捷,方便和多样化的需求。     

基于HSI彩色空间模型的域替换降噪算法

对彩色数字图像进行降噪处理时,为兼顾标量处理法的简便性和矢量处理法的有效性,提出一种对彩色空间域实行替换的方法。采用均值滤波法在RGB域内进行滤波处理,将图像数据从RGB域转换到HSI域,用滤波处理后的H域、S域替换含有噪声的H域、 S域。对由双目内窥镜采集的实际彩色图像进行实验,结果证明该域替换法可以结合标量处理和矢量处理的优点,达到比一般标量处理法及简单矢量处理法更好的降噪效果。     

基于双目视觉的盆腔微创手术增强现实导航仿真系统的设计

基于内窥镜视觉导航的盆腔微创手术,往往因为病灶的复杂解剖位置以及内窥镜视野的局限性,对手术的顺利开展提出了重要的挑战。针对这个问题,设计并开发一套基于立体视觉的盆腔微创手术的增强现实（AR）导航仿真系统。首先,利用术前的CT影像重建骨盆3D模型以及盆腔的真实手术视频,生成带有纹理信息的骨盆3D模型,仿真一套具有真实轨迹的手术视频;然后,利用基于可视点颜色一致性的2D/3D配准技术,实现术前重建模型与手术视野的初始化配准。利用立体视觉跟踪算法,对手术过程中的内窥镜位置进行跟踪,根据内窥镜多自由度的变换矩阵来实现术前3D模型与手术视野的融合与增强现实导航。估计轨迹与真实轨迹的均方根误差为2.3933 mm,仿真实验表明,导航系统为视觉导航提供良好的增强现实显示效果。     

Single Sensor Compressive Light Field Video Camera

This paper presents a novel compressed sensing (CS) algorithm and camera design for light field video capture using a single sensor consumer camera module. Unlike microlens light field cameras which sacrifice spatial resolution to obtain angular information, our CS approach is designed for capturing light field videos with high angular, spatial, and temporal resolution. The compressive measurements required by CS are obtained using a random color-coded mask placed between the sensor and aperture planes. The convolution of the incoming light rays from different angles with the mask results in a single image on the sensor; hence, achieving a significant reduction on the required bandwidth for capturing light field videos. We propose to change the random pattern on the spectral mask between each consecutive frame in a video sequence and extracting spatio-angular-spectral-temporal 6D patches. Our CS reconstruction algorithm for light field videos recovers each frame while taking into account the neighboring frames to achieve significantly higher reconstruction quality with reduced temporal incoherencies, as compared with previous methods. Moreover, a thorough analysis of various sensing models for compressive light field video acquisition is conducted to highlight the advantages of our method. The results show a clear advantage of our method for monochrome sensors, as well as sensors with color filter arrays.