Nonstationary color tracking for vision-based human-computer interaction

Skin color offers a strong cue for efficient localization and tracking of human body parts in video sequences for vision-based human-computer interaction. Color-based target localization could be achieved by analyzing segmented skin color regions. However, one of the challenges of color-based target tracking is that color distributions would change in different lighting conditions such that fixed color models would be inadequate to capture nonstationary color distributions over time. Meanwhile, using a fixed skin color model trained by the data of a specific person would probably not work well for other people. Although some work has been done on adaptive color models, this problem still needs further studies. We present our investigation of color-based image segmentation and nonstationary color-based target tracking, by studying two different representations for color distributions. We propose the structure adaptive self-organizing map (SASOM) neural network that serves as a new color model. Our experiments show that such a representation is powerful for efficient image segmentation. Then, we formulate the nonstationary color tracking problem as a model transduction problem, the solution of which offers a way to adapt and transduce color classifiers in nonstationary color distributions. To fulfill model transduction, we propose two algorithms, the SASOM transduction and the discriminant expectation-maximization (EM), based on the SASOM color model and the Gaussian mixture color model, respectively. Our extensive experiments on the task of real-time face/hand localization show that these two algorithms can successfully handle some difficulties in nonstationary color tracking. We also implemented a real-time face/hand localization system based on such algorithms for vision-based human-computer interaction.     

基于视觉的三维重建技术综述*

基于视觉的三维重建仍然存在较大局限性。通过介绍基于视觉的三维重建技术的主要方法及其研究现状,对各种方法优缺点进行了比较分析,期望能够对该领域有较全面的把握,进一步明确未来的研究方向。     

Real-time validation of vision-based over-height vehicle detection system

Over-height vehicle strikes with low bridges and tunnels are an ongoing problem worldwide. While previous methods have used vision-based systems to address the over-height warning problem, such methods are sensitive to wind. In this paper, we perform a full validation of the system using a constraint-based approach to minimize the number of over-height vehicle misclassifications due to windy conditions. The dataset includes a total of 102 over-height vehicles recorded at frame rates of 25 and 30fps. An analysis is performed of wind and vehicle displacements to track over-height features using optical flow paired with SURF feature detectors. Motion captured within the region of interest was treated as a standard two-class binary linear classification problem with 1 indicating over-height vehicle presence and 0 indicating noise. The algorithm performed with 100% recall, 83.3% precision, false positive rate of 0.2% and warning accuracy of 96.6%.     

Analysis of camera movement errors in vision-based vehicle tracking

When a camera is used to provide the navigational parameters in autonomous vehicle operations, it is subjected to unexpected movements or vibrations of the mounting platform. This paper presents a framework for analyzing the effect of uncontrollable camera movements on the navigational parameters, in particular on the range and-heading angle in vision-based vehicle tracking. The noise introduced by the platform movements is modeled in two ways: camera noise approach and image noise approach. The parameter space of the camera is divided into a controllable subspace consisting of its height and depression angle, and an uncontrollable subspace consisting of the tracked object coordinates and rotation angle errors. A consistent detectable region is then obtained such that the tracked object is always seen by the camera. Based on this region, a reliable region consisting of no singularity points is established so that the range error does not become infinity. The optimum parameters of the controllable subspace with respect to the uncontrollable subspace are found by employing two estimation schemes: (a) the mini-max estimator to provide the worst case effect, and (b) the minimum-mean-square estimator to provide the average or overall effect. From the results obtained, it is shown how an optimum imaging geometry of a monocular vision-based tracking system can be designed in order to satisfy prescribed levels of range and heading angle errors     

融合检测和跟踪的实时人脸跟踪

目的 在实时人脸跟踪过程中,因光照变化、目标被遮挡以及跟踪时间长等因素,导致的误差累积都会影响系统的整体性能。针对这些问题,提出一种融合检测和跟踪技术的方法,其中包含了检测、控制和跟踪3个模块(简称DCT)。方法 在检测模块中,利用AdaBoost算法提取人脸的相关信息,并将信息传递给跟踪模块进行跟踪处理;在跟踪模块中,采用在线随机蕨和SURF(speeded up robust features)算法对目标进行跟踪。同时,在每次检测到目标之后,会通过控制模块对当前跟踪目标准确性进行判断。结果 选取国际标准数据组并与LBP+Camshift+Kalman滤波算法、SEMI算法、TLD(tracking-learning-detection)算法比较,实验结果表明,DCT方法在目标发生尺度较大变化、目标遮挡、旋转、形变以及光照发生变化时都具有良好的跟踪识别效果,DCT方法识别准确率在95%以上,平均误识别率和漏识别率分别为0.86%和0.78%。结论 DCT方法具有消除误差累积,跟踪失败后自动恢复等特点,同时可以消除环境中光照、遮挡和仿射变换的影响并满足系统跟踪的实时性要求,运用于视频人脸跟踪系统中能够提高系统的实时性及鲁棒性。     

Real-time robust vision-based hand gesture recognition using stereo images

This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.     

An efficient camera calibration method for vision-based head tracking

The aim of this study is to develop and evaluate an efficient camera calibration method for vision-based head tracking. Tracking head movements is important in the design of an eye-controlled human/computer interface. A vision-based head tracking system is proposed to allow the user's head movements in the design of the eye-controlled human/computer interface. We propose an efficient camera calibration method to track the three-dimensional position and orientation of the user's head accurately. We also evaluate the performance of the proposed method and the influence of the configuration of calibration points on the performance. The experimental error analysis results showed that the proposed method can provide more accurate and stable pose (i.e. position and orientation) of the camera than the direct linear transformation method which has been used in camera calibration. The results for this study can be applied to the tracking of head movements related to the eye-controlled human/computer interface and the virtual reality technology.     

Novel target segmentation and tracking based on fuzzy membership distribution for vision-based target tracking system

One of the basic processes of a vision-based target tracking system is the detection process that separates an object from the background in a given image. A novel target detection technique for suppression of the background clutter is presented that uses a predicted point that is estimated from a tracking filter. For every pixel, the three-dimensional feature that is composed of the x-position, the y-position and the gray level of its position is used for evaluating the membership value that describes the probability of whether the pixel belongs to the target or to the background. These membership values are transformed into the membership level histogram. We suggest an asymmetric Laplacian model for the membership distribution of the background pixel and determine the optimal membership value for detecting the target region using the likelihood criterion. The proposed technique is applied to several infra-red image sequences and CCD image sequences to test segmentation and tracking. The feasibility of the proposed method is verified through comparison of the experimental results with the other techniques.     

Real-time identification of disaster areas by an open-access vision-based tool

Structural damages caused by natural catastrophic events cover a wide area and it is convenient to supervise the event consequences by vision tools. The aim of this paper is to supply a rapid damage detector designed as a way to aid in risk assessment, damage control and disaster prevention as well as a way to speed the examination of catastrophic effects for emergency studies. The satellite pictures covering the area of interest represent the required bits of information to manage the developed telematics tool. A case study is discussed in order to provide experimental evidence of the proposed procedure potential. Moreover, a multi-view image/video fusion system is integrated in the image process to detect the damage levels of structures to overcome the limitations on the vertical information provided by a satellite. In synthesis, this study shows how a GIS-based real time monitoring system can be effectively used for a rapid evaluation of structural damage and disaster management.     

Real-time fingertip tracking and gesture recognition

Augmented desk interfaces and other virtual reality systems depend on accurate, real-time hand and fingertip tracking for seamless integration between real objects and associated digital information. We introduce a method for discerning fingertip locations in image frames and measuring fingertip trajectories across image frames. We also propose a mechanism for combining direct manipulation and symbolic gestures based on multiple fingertip motions. Our method uses a filtering technique, in addition to detecting fingertips in each image frame, to predict fingertip locations in successive image frames and to examine the correspondences between the predicted locations and detected fingertips. This lets us obtain multiple complex fingertip trajectories in real time and improves fingertip tracking. This method can track multiple fingertips reliably even on a complex background under changing lighting conditions without invasive devices or color markers.     

Data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets.     

Real-time tracking using trust-region methods

Optimization methods based on iterative schemes can be divided into two classes: line-search methods and trust-region methods. While line-search techniques are commonly found in various vision applications, not much attention is paid to trust-region ones. Motivated by the fact that line-search methods can be considered as special cases of trust-region methods, we propose to establish a trust-region framework for real-time tracking. Our approach is characterized by three key contributions. First, since a trust-region tracking system is more effective, it often yields better performances than the outcomes of other trackers that rely on iterative optimization to perform tracking, e.g., a line-search-based mean-shift tracker. Second, we have formulated a representation model that uses two coupled weighting schemes derived from the covariance ellipse to integrate an object's color probability distribution and edge density information. As a result, the system can address rotation and nonuniform scaling in a continuous space, rather than working on some presumably possible discrete values of rotation angle and scale. Third, the framework is very flexible in that a variety of distance functions can be adapted easily. Experimental results and comparative studies are provided to demonstrate the efficiency of the proposed method.     

Real-time vision-based navigation and 3D depth estimation for an indoor autonomous mobile robot

A Transputer based artificial vision system that allows estimating a mobile robot's absolute position (navigation), it's motion parameters (egomotion) and a depth map of the sight of view is presented. Navigation is achieved by tracking expected features present in basic 3D CAD information of the environment. Egomotion and relative depth are estimated using optic flow calculated on closed contours of points in the scene presenting high spatio-temporal gradients and require no a-priori knowledge.     

Heads up and camera down: a vision-based tracking modality for mobile mixed reality

Anywhere Augmentation pursues the goal of lowering the initial investment of time and money necessary to participate in mixed reality work, bridging the gap between researchers in the field and regular computer users. Our paper contributes to this goal by introducing the GroundCam, a cheap tracking modality with no significant setup necessary. By itself, the GroundCam provides high frequency, high resolution relative position information similar to an inertial navigation system, but with significantly less drift. We present the design and implementation of the GroundCam, analyze the impact of several design and run-time factors on tracking accuracy, and consider the implications of extending our GroundCam to different hardware configurations. Motivated by the performance analysis, we developed a hybrid tracker that couples the GroundCam with a wide area tracking modality via a complementary Kalman filter, resulting in a powerful base for indoor and outdoor mobile mixed reality work. To conclude, the performance of the hybrid tracker and its utility within mixed reality applications is discussed.     

Real-time visual tracking of complex structures

Presents a framework for three-dimensional model-based tracking. Graphical rendering technology is combined with constrained active contour tracking to create a robust wire-frame tracking system. It operates in real time at video frame rate (25 Hz) on standard hardware. It is based on an internal CAD model of the object to be tracked which is rendered using a binary space partition tree to perform hidden line removal. A Lie group formalism is used to cast the motion computation problem into simple geometric terms so that tracking becomes a simple optimization problem solved by means of iterative reweighted least squares. A visual servoing system constructed using this framework is presented together with results showing the accuracy of the tracker. The paper then describes how this tracking system has been extended to provide a general framework for tracking in complex configurations. The adjoint representation of the group is used to transform measurements into common coordinate frames. The constraints are then imposed by means of Lagrange multipliers. Results from a number of experiments performed using this framework are presented and discussed     

Real-time vehicle tracking for driving assistance

Detecting car taillights at night is a task which can nowadays be accomplished very fast on cheap hardware. We rely on such detections to build a vision-based system that, coupling them in a rule-based fashion, is able to detect and track vehicles. This allows the generation of an interface that informs a driver of the relative distance and velocity of other vehicles in real time and triggers a warning when a potentially dangerous situation arises. We demonstrate the system using sequences shot using a camera mounted behind a car’s windshield.     

基于卷积神经网络的实时跟踪算法

利用卷积神经网络(CNN)强大的特征学习能力,提出了一种基于卷积神经网络的实时跟踪算法.通过对双通道卷积神经网络进行离线训练,学习相邻两帧之间的差异,得到跟踪目标的表观特征与运动之间的普遍规律.在不需要对网络模型在线更新的情况下,直接通过网络回归得到对目标的位置和对应置信度的预测.在VOT2014数据集中进行实验,结果表明:提出的跟踪算法的性能达到了当前领先水平.同时,跟踪算法的运行速度可以达到90帧/s,表现出非常不错的实时性.     

Real-time shape tracking of facial landmarks

Detection of facial landmarks and accurate tracking of their shape are essential in real-time applications such as virtual makeup, where users can see the makeup’s effect by moving their face in diverse directions. Typical face tracking techniques detect facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are enough to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to recalculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet and the point loss problem of the KLT point tracker, which leads to accurate real-time shape tracking. We performed several experiments to evaluate the performance of our method and report some of the results herein.