Dynamic disparity adjustment and histogram-based filtering of range data for fast 3-D hand tracking

Range data is very important in human-computer interaction applications. Although less costly, range acquisition and processing still presents a speed vs data reliability tradeoff. This paper proposes a method that, given noisy and generally unreliable range data, can filter out erroneous information using range histograms for regions of interest selected in registered color data. Then, using the resulting consistent data that has passed filters, this method limits the depth search space dynamically using motion history and its current state. Experimental results demonstrate the success of the proposed algorithm. Using filtered range data, the algorithm correctly identified the hand involved in manipulation 99.85% of the time. Dynamic disparity adjustment produced a speedup of 60.17% over a static disparity range selection. An application to virtual reality navigation is also presented.     

Real-time fingertip localization conditioned on hand gesture classification

A method to obtain accurate hand gesture classification and fingertip localization from depth images is proposed. The Oriented Radial Distribution feature is utilized, exploiting its ability to globally describe hand poses, but also to locally detect fingertip positions. Hence, hand gesture and fingertip locations are characterized with a single feature calculation. We propose to divide the difficult problem of locating fingertips into two more tractable problems, by taking advantage of hand gesture as an auxiliary variable. Along with the method we present the ColorTip dataset, a dataset for hand gesture recognition and fingertip classification using depth data. ColorTip contains sequences where actors wear a glove with colored fingertips, allowing automatic annotation. The proposed method is evaluated against recent works in several datasets, achieving promising results in both gesture classification and fingertip localization.     

Position estimation using principal components of range data

This paper describes a new approach to mobile robot position estimation, based on principal component analysis of laser range data. An eigenspace is constructed from the principal components of a large number of range data sets. The structure of an environment, as seen by a range sensor, is represented as a family of surfaces in this space. Subsequent range data sets from the environment project as a point in this space. Associating this point to the family of surfaces gives a set of candidate positions and orientations (poses) for the sensor. These candidate poses correspond to positions and orientations in the environment which have similar range profiles. A Kalman filter can be used to select the most likely candidate pose based on coherence with small movements.

The first part of this paper describes how a relatively small number of depth profiles of an environment can be used to generate a complete eigenspace. This space is used to build a representation of the range scan profiles obtained from a regular grid of positions and orientations (poses). This representation has the form of a family of surface (a manifold). This representation converts the problem of associating a range profile to possible positions and orientations into a table lookup. As a side benefit, the method provides a simple means to detect obstacles in a range profile. The final section of the paper reviews the use of estimation theory to determine the correct pose hypothesis by tracking.     

Real-time hand tracking using a mean shift embedded particle filter

Particle filtering and mean shift (MS) are two successful approaches to visual tracking. Both have their respective strengths and weaknesses. In this paper, we propose to integrate advantages of the two approaches for improved tracking. By incorporating the MS optimization into particle filtering to move particles to local peaks in the likelihood, the proposed mean shift embedded particle filter (MSEPF) improves the sampling efficiency considerably. Our work is conducted in the context of developing a hand control interface for a robotic wheelchair. We realize real-time hand tracking in dynamic environments of the wheelchair using MSEPF. Extensive experimental results demonstrate that MSEPF outperforms the MS tracker and the conventional particle filter in hand tracking. Our approach produces reliable tracking while effectively handling rapid motion and distraction with roughly 85% fewer particles. We also present a simple method for dynamic gesture recognition. The hand control interface based on the proposed algorithms works well in dynamic environments of the wheelchair.     

An efficient posture recognition method using fuzzy logic

Computer-human interaction plays an important role in virtual reality. Glove-based input devices have many desirable features which make direct interactions between the user and the virtual world possible. However, due to the complexity of the human hand, recognising hand functions precisely and efficiently is not an easy task. Existing algorithms are either imprecise or computationally expensive, making them impractical to integrate with VR applications, which are usually very CPU intensive.In the problem of posture and gesture recognition, both the sample patterns stored in the database and the ones to be recognised may be imprecise. This kind of imprecise knowledge can be best dealt with using fuzzy logic. A fast and simple posture recognition method using fuzzy logic is presented in this paper. Our model consists of three components: the posture database, the classifier and the identifier. The classifier roughly classifies the sample postures before they are put into the posture database. The identifier compares an input posture with the records in the identified class and finds the right match efficiently. Fuzzy logic is applied in both the classification and identification processes to cope with imprecise data. The main goal of this method is to recognise hand functions in an accurate and efficient manner. The accuracy, efficiency and the noise tolerance of the model have been examined through a number of experiments.     

Classification of hand postures against complex backgrounds using elastic graph matching

A system for person-independent classification of hand postures against complex backgrounds in video images is presented. The system employs elastic graph matching, which has already been successfully applied for object and face recognition. We use the bunch graph technique to model variance in hand posture appearance between different subjects and variance in backgrounds. Our system does not need a separate segmentation stage but closely integrates finding the object boundaries with posture classification.     

Recognition of dynamic hand gestures

This paper is concerned with the problem of recognition of dynamic hand gestures. We have considered gestures which are sequences of distinct hand poses. In these gestures hand poses can undergo motion and discrete changes. However, continuous deformations of the hand shapes are not permitted. We have developed a recognition engine which can reliably recognize these gestures despite individual variations. The engine also has the ability to detect start and end of gesture sequences in an automated fashion. The recognition strategy uses a combination of static shape recognition (performed using contour discriminant analysis), Kalman filter based hand tracking and a HMM based temporal characterization scheme. The system is fairly robust to background clutter and uses skin color for static shape recognition and tracking. A real time implementation on standard hardware is developed. Experimental results establish the effectiveness of the approach.     

Robust face recognition using 2D and 3D data: Pose and illumination compensation

The paper addresses the problem of face recognition under varying pose and illumination. Robustness to appearance variations is achieved not only by using a combination of a 2D color and a 3D image of the face, but mainly by using face geometry information to cope with pose and illumination variations that inhibit the performance of 2D face recognition. A face normalization approach is proposed, which unlike state-of-the-art techniques is computationally efficient and does not require an extended training set. Experimental results on a large data set show that template-based face recognition performance is significantly benefited from the application of the proposed normalization algorithms prior to classification.     

An interactive image clipping system using hand motion recognition

We present an efficient hand recognition algorithm for an interactive image clipping system, which is widely used for environments such as public facilities and security environments where personal capturing devices including mobile phones are not allowed. User-friendly interface and accurate image capturing function are required for an image clipping system. We build the system by combining Microsoft Kinect, HD webcam and projector. The Kinect and webcam are used to capture the motions of users׳ hand and project is to display the user-selected area from the capturing material. Hand recognition is composed of three steps: (i) the region occupied by users׳ hand is extracted from an image, (ii) the fingertips of the extracted hand region are analyzed using k-curvature algorithm, and (iii) the height of the fingertip is estimated using the depth image from Kinect. The height of the fingertip informs whether users׳ finger touched the surface of the target. The region captured by the fingertip is clipped from the image and stored as the target image. The excellence of our hand recognition algorithm is proved through a user test.     

基于BOF-Gist特征的手势识别算法研究

针对静态手势识别算法存在特征计算复杂度高,实时性差的问题,提出了一种新的BOF-Gist特征对手势图像进行表示。该特征的优势是在保持Gist特征原有优势的基础上,有效地表征手势图像的局部特征和全局特征,并且特征维数明显降低,实时性好。在标准数据库上的测试表明,该算法对于简单、复杂背景下十种手语手势分别得到了90.42%与79.05%的正确识别率,同时验证了算法的实时性。     

Real-time measurement of pointing action by using DSP

This paper presents an approach of measuring in real-time the vector of finger that is pointing to an object. DSP is used in the operation processing unit in order to do the real-time processing. The steps include the extraction of flesh-colored regions from an image, the labeling of the flesh-colored regions, and the detection of two characteristic positions on the finger so that the direction that the finger is pointing at will be calculated. The entire process takes about 29 msec, which makes it possible to have the frame rate of 34 fps. With this frame rate, this measurement approach is considered real-time and promising to be merged into other application systems. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Automated online measurement of limestone particle size distributions using 3D range data

Fully automated online measurement of the size distribution of limestone particles on conveyor belt is presented based on 3D range data collected every minute during 13 h of production. The research establishes the necessary analysis and measurement capabilities to facilitate automatic control of rock crushing or particle agglomeration processes to improve both energy efficiency and product quality. 3D data from laser triangulation is used to provide high resolution data of the surface of the piled limestone particles. The 3D data is unaffected by color variation in the material and is not susceptible to scale or perspective distortion common in 2D imaging. Techniques are presented covering; sizing of particles, determination of non-overlapped and overlapped particles, and mapping of sizing results to distributions comparable to sieving. Detailed variations in the product sieve-size are shown with abrupt changes when the size range of the limestone particles is changed.     

Real-time robust vision-based hand gesture recognition using stereo images

This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.     

Gesture recognition using Bezier curves for visualization navigation from registered 3-D data

This paper presents a gesture recognition system for visualization navigation. Scientists are interested in developing interactive settings for exploring large data sets in an intuitive environment. The input consists of registered 3-D data. A geometric method using Bezier curves is used for the trajectory analysis and classification of gestures. The hand gesture speed is incorporated into the algorithm to enable correct recognition from trajectories with variations in hand speed. The method is robust and reliable: correct hand identification rate is 99.9% (from 1641 frames), modes of hand movements are correct 95.6% of the time, recognition rate (given the right mode) is 97.9%. An application to gesture-controlled visualization of 3D bioinformatics data is also presented.     

Pose estimation for objects with planar surfaces using eigenimage and range data analysis

In this paper we present a novel method for estimating the object pose for 3D objects with well-defined planar surfaces. Specifically, we investigate the feasibility of estimating the object pose using an approach that combines the standard eigenspace analysis technique with range data analysis. In this sense, eigenspace analysis was employed to constrain one object rotation and reject surfaces that are not compatible with a model object. The remaining two object rotations are estimated by computing the normal to the surface from the range data. The proposed pose estimation scheme has been successfully applied to scenes defined by polyhedral objects and experimental results are reported.     

基于MATLAB实时视窗目标的实时仿真系统设计

在控制系统的设计开发中,传统的开发过程已不能满足市场对产品的快速性和多样性要求,该文提供了一种实现控制系统快速原型开发和实时仿真的新方法。其基本思路是将实时仿真的方法应用于控制系统的开发过程,使控制系统设计、实现、测试和生产准备过程同时进行,以实现产品的快速原型开发。该文中详细介绍了基于MATLAB实时视窗目标的实时仿真系统的构建,并且进一步深入分析了该系统的体系结构和实时机制,最终结合实例给出了相应的实时仿真的开发流程和控制方式。     

Robust real-time 3D head pose estimation from range data

In this paper a real-time 3D pose estimation algorithm using range data is described. The system relies on a novel 3D sensor that generates a dense range image of the scene. By not relying on brightness information, the proposed system guarantees robustness under a variety of illumination conditions, and scene contents. Efficient face detection using global features and exploitation of prior knowledge along with novel feature localization and tracking techniques are described. Experimental results demonstrate accurate estimation of the six degrees of freedom of the head and robustness under occlusions, facial expressions, and head shape variability.     

EMG pattern classification using SOFMs for hand signal recognition

 We propose a method of pattern classification of electromyographic (EMG) signals using a set of self- organizing feature maps (SOFMs). The proposed method is simple to apply in that the EMG signals are directly input to the SOFMs without preprocessing. Experimental results are presented that show the effectiveness of the SOFM based classifier for the recognition of the hand signal version of the Korean alphabet from EMG signal patterns.     

Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

Multimedia Tools and Applications - A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve...