Dynamic disparity adjustment and histogram-based filtering of range data for fast 3-D hand tracking

Range data is very important in human-computer interaction applications. Although less costly, range acquisition and processing still presents a speed vs data reliability tradeoff. This paper proposes a method that, given noisy and generally unreliable range data, can filter out erroneous information using range histograms for regions of interest selected in registered color data. Then, using the resulting consistent data that has passed filters, this method limits the depth search space dynamically using motion history and its current state. Experimental results demonstrate the success of the proposed algorithm. Using filtered range data, the algorithm correctly identified the hand involved in manipulation 99.85% of the time. Dynamic disparity adjustment produced a speedup of 60.17% over a static disparity range selection. An application to virtual reality navigation is also presented.     

Real-time fingertip localization conditioned on hand gesture classification

A method to obtain accurate hand gesture classification and fingertip localization from depth images is proposed. The Oriented Radial Distribution feature is utilized, exploiting its ability to globally describe hand poses, but also to locally detect fingertip positions. Hence, hand gesture and fingertip locations are characterized with a single feature calculation. We propose to divide the difficult problem of locating fingertips into two more tractable problems, by taking advantage of hand gesture as an auxiliary variable. Along with the method we present the ColorTip dataset, a dataset for hand gesture recognition and fingertip classification using depth data. ColorTip contains sequences where actors wear a glove with colored fingertips, allowing automatic annotation. The proposed method is evaluated against recent works in several datasets, achieving promising results in both gesture classification and fingertip localization.     

Real-time hand tracking using a mean shift embedded particle filter

Particle filtering and mean shift (MS) are two successful approaches to visual tracking. Both have their respective strengths and weaknesses. In this paper, we propose to integrate advantages of the two approaches for improved tracking. By incorporating the MS optimization into particle filtering to move particles to local peaks in the likelihood, the proposed mean shift embedded particle filter (MSEPF) improves the sampling efficiency considerably. Our work is conducted in the context of developing a hand control interface for a robotic wheelchair. We realize real-time hand tracking in dynamic environments of the wheelchair using MSEPF. Extensive experimental results demonstrate that MSEPF outperforms the MS tracker and the conventional particle filter in hand tracking. Our approach produces reliable tracking while effectively handling rapid motion and distraction with roughly 85% fewer particles. We also present a simple method for dynamic gesture recognition. The hand control interface based on the proposed algorithms works well in dynamic environments of the wheelchair.     

An efficient posture recognition method using fuzzy logic

Computer-human interaction plays an important role in virtual reality. Glove-based input devices have many desirable features which make direct interactions between the user and the virtual world possible. However, due to the complexity of the human hand, recognising hand functions precisely and efficiently is not an easy task. Existing algorithms are either imprecise or computationally expensive, making them impractical to integrate with VR applications, which are usually very CPU intensive.In the problem of posture and gesture recognition, both the sample patterns stored in the database and the ones to be recognised may be imprecise. This kind of imprecise knowledge can be best dealt with using fuzzy logic. A fast and simple posture recognition method using fuzzy logic is presented in this paper. Our model consists of three components: the posture database, the classifier and the identifier. The classifier roughly classifies the sample postures before they are put into the posture database. The identifier compares an input posture with the records in the identified class and finds the right match efficiently. Fuzzy logic is applied in both the classification and identification processes to cope with imprecise data. The main goal of this method is to recognise hand functions in an accurate and efficient manner. The accuracy, efficiency and the noise tolerance of the model have been examined through a number of experiments.     

Classification of hand postures against complex backgrounds using elastic graph matching

A system for person-independent classification of hand postures against complex backgrounds in video images is presented. The system employs elastic graph matching, which has already been successfully applied for object and face recognition. We use the bunch graph technique to model variance in hand posture appearance between different subjects and variance in backgrounds. Our system does not need a separate segmentation stage but closely integrates finding the object boundaries with posture classification.     

Recognition of dynamic hand gestures

This paper is concerned with the problem of recognition of dynamic hand gestures. We have considered gestures which are sequences of distinct hand poses. In these gestures hand poses can undergo motion and discrete changes. However, continuous deformations of the hand shapes are not permitted. We have developed a recognition engine which can reliably recognize these gestures despite individual variations. The engine also has the ability to detect start and end of gesture sequences in an automated fashion. The recognition strategy uses a combination of static shape recognition (performed using contour discriminant analysis), Kalman filter based hand tracking and a HMM based temporal characterization scheme. The system is fairly robust to background clutter and uses skin color for static shape recognition and tracking. A real time implementation on standard hardware is developed. Experimental results establish the effectiveness of the approach.     

Robust face recognition using 2D and 3D data: Pose and illumination compensation

The paper addresses the problem of face recognition under varying pose and illumination. Robustness to appearance variations is achieved not only by using a combination of a 2D color and a 3D image of the face, but mainly by using face geometry information to cope with pose and illumination variations that inhibit the performance of 2D face recognition. A face normalization approach is proposed, which unlike state-of-the-art techniques is computationally efficient and does not require an extended training set. Experimental results on a large data set show that template-based face recognition performance is significantly benefited from the application of the proposed normalization algorithms prior to classification.     

An interactive image clipping system using hand motion recognition

We present an efficient hand recognition algorithm for an interactive image clipping system, which is widely used for environments such as public facilities and security environments where personal capturing devices including mobile phones are not allowed. User-friendly interface and accurate image capturing function are required for an image clipping system. We build the system by combining Microsoft Kinect, HD webcam and projector. The Kinect and webcam are used to capture the motions of users׳ hand and project is to display the user-selected area from the capturing material. Hand recognition is composed of three steps: (i) the region occupied by users׳ hand is extracted from an image, (ii) the fingertips of the extracted hand region are analyzed using k-curvature algorithm, and (iii) the height of the fingertip is estimated using the depth image from Kinect. The height of the fingertip informs whether users׳ finger touched the surface of the target. The region captured by the fingertip is clipped from the image and stored as the target image. The excellence of our hand recognition algorithm is proved through a user test.     

基于BOF-Gist特征的手势识别算法研究

针对静态手势识别算法存在特征计算复杂度高,实时性差的问题,提出了一种新的BOF-Gist特征对手势图像进行表示。该特征的优势是在保持Gist特征原有优势的基础上,有效地表征手势图像的局部特征和全局特征,并且特征维数明显降低,实时性好。在标准数据库上的测试表明,该算法对于简单、复杂背景下十种手语手势分别得到了90.42%与79.05%的正确识别率,同时验证了算法的实时性。     

Automated online measurement of limestone particle size distributions using 3D range data

Fully automated online measurement of the size distribution of limestone particles on conveyor belt is presented based on 3D range data collected every minute during 13 h of production. The research establishes the necessary analysis and measurement capabilities to facilitate automatic control of rock crushing or particle agglomeration processes to improve both energy efficiency and product quality. 3D data from laser triangulation is used to provide high resolution data of the surface of the piled limestone particles. The 3D data is unaffected by color variation in the material and is not susceptible to scale or perspective distortion common in 2D imaging. Techniques are presented covering; sizing of particles, determination of non-overlapped and overlapped particles, and mapping of sizing results to distributions comparable to sieving. Detailed variations in the product sieve-size are shown with abrupt changes when the size range of the limestone particles is changed.     

Real-time measurement of pointing action by using DSP

This paper presents an approach of measuring in real-time the vector of finger that is pointing to an object. DSP is used in the operation processing unit in order to do the real-time processing. The steps include the extraction of flesh-colored regions from an image, the labeling of the flesh-colored regions, and the detection of two characteristic positions on the finger so that the direction that the finger is pointing at will be calculated. The entire process takes about 29 msec, which makes it possible to have the frame rate of 34 fps. With this frame rate, this measurement approach is considered real-time and promising to be merged into other application systems. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Real-time robust vision-based hand gesture recognition using stereo images

This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.     

Gesture recognition using Bezier curves for visualization navigation from registered 3-D data

This paper presents a gesture recognition system for visualization navigation. Scientists are interested in developing interactive settings for exploring large data sets in an intuitive environment. The input consists of registered 3-D data. A geometric method using Bezier curves is used for the trajectory analysis and classification of gestures. The hand gesture speed is incorporated into the algorithm to enable correct recognition from trajectories with variations in hand speed. The method is robust and reliable: correct hand identification rate is 99.9% (from 1641 frames), modes of hand movements are correct 95.6% of the time, recognition rate (given the right mode) is 97.9%. An application to gesture-controlled visualization of 3D bioinformatics data is also presented.     

Pose estimation for objects with planar surfaces using eigenimage and range data analysis

In this paper we present a novel method for estimating the object pose for 3D objects with well-defined planar surfaces. Specifically, we investigate the feasibility of estimating the object pose using an approach that combines the standard eigenspace analysis technique with range data analysis. In this sense, eigenspace analysis was employed to constrain one object rotation and reject surfaces that are not compatible with a model object. The remaining two object rotations are estimated by computing the normal to the surface from the range data. The proposed pose estimation scheme has been successfully applied to scenes defined by polyhedral objects and experimental results are reported.     

Robust real-time 3D head pose estimation from range data

In this paper a real-time 3D pose estimation algorithm using range data is described. The system relies on a novel 3D sensor that generates a dense range image of the scene. By not relying on brightness information, the proposed system guarantees robustness under a variety of illumination conditions, and scene contents. Efficient face detection using global features and exploitation of prior knowledge along with novel feature localization and tracking techniques are described. Experimental results demonstrate accurate estimation of the six degrees of freedom of the head and robustness under occlusions, facial expressions, and head shape variability.     

EMG pattern classification using SOFMs for hand signal recognition

 We propose a method of pattern classification of electromyographic (EMG) signals using a set of self- organizing feature maps (SOFMs). The proposed method is simple to apply in that the EMG signals are directly input to the SOFMs without preprocessing. Experimental results are presented that show the effectiveness of the SOFM based classifier for the recognition of the hand signal version of the Korean alphabet from EMG signal patterns.     

Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

Multimedia Tools and Applications - A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve...     

Web data retrieval: solving spatial range queries using k-nearest neighbor searches

As Geographic Information Systems (GIS) technologies have evolved, more and more GIS applications and geospatial data are available on the web. Spatial objects in a given query range can be retrieved using spatial range query − one of the most widely used query types in GIS and spatial databases. However, it can be challenging to retrieve these data from various web applications where access to the data is only possible through restrictive web interfaces that support certain types of queries. A typical scenario is the existence of numerous business web sites that provide their branch locations through a limited “nearest location” web interface. For example, a chain restaurant’s web site such as McDonalds can be queried to find some of the closest locations of its branches to the user’s home address. However, even though the site has the location data of all restaurants in, for example, the state of California, it is difficult to retrieve the entire data set efficiently due to its restrictive web interface. Considering that k-Nearest Neighbor (k-NN) search is one of the most popular web interfaces in accessing spatial data on the web, this paper investigates the problem of retrieving geospatial data from the web for a given spatial range query using only k-NN searches. Based on the classification of k-NN interfaces on the web, we propose a set of range query algorithms to completely cover the rectangular shape of the query range (completeness) while minimizing the number of k-NN searches as possible (efficiency). We evaluated the efficiency of the proposed algorithms through statistical analysis and empirical experiments using both synthetic and real data sets.

Face recognition based on 3D ridge images obtained from range data

In this paper, we present an approach for 3D face recognition from frontal range data based on the ridge lines on the surface of the face. We use the principal curvature, k_max, to represent the face image as a 3D binary image called ridge image. The ridge image shows the locations of the ridge points around the important facial regions on the face (i.e., the eyes, the nose, and the mouth). We utilized the robust Hausdorff distance and the iterative closest points (ICP) for matching the ridge image of a given probe image to the ridge images of the facial images in the gallery. To evaluate the performance of our approach for 3D face recognition, we performed experiments on GavabDB face database (a small size database) and Face Recognition Grand Challenge V2.0 (a large size database). The results of the experiments show that the ridge lines have great capability for 3D face recognition. In addition, we found that as long as the size of the database is small, the performance of the ICP-based matching and the robust Hausdorff matching are comparable. But, when the size of the database increases, ICP-based matching outperforms the robust Hausdorff matching technique.