Model-based analysis of hand posture

Researchers have successfully recognized specific fingers of the hand by silhouette images and distinguished a small set of hand signs by contour features of images. However, the silhouette or contour features recovered from the images do not provide sufficient information to generate a 3D hand posture with the fingers positioned properly. This failure led to our study, which developed a new method employing a hand model that can automatically analyze 3D hand postures using static stereo images. Guided by internal constraints and external forces, the model is automatically fitted to the hand image. Our method differs from previous methods in that it captures 27 interacting hand parameters, including finger joint angles, enabling suitable reconstruction of 3D hand posture     

Three-dimensional finger joint angles by hand posture and object properties

The objective of this study was to identify three-dimensional finger joint angles for various hand postures and object properties. Finger joint angles were measured using a VICON system for 10 participants while they pinched objects with two, three, four and five fingers and grasped them with five fingers. The objects were cylinders and square pillars with diameters of 2, 4, 6 and 8 cm and weights of 400, 800, 1400 and 1800 g. Hand posture and object size more significantly affected the joint flexion angles than did object shape and weight. Object shape affected only the metacarpophalangeal (MCP) joint angle of the index finger and the flexion angle of the MCP joint of the little finger. Larger flexion angles resulted when the hand posture was grasping with five fingers. The joint angle increased linearly as the object size decreased. This report provides fundamental information about the specific joint angles of the thumb and fingers.

Practitioner Summary: Three-dimensional finger joint angles are of special interest in ergonomics because of their importance in handheld devices and musculoskeletal hand disorders. In this study, the finger joint angles corresponding to various hand postures and objects with different properties were determined.     

A multi-view vision-based hand motion capturing system

Vision-based hand motion capturing approaches play a critical role in human computer interface owing to its non-invasiveness, cost effectiveness, and user friendliness. This work presents a multi-view vision-based method to capture hand motion. A 3-D hand model with structural and kinematical constraints is developed to ensure that the proposed hand model behaves similar to an ordinary human hand. Human hand motion in a high degree of freedom space is estimated by developing a separable state based particle filtering (SSBPF) method to track the finger motion. By integrating different features, including silhouette, Chamfer distance, and depth map in different view angles, the proposed motion tracking system can capture the hand motion parameter effectively and solve the self-occlusion problem of the finger motion. Experimental results indicate that the hand joint angle estimation generates an average error of 11°.     

A finger motion model for reach and grasp

This study aimed to develop a model that describes human finger motion for simulation of reach and grasp for selected objects and tasks. Finger joint angles and timing of their changes were measured for six subjects as they reached 20-40 cm and grasped cylindrical handles (1.3-10.2 cm D) of varying orientation (vertical/axial). The empirical results from multiple regression analyses served as inputs to allow a fourth order polynomial to predict motion of each finger joint. The proposed model showed good fit with observations, with high coefficients of determination from 0.54 to 1 and reasonable errors from 0.04° to 5.44° for all conditions considered. The proposed finger motion model was implemented in an existing kinematic hand model to employ a contact algorithm for refined prediction of grip posture and to illustrate its predictive power by graphically displaying the opening and closing of the hand.

Relevance to industry

Finger joint motions during reach and grasp are needed for prediction of (1) tendon excursions for study of work-related musculoskeletal disorders, (2) required space for the hand, (3) finger locations on work objects, and (4) hand grip postures and strength.     

American Sign Language word recognition with a sensory glove using artificial neural networks

An American Sign Language (ASL) recognition system is being developed using artificial neural networks (ANNs) to translate ASL words into English. The system uses a sensory glove called the Cyberglove™ and a Flock of Birds^® 3-D motion tracker to extract the gesture features. The data regarding finger joint angles obtained from strain gauges in the sensory glove define the hand shape, while the data from the tracker describe the trajectory of hand movements. The data from these devices are processed by a velocity network with noise reduction and feature extraction and by a word recognition network. Some global and local features are extracted for each ASL word. A neural network is used as a classifier of this feature vector. Our goal is to continuously recognize ASL signs using these devices in real time. We trained and tested the ANN model for 50 ASL words with a different number of samples for every word. The test results show that our feature vector extraction method and neural networks can be used successfully for isolated word recognition. This system is flexible and open for future extension.     

Analyzing and capturing articulated hand motion in image sequences

Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets.     

Quick and accurate estimation of human 3D hand posture

We propose a three-dimensional hand posture estimation system that can retrieve a hand posture image most similar to the input data from a non-multilayer database. Our system uses, at the first stage, coarse screening by the proportional information on the hand images, which roughly correspond to forearm rotation or bending of the thumb or four fingers; then, at the second stage, performs a detailed search for similarity for selected candidates. To describe forearm rotation, and wrist’s internal and external rotations, bending and stretching, no separate processes were used for estimating the corresponding joint angles. By estimating the sequential images of the finger shape using this method, we successfully realized a process involving a joint angle estimation error within two or three degrees, a processing time of approximately 80 fps or more, using only one Note PC and high-speed camera, even when the wrist was freely rotating. Since the image information and the joint angle information are paired in the database, as well as the wrist joint, the system can generate the imitative motions as those of the fingers and wrist of a human being with no time delay by means of a robot, by outputting the estimation results to the robot hand.     

Wrist posture affects hand and forearm muscle stress during tapping

Non-neutral wrist posture is a risk factor of the musculoskeletal disorders among computer users. This study aimed to assess internal loads on hand and forearm musculature while tapping in different wrist postures. Ten healthy subjects tapped on a key switch using their index finger in four wrist postures: straight, ulnar deviated, flexed and extended. Torque at the finger and wrist joints were calculated from measured joint postures and fingertip force. Muscle stresses of the six finger muscles and four wrist muscles that balanced the calculated joint torques were estimated using a musculoskeletal model and optimization algorithm minimizing the squared sum of muscle stress. Non-neutral wrist postures resulted in greater muscle stresses than the neutral (straight) wrist posture, and the stress in the extensor muscles were greater than the flexors in all conditions. Wrist extensors stress remained higher than 4.5 N/cm² and wrist flexor stress remained below 0.5 N/cm² during tapping. The sustained high motor unit recruitment of extensors suggests a greater risk than other muscles especially in flexed wrist posture. This study demonstrated from the perspective of internal tissue loading the importance of maintaining neutral wrist posture during keying activities.     

Combined Direct and Inverse Kinematic Control for Articulated Figure Motion Editing

A new approach for the animation of articulated figures is presented. We propose a system of articulated motion design which offers a full combination of both direct and inverse kinematic control of the joint parameters. Such an approach allows an animator to specify interactively goal-directed changes to existing sampled joint motions, resulting in a more general and expressive class of possible joint motions. The fundamental idea is to consider any desired-joint space motion as a reference model inserted into the secondary task of an inverse kinematic control scheme. This approach profits from the use of half-space Cartesian main tasks in conjunction with a parallel control of the articulated figure called the coach-trainee metaphor. In addition, a transition function is introduced so as to guarantee the continuity of the control. The resulting combined kinematic control scheme leads to a new methodology of joint-motion editing which is demonstrated through the improvement of a functional model of human walking.     

Contact-free and pose-invariant hand-biometric-based personal identification system using RGB and depth data

Hand-biometric-based personal identification is considered to be an effective method for automatic recognition. However, existing systems require strict constraints during data acquisition, such as costly devices,specified postures, simple background, and stable illumination. In this paper, a contactless personal identification system is proposed based on matching hand geometry features and color features. An inexpensive Kinect sensor is used to acquire depth and color images of the hand. During image acquisition, no pegs or surfaces are used to constrain hand position or posture. We segment the hand from the background through depth images through a process which is insensitive to illumination and background. Then finger orientations and landmark points, like finger tips or finger valleys, are obtained by geodesic hand contour analysis. Geometric features are extracted from depth images and palmprint features from intensity images. In previous systems, hand features like finger length and width are normalized, which results in the loss of the original geometric features. In our system, we transform 2D image points into real world coordinates, so that the geometric features remain invariant to distance and perspective effects. Extensive experiments demonstrate that the proposed hand-biometric-based personal identification system is effective and robust in various practical situations.     

基于分类特征提取的手部动作识别方法的研究及应用

提出一种基于分类特征提取的手部动作识别方法,该方法通过自适应的混合高斯模型构建背景模型,使用背景减除法并充分利用人体手部肤色信息分割出人体手部区域,结合手部关节、骨骼特征及肤色信息估算手部关节点位置,构建三维手部骨架模型,然后提取手部各关节角度、位置信息并利用隐马尔柯夫(HMM)模型对其所表示的动作进行训练识别。     

Dynamic grasp planning of multifingered robot hands based onasymptotic stability

We describe an approach for planning grasps of multifingered robot hands based on a small vibration model. Using features of the grasp configuration, we analyze asymptotic stability, contact situations, and uniaxial fingertip force constraints for the combined planning of finger posture and finger position, and characterize the generalized mass, damping, and stiffness. Choosing the largest time constant of the vibration model as an optimization criterion for planning finger postures and positions, the original problem of dynamic grasp planning is formulated as a nonlinear program. Simulation examples for a three-fingered robot hand grasping a spherical object demonstrate the effectiveness of the approach.     

Design of optimum grip and control area for one-handed manual control devices

The objective of this paper was to make a design specification of the control area in dual tasks of the hand grip and manual control. The grip postures were analyzed for three types of hand tools. An experiment was performed to measure the position of the fingers and the maximum finger forces at 4 different postures for nine subjects. It was found out that the finger forces were significantly affected by the subjects, the fingers, and the grip postures. The maximum force of women was 62% of men's. From the experiment, the primary control area was defined as 10–13cm and the secondary control area as 8–12cm from the wrist origin. The preferred hand posture of the index and the middle fingers was found to be 3045 degrees at metacarpophalangeal joint and 4050 degrees at proximal interphalangeal joint. It was also found out that the design of one-handed manual control devices should include the characteristics of the user, grip posture, finger force, and the control arrangement.     

Development of statistical geometric models for prediction of a driver's hip and eye locations

The Society of Automotive Engineers (SAE) J1517 and J941 models of a driver-selected seat position and a driver's eye location mainly rely on their statistical linear relationships with seat configuration and package variables. Although the SAE models are useful for vehicle interior design, their prediction performance was not provided. The present study was intended to develop accurate prediction models of a driver's hip location (HL) and eye location (EL) based on their statistical geometric relationships with anthropometric dimensions and driving postures. A driving simulation experiment was conducted with 40 Korean drivers (20 males and 20 females) in a seating buck reconfigurable to various package conditions. The anthropometric measurements, HLs, ELs, and joint angles of the participants were collected using an anthropometer, a motion capture system, and a digital human model simulation program. Two types (full model and simplified model) of statistical geometric models (SGMs) for HL and EL prediction were developed by multiple regression analysis of the anthropometric measurements and driving postures on the HLs and ELs. The average adjusted R² and RMSE of the SGMs were .82 (± .06) and 25.7 (±3.3) mm, respectively. The SGMs showed accurate and stable prediction performance because the SGMs additionally incorporated the geometric relationships of HL and EL with anthropometric dimensions and joint angles. The SGMs would be useful to predict the HLs and ELs of drivers with various body sizes and joint angles in occupant packaging.     

The effect of viewing angle on wrist posture estimation from photographic images using novice raters

Observational assessment of wrist posture using photographic methods is theoretically affected by camera view angle. A study was conducted to investigate whether wrist flexion/extension and radial/ulnar deviation postures were estimated differently by raters depending on the viewing angle and compared to predictions using a quantitative 2D model of parallax. Novice raters (n = 26) estimated joint angles from images of wrist postures photographed from ten different viewing angles. Results indicated that ideal views, orthogonal to the plane of motion, produced more accurate estimates of posture compared to non-ideal views. The neutral (0°) posture was estimated the most accurately even at different viewing angles. Raters were more accurate than model predictions. Findings demonstrate a need for more systematic methods for collecting and analyzing photographic data for observational studies of posture. Renewed caution in interpreting existing studies of wrist posture where viewing angle was not controlled is advised.     

Classification of hand postures against complex backgrounds using elastic graph matching

A system for person-independent classification of hand postures against complex backgrounds in video images is presented. The system employs elastic graph matching, which has already been successfully applied for object and face recognition. We use the bunch graph technique to model variance in hand posture appearance between different subjects and variance in backgrounds. Our system does not need a separate segmentation stage but closely integrates finding the object boundaries with posture classification.