Visual rendering of shapes on 2D display devices guided by hand gestures

Designing of touchless user interface is gaining popularity in various contexts. Users can interact with electronic devices using such interfaces even when their hands are dirty or non-conductive. Also, users with partial physical disability can interact with electronic devices with the help of touchless interfaces. In this paper, we propose a Leap Motion controller-based methodology to facilitate rendering of 2D and 3D shapes on display devices. The proposed method tracks finger movements while users perform natural gestures within the field of view of the motion sensor. Then, trajectories are analyzed to extract extended Npen++ features in 3D. These features capture finger movements during the gestures and they are fed to unidirectional left-to-right Hidden Markov Model (HMM) for training. A one-to-one mapping between gestures and shapes, is proposed. Finally, the shapes corresponding to these gestures are rendered over the display using a typical MuPad supported interface. We have created a dataset of 5400 samples recorded by 10 volunteers. Our dataset contains 18 geometric and 18 non-geometric shapes such as “circle”, “rectangle”, “flower”, “cone”, “sphere”, etc. The proposed method has achieved 92.87% accuracy using a 5-fold cross validation scheme. Experiments reveal that the extended 3D features perform better than the existing 3D features when applied for shape representation and classification. The method can be used for developing diverse HCI applications suitable for smart display devices.     

Recognition of dynamic hand gestures

This paper is concerned with the problem of recognition of dynamic hand gestures. We have considered gestures which are sequences of distinct hand poses. In these gestures hand poses can undergo motion and discrete changes. However, continuous deformations of the hand shapes are not permitted. We have developed a recognition engine which can reliably recognize these gestures despite individual variations. The engine also has the ability to detect start and end of gesture sequences in an automated fashion. The recognition strategy uses a combination of static shape recognition (performed using contour discriminant analysis), Kalman filter based hand tracking and a HMM based temporal characterization scheme. The system is fairly robust to background clutter and uses skin color for static shape recognition and tracking. A real time implementation on standard hardware is developed. Experimental results establish the effectiveness of the approach.     

Evaluation of in-air hand gestures interaction for older people

Universal Access in the Information Society - In-air gestural interfaces are gaining popularity due to the increasing availability and low cost of gesture recognition hardware. However, the current...     

基于多权值神经网络的静态手势字母识别的研究

提出了一种基于多权值神经网络模型的静态手势语识别方法.应用手势字母图像圆周极径序列的傅立叶频谱信息来提取特征,再结合多权值神经网络的训练算法与识别算法,实现静态手势字母的识别,并取得了很好的识别效果.     

A taxonomy and notation method for three-dimensional hand gestures

Recently, studies on gesture-based interfaces have made an effort to improve the intuitiveness of gesture commands by asking users to define a gesture for a command. However, there are few methods to organize and notate user-defined gestures in a systematic approach. To resolve this, we propose a three-dimensional (3D) Hand Gesture Taxonomy and Notation Method. We first derived elements of a hand gesture by analyzing related studies and subsequently developed the 3D Hand Gesture Taxonomy based on the elements. Moreover, we devised a Notation Method based on a combination of the elements and also matched a code to each element for easy notation. Finally, we have verified the usefulness of the Notation Method by training participants to notate hand gestures and by asking another set of participants to recreate the notated gestures. In short, this research proposes a novel and systematic approach to notate hand gesture commands.     

Visual recognition of pointing gestures for human–robot interaction

In this paper, we present an approach for recognizing pointing gestures in the context of human–robot interaction. In order to obtain input features for gesture recognition, we perform visual tracking of head, hands and head orientation. Given the images provided by a calibrated stereo camera, color and disparity information are integrated into a multi-hypothesis tracking framework in order to find the 3D-positions of the respective body parts. Based on the hands’ motion, an HMM-based classifier is trained to detect pointing gestures. We show experimentally that the gesture recognition performance can be improved significantly by using information about head orientation as an additional feature. Our system aims at applications in the field of human–robot interaction, where it is important to do run-on recognition in real-time, to allow for robot egomotion and not to rely on manual initialization.     

A versatile interaction framework for robot programming based on hand gestures and poses

This paper proposes a framework for industrial and collaborative robot programming based on the integration of hand gestures and poses. The framework allows operators to control the robot via both End-Effector (EE) and joint movements and to transfer compound shapes accurately to the robot. Seventeen hand gestures, which cover the position and orientation controls of the robotic EE and other auxiliary operations, are designed according to cognitive psychology. Gestures are classified by a deep neural network, which is pre-trained for two-hand pose estimation and fine-tuned on a custom dataset, achieving a test accuracy of 99%. The index finger’s pointing direction and the hand’s orientation are extracted via 3D hand pose estimation to indicate the robotic EE’s moving direction and orientation, respectively. The number of stretched fingers is detected via two-hand pose estimation to represent decimal digits for selecting robot joints and inputting numbers. Finally, we integrate these three manners seamlessly to form a programming framework.We conducted two interaction experiments. The reaction time of the proposed hand gestures in indicating randomly given instructions is significantly less than that of other gesture sets, such as American Sign Language (ASL). The accuracy of our method in compound shape reconstruction is much better than that of hand movement trajectory-based methods, and the operating time is comparable with that of teach pendants.     

多媒体技术研究:2013——面向智能视频监控的视觉感知与处理

目的 随着视频监控技术的日益成熟和监控设备的普及,视频监控应用日益广泛,监控视频数据量呈现出爆炸性的增长,已经成为大数据时代的重要数据对象。然而由于视频数据本身的非结构化特性,使得监控视频数据的处理和分析相对困难。面对大量摄像头采集的监控视频大数据,如何有效地按照视频的内容和特性去传输、存储、分析和识别这些数据,已经成为一种迫切的需求。方法 本文面向智能视频监控中大规模视觉感知与智能处理问题,围绕监控视频编码、目标检测与跟踪、监控视频增强、视频运动与异常行为识别等4个主要研究方向,系统阐述2013年度的技术发展状况,并对未来的发展趋势进行展望。结果 中国最新制定的国家标准AVS2在对监控视频的编码效率上比最新国际标准H.265/HEVC高出一倍,标志着我国的视频编码技术和标准在视频监控领域已经实现跨越;视频运动目标检测跟踪的研究主要集中在有效特征提取和分类器训练等方面,机器学习等方法的引入,使得基于多实例学习、稀疏表示的运动目标检测跟踪成为研究的热点;监控视频质量增强主要包括去雾、去夜色、去雨雪、去模糊和超分辨率增强等多方面的内容,现有的算法均是对某类图像清晰化效果较好,而对其他类则相对较差,普适性不高;现有的智能动作分析与异常行为识别技术虽然得到了不断发展,算法的性能也在不断提高,但是从实用角度,除了简单的特定或可控场景外,还没有太多成熟的应用系统。结论 随着大数据时代的到来,智能视频监控的需求将日益迫切,面对众多挑战的同时,该研究领域将迎来前所未有的重大机遇,必将产生越来越多可以实用的研究成果。     

Visual cryptographic techniques for secret image sharing: a review

ABSTRACT

A visual cryptography scheme (VCS) allows one to decrypt images without any knowledge of cryptography and computational efforts. VCS allows secret image sharing such that we can divide the original image into meaningful or nonmeaningful shares. The shares are distributed among participants; during decryption, the original secret image is recovered through stacking all or some of the shares by the human visual system. Various techniques of visual cryptography were constructed mainly for binary images but later, they were enhanced to handle gray-scale or color images. This article focuses on the study of various visual cryptographic schemes and analyses the performance on the basis of various parameters such as, pixel expansion, type of shares generated, image format, number of secret images, encryption method, etc.. In the proposed work, we give a precise and complete review of various visual cryptographic schemes based on different research works related to this area and cite the relevant literature.     

Recognition of hand gestures with 3D,nonlinear arm movement

Hand gesture is a useful modality of human interaction. In this paper, we propose an approach to recognition of space-time variable patterns of nonlinear arm movement and integration with other attributes to find the proper interpretation. At the encoding stage, we first extract the essential 2D trajectory from 3D arm movement by a plane fitting method. Pause information between the consecutive gestures is also modeled and integrated into the encoding. Codified information is then applied is a hidden Markov model (HMM) network which is responsible for segmentation and recognition of continuous arm movements. As a whole, three major attributes of hand gestures are processed in parallel and independently, followed by the inter-attribute communication for finding the proper interpretation.     

Visual interpretation of regression error

Several sophisticated machine learning tools (e.g., ensembles or deep networks) have shown outstanding performance in different regression forecasting tasks. In many real world application domains the numeric predictions of the models drive important and costly decisions. Nevertheless, decision makers frequently require more than a black box model to be able to “trust” the predictions up to the point that they base their decisions on them. In this context, understanding these black boxes has become one of the hot topics in Machine Learning research. This paper proposes a series of visualization tools that explain the relationship between the expected predictive performance of black box regression models and the values of the input variables of any given test case. This type of information thus allows end-users to correctly assess the risks associated with the use of a model, by showing how concrete values of the predictors may affect the performance of the model. Our illustrations with different real world data sets and learning algorithms provide insights on the type of usage and information these tools bring to both the data analyst and the end-user. Furthermore, a thorough evaluation of the proposed tools is performed to showcase the reliability of this approach.     

Tracking,recognition, and distance detection of hand gestures for a 3‐D interactive display

Abstract— This study proposes an interactive system for displays, the technologies of which consists of three main parts: hand‐gesture tracking, recognition, and depth measurement. The proposed interactive system can be applied to a general 3‐D display. In this interactive system, for hand‐gesture tracking, Haar‐like features are employed to detect a specific hand gesture to start tracking, while the mean‐shift algorithm and Kalman filter are adopted for fast tracking. First, for recognizing hand gestures, a principal component analysis (PCA) algorithm is used to localize colored areas of skin, and then hand gestures are identified by comparison with a prepared database. Second, a simple optical system is set up with an infrared laser source and a grid mask in order to project a proposed horizontal stripe pattern. Third, the projected patterns are deciphered to extract the depth information using the Hough‐transform algorithm. The system containing hand‐gesture localization, recognition, and associated depth detection (the distance between the display and the hand), was included in a prototype of an interactive display. Demonstration of rotation recognition of a finger‐pointing hand gesture was successful by using the algorithm of radar‐like scanning.     

Real-time estimation of hand gestures based on manifold learning from monocular videos

Object pose estimation by manifold learning has become a hot research area recently. In this paper, we propose an efficient method that can recover pose and viewpoints for numerous hand gestures from monocular videos based on Locality Preserving Projections. We first select some hand dynamic gestures as primitive hand motions and set a 3D-2D mapping table to relate 3D joint angles of sampling static pose with their projective silhouettes from arbitrary viewpoints. Then the embedding space and explicit mapping function are learnt for every primitive motion. In order to make classification and prediction among those embedding spaces, a Subspace Filtering Algorithm is also proposed which can recognize and recover numerous hand dynamic gestures by the combination of primitive gestures. At last, by using skin color cues and oriented k-Dops, multi-hands can be labeled and tracked separately and accurately. Extensive experimental results demonstrate qualitatively and quantitatively that 3D pose recovery of hands can be achieved by our method robustly and efficiently.     

Rapid and efficient hand gestures recognizer based on classes discriminator wavelet networks

Multimedia Tools and Applications - The vision based on hand gesture recognition is one of the key challenges in behavior understanding and computer vision. It offers to machines the possibility of...     

Visual recognition of continuous hand postures

This paper describes GREFIT (Gesture REcognition based on FInger Tips), a neural network-based system which recognizes continuous hand postures from gray-level video images (posture capturing). Our approach yields a full identification of all finger joint angles (making, however, some assumptions about joint couplings to simplify computations). This allows a full reconstruction of the three-dimensional (3-D) hand shape, using an articulated hand model with 16 segments and 20 joint angles. GREFIT uses a two-stage approach to solve this task. In the first stage, a hierarchical system of artificial neural networks (ANNs) combined with a priori knowledge locates the two-dimensional (2-D) positions of the finger tips in the image. In the second stage, the 2-D position information is transformed by an ANN into an estimate of the 3-D configuration of an articulated hand model, which is also used for visualization. This model is designed according to the dimensions and movement possibilities of a natural human hand. The virtual hand imitates the user's hand to an remarkable accuracy and can follow postures from gray scale images at a frame rate of 10 Hz.     

Real-time automated detection of older adults' hand gestures in home and clinical settings

Neural Computing and Applications - There is an urgent need, accelerated by the COVID-19 pandemic, for methods that allow clinicians and neuroscientists to remotely evaluate hand movements. This...     

Gaussian Process Dynamical Models for hand gesture interpretation in Sign Language

Classifying human hand gestures in the context of a Sign Language has been historically dominated by Artificial Neural Networks and Hidden Markov Model with varying degrees of success. The main objective of this paper is to introduce Gaussian Process Dynamical Model as an alternative machine learning method for hand gesture interpretation in Sign Language. In support of this proposition, the paper presents the experimental results for Gaussian Process Dynamical Model against a database of 66 hand gestures from the Malaysian Sign Language. Furthermore, the Gaussian Process Dynamical Model is tested against established Hidden Markov Model for a comparative evaluation. A discussion on why Gaussian Process Dynamical Model is superior over existing methods in Sign Language interpretation task is then presented.     

Visual lane analysis and higher-order tasks: a concise review

Lane detection, lane tracking, or lane departure warning have been the earliest components of vision-based driver assistance systems. At first (in the 1990s), they have been designed and implemented for situations defined by good viewing conditions and clear lane markings on highways. Since then, accuracy for particular situations (also for challenging conditions), robustness for a wide range of scenarios, time efficiency, and integration into higher-order tasks define visual lane detection and tracking as a continuing research subject. The paper reviews past and current work in computer vision that aims at real-time lane or road understanding under a comprehensive analysis perspective, for moving on to higher-order tasks combined with various lane analysis components, and introduces related work along four independent axes as shown in Fig. 2. This concise review provides not only summarizing definitions and statements for understanding key ideas in related work, it also presents selected details of potentially applicable methods, and shows applications for illustrating progress. This review helps to plan future research which can benefit from given progress in visual lane analysis. It supports the understanding of newly emerging subjects which combine lane analysis with more complex road or traffic understanding issues. The review should help readers in selecting suitable methods for their own targeted scenario.