基于视觉的手势界面关键技术研究

针对视觉手势界面存在的问题,提出了一套行之有效的解决方案.首先,为了解决视觉手势交互中的MidasTouch问题,以人类注意的信息加工模型为理论依据提出了一个可扩展的视觉手势交互模型,该模型将手势交互过程分为选择性处理、分配性处理和集中处理3个不同阶段;然后,基于该模型提出了一个视觉手势识别框架,并结合认知心理学从手势检测、跟踪和识别3个方面对该框架的各个组成模块的关键技术进行了阐述,其中手势检测模块和识别管理模块能够辅助系统在复杂的背景中滤除掉不相关信息而选择性地搜索人手并根据上下文信息对手势识别任务重定向,从而避免了系统时刻都处于激活状态并对所有的手势动作都进行识别分析,有效解决了Midas Touch问题.文中介绍了使用该方法实现的IEToolkit手势界面工具平台,并基于一个视觉手势交互系统进行了实验测试与评估,结果验证了文中方法的可用性.     

面向智能电视的隐式手势交互建模与算法

针对基于视觉的智能电视手势交互中用户认知负荷和操作负荷较重的问题,提出融合显式交互和隐式交互的手势交互算法.通过对基于视觉的智能电视手势交互场景分析,首先建立了基于用户行为和智能电视状态的多层次上下文模型,实现上下文的数据融合与特征提取;其次设计并实现了CDL-DFCM推理模型和显隐信息融合的隐式交互模型,识别交互情景事件并感知用户意图;最后实现用户与智能电视的隐式交互.实验结果表明,与现有算法相比,该算法在操作准确率、时间开销和手势移动距离等方面得到了明显改善,并有效地提升了用户体验.     

基于多重触控的多通道视觉界面信息传达仿真

多通道视觉界面信息操作手势多样化,为满足不同用户需求,提出一种基于多重触控的多通道视觉界面信息传达方法.使用神经网络对原子手势建模,引入逻辑、时序和空间关系描述符分析组合手势架构,运用BP网络分类器检测原子手势,触发组合手势模型转移,实现多重触控手势识别;定义交互界面与程序主体功能不同路径,划分交互设备与信息处理过程,实现多通道信息整合与交互控制;最后将初始数据材料变换成信息模式,变量删除网络界面,把所有删除变量安置在删除树内组建连接树,将观测变量似然函数进行势更新,完成视觉界面信息传达全部过程.仿真结果表明,人机交互自然性有效提高,可为用户提供更加多元化网络服务.     

UIDT:一种基于摄像头的用户界面模型

为了弥补WIMP界面模型对视觉交互描述的不足,提出一种基于视觉交互的用户界面模型．首先在分析和总结视觉交互过程的基础上,以活动理论为基础,提出一种以用户为中心、面向任务和基于事件驱动的用户界面模型;然后描述了该模型的组成结构及其相互关系;最后给出了一个基于此模型的视频手势交互系统实例．实验结果表明,该模型能够有效地支持视觉交互设计,使用户界面满足可用性要求．     

基于自然手势跟踪的直接操作型三维人机交互范式

针对自然手势单通道条件下建立统一交互模型的难点问题和关键问题,该文提出一种基于二级行为模型的3D手势跟踪和交互方法,实现了一种基于自然手势的直接操作型3D人机交互界面范式原型系统.首先,建立了二级行为模型,然后,以行为模型为基础,设计并实现了一种基于行为模型的三维人机交互界面范式.文中主要创新点在于:建立了基本手势库的二级行为模型;用"令牌环"技术捕捉用户的交互意图;建立"多选一"的交互模型;用"替身"技术解决人手模型与不同物体的抓取过程中的多样性和复杂性问题,建立了抓取和释放操作的统一范式并提出了相关算法.文中算法在多个交互型虚拟装配平台上得到了验证.实验结果表明,与现有相关算法相比较,文中算法在时间开销和跟踪精度等方面得到了明显改善.     

基于傅立叶描述子和HMM的手势识别

针对家庭服务机器人平台中人机交互的问题,提出基于视觉的手势识别作为人与机器人交互的方式,研究利用傅立叶描述子对手势形状进行描述,并结合支持向量机和隐马尔可夫模型分别对静态手势和动态手势进行分类,实现了静态手势和动态手势的识别。该系统基于新型传感器Kinect,在图像分割阶段结合图像深度信息,可以有效的将手势区域提取出来,在一定范围内具有较强的鲁棒性,特征提取阶段基于傅立叶描述子,使手势识别具有旋转、缩放、平移不变性。针对七种常见静态手势和四种动态手势进行测试,平均识别率分别达到98.8%和96.7%,实验结果表明该系统具有较高的准确度。     

基于特征包支持向量机的手势识别

针对类肤色信息或复杂背景的影响,难以通过手势分割得到精确手势轮廓而对后期手势识别率与实时交互的影响,提出了一种基于特征包支持向量机(BOF-SVM)的手势识别方法。采用SIFT算法提取手势图像局部不变性特征点,将手势局部特征向量(尺度不变特征变换(SIFT)描述子)进行K-means聚类生成视觉码书,并通过视觉码书量化每一幅手势图像的视觉码字集合,以此获得手势图像的固定维数的表征向量来训练支持向量机(SVM)多类分类器。该方法只需框定手势所在区域,无需精确地分割人手。实验表明,该方法对9种交互手势的平均识别率达到92.1%,并具有很好的鲁棒性及实时性,能适应环境的变化。     

连续交互空间下的混合手势交互模型

分析了触控交互技术在移动手持设备及可穿戴设备应用的应用现状及存在的问题;基于交互动作的时间连续性及空间连续性,提出了将触控交互动作的接触面轨迹与空间轨迹相结合,同时具有空中手势及触控手势的特性及优点的混合手势输入方法;基于连续交互空间的概念,将混合交互手势,空中手势、表面触控手势进行统一,建立了包括空中层、表面层、混合层的连续交互空间分层处理模型;给出了统一的信息数据定义及数转换流程;构建了通用性的手势识别框架,并对轨迹切分方法及手势分类识别方法进行了阐述.最后设计了应用实例,通过实验,对混合交互手势的可用性及连续空间分层处理模型的可行性进行了验证.实验表明,混合手势输入方式同时兼具了表面触控输入及空中手势输入的优点,在兼顾识别效率的同时,具有较好的空间自由度.     

基于自适应遗传算法的手势识别

基于小样本库的手势识别是先进人机交互研究中的一个重要分支.根据Tortoise人手模型训练手势模式库并结合交互者的具体手部特征进行手形训练,生成适用于特定交互者的手势模式库.在交互过程中,根据来自一个或多个同步摄像头的视频信息进行基于自适应遗传算法的手势识别.实验结果表明,在环境光照基本稳定的条件下,文中算法可以实现鲁棒的实时手势识别.     

基于手势识别的人机交互技术

文章将现有人机交互方法与基于计算机视觉交互方法进行了对比,列举了该交互技术的优点及可行性,并提出了一种利用摄像头采集手势进行人机界面交互的方法,研究了并进一步实现了基于图像的手势分析、识别等关键技术.通过一系列实验结果表明,基于文中技术实现的一套系统能够实时地跟踪手的运动,并识别出手势结果,实现实时的人机手势交互.     

VICs: A modular HCI framework using spatiotemporal dynamics

Many vision-based human-computer interaction systems are based on the tracking of user actions. Examples include gaze tracking, head tracking, finger tracking, etc. In this paper, we present a framework that employs no user tracking; instead, all interface components continuously observe and react to changes within a local neighborhood. More specifically, components expect a predefined sequence of visual events called visual interface cues (VICs). VICs include color, texture, motion, and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-based HCI paradigm and then focus specifically on an appearance-based system in which a hidden Markov model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button push with a 96% success rate.Published online: 19 November 2004     

Optimizing Heuristic Evaluation Process in E-Commerce: Use of the Taguchi Method

This article proposes a 3-dimensional (3D) vision-based ambient user interface as an interaction metaphor that exploits a user's personal space and its dynamic gestures. In human-computer interaction, to provide natural interactions with a system, a user interface should not be a bulky or complicated device. In this regard, the proposed ambient user interface utilizes an invisible personal space to remove cumbersome devices where the invisible personal space is virtually augmented through exploiting 3D vision techniques. For natural interactions with the user's dynamic gestures, the user of interest is extracted from the image sequences by the proposed user segmentation method. This method can retrieve 3D information from the segmented user image through 3D vision techniques and a multiview camera. With the retrieved 3D information of the user, a set of 3D boxes (SpaceSensor) can be constructed and augmented around the user; then the user can interact with the system by touching the augmented SpaceSensor. In the user's dynamic gesture tracking, the computational complexity of SpaceSensor is relatively lower than that of conventional 2-dimensional vision-based gesture tracking techniques, because the touched positions of SpaceSensor are tracked. According to the experimental results, the proposed ambient user interface can be applied to various systems that require real-time user's dynamic gestures for their interactions both in real and virtual environments.     

基于视觉的手指空间位置检测技术

为了摆脱传统交互方式的局限性,指出了用人的指示行为作为人机交互输入方式所带来的直观性和方便性.在指示行为的视觉识别过程中,手指的空间位置检测是极为关键的.为了识别人的指示行为,重点从人手的视觉检测方法、手指和指尖的空间定位两个方面系统地阐述了手指空间位置检测技术,对各种方法的适用性及其优缺点进行了分析与归纳,并说明了基于视觉的手指空间位置检测的一般过程,探讨了指示行为在人机交互中的应用现状及前景.     

An efficient algorithm for 3D hand gesture recognition using combined neural classifiers

Gestures are the dynamic movements of hands within a certain time interval, which are of practical importance in many areas, such as human–computer interaction, computer vision, and computer graphics. The human hand gesture can provide a free and natural alternative to today’s cumbersome interface devices so as to improve the efficiency and effectiveness of human–computer interaction. This paper presents a neural-based combined classifier for 3D gesture recognition. The combined classifier is based on varying the parameters related to both the design and training of neural network classifier. The boosting algorithm is used to make perturbation of the training set employing the Multi-Layer Perceptron as base classifier. The final decision of the ensemble of classifiers is based on the majority voting rule. Experiments performed on 3D gesture database show the robustness of the proposed technique.     

复杂背景下基于傅立叶描述子的手势识别

人的手势是人们日常生活中最广泛使用的一种交流方式。由于在人机交互界面和虚拟现实环境中的应用,手势识别的研究受到了越来越广泛的关注。但是目前基于单目视觉的手势识别技术中,手势分割要求背景简单或者要求识别者戴着笨重的数据手套。而该文结合了运动信息和基于KL变换的肤色模型,在复杂背景下进行手势分割,与传统的基于RGB肤色模型的手势分割相比,在复杂背景环境下得到了很好的分割效果。在对分割的手势区域进行预处理后,该文使用了一种归一化的傅立叶描述子进行手势的特征提取,相比传统的傅立叶描述子更加准确,最后采用了传统的三层BP网络作为模式识别器,手势训练集和测试集的识别率分别达到了95．9％和95％。     

Target-shooting exergame with a hand gesture control

Exertion games (exergames) pose interesting challenges in terms of user interaction techniques. Players are commonly unable to use traditional input devices such as mouse and keyboard, given the body movement requirements of this type of videogames. In this work we propose a hand gesture interface to direct actions in a target-shooting exertion game that is played while exercising on an ergo-bike. A vision-based hand gesture interface for interacting with objects in a 3D videogame is designed and implemented. The system is capable to issue game commands to any computer game that normally responds to mouse and keyboard without modifying the underlying source code of the game. The vision system combines Bag-of-features and Support Vector Machine (SVM) to achieve user-independent and real-time hand gesture recognition. In particular, a Finite State Machine (FSM) is used to build the grammar that generates gesture commands for the game. We carried out a user study to gather feedback from participants, and our preliminary results show the high level of interest from users use this multimedia system that implements a natural way of interaction. Albeit some concerns in terms of comfort, users had a positive experience using our exertion game and they expressed their positive intention to use a system like this in their daily lives.     

GIS中线元的熵不确定带研究

空间数据的不确定性将直接影响地理信息产品的质量有GIS空间决策的可靠性，现已把它作为一个重要的基础理论问题加以研究，其中线元的位置不确定性是研究的一个热点，针对现有的线元位置不确定性模型的不足，通过引入信息熵理论，首先提出了二维随机点的熵误差椭圆指标与三维随机点的熵误差椭球指标；然后将它们扩展到线元的熵不确定带，实践证明，由于该模型能够根据联合熵唯一确定，且与置信水平的选取无关，因此比较适合作为线元位置不确定性度量的指标。     

基于Kinect深度信息的手势提取与识别研究

针对基于视觉的手势识别技术对环境背景要求较高的问题,提出了一种使用深度信息进行手势提取和识别的研究方案。采用了微软Kinect摄像头进行手势深度图的采集,再将深度图转换为三维点云,根据深度信息过滤来提取手势数据。对手势数据进行方向校正后统计手势数据中深度信息的区间分布特征并输入到支持向量机进行训练,从而实现了对数字手势1～5的手势识别。实验结果证明,该手势识别方案的平均识别率达到95%,使用设备简单且精度较高,鲁棒性较好。     

Prosody based audiovisual coanalysis for coverbal gesture recognition

Despite recent advances in vision-based gesture recognition, its applications remain largely limited to artificially defined and well-articulated gesture signs used for human-computer interaction. A key reason for this is the low recognition rates for "natural" gesticulation. Previous attempts of using speech cues to reduce error-proneness of visual classification have been mostly limited to keyword-gesture coanalysis. Such scheme inherits complexity and delays associated with natural language processing. This paper offers a novel "signal-level" perspective, where prosodic manifestations in speech and hand kinematics are considered as a basis for coanalyzing loosely coupled modalities. We present a computational framework for improving continuous gesture recognition based on two phenomena that capture voluntary (coarticulation) and involuntary (physiological) contributions of prosodic synchronization. Physiological constraints, manifested as signal interruptions during multimodal production, are exploited in an audiovisual feature integration framework using hidden Markov models. Coarticulation is analyzed using a Bayesian network of naive classifiers to explore alignment of intonationally prominent speech segments and hand kinematics. The efficacy of the proposed approach was demonstrated on a multimodal corpus created from the Weather Channel broadcast. Both schemas were found to contribute uniquely by reducing different error types, which subsequently improves the performance of continuous gesture recognition.