基于OpenGL的三维虚拟人手语动画的研究与实现

研究了一种基于OpenGL实现三维虚拟人手语动画的方法.首先构造了三维虚拟人的最小骨骼模型;其次研究分析了手部的关键动作,提出了手部动厕的关键帧手势;又考虑到手部的运动干涉和运动约束,给出了手语动画中的不可能手势,接着基于关键帧手势以及不可能的手势,利用手部各关节之间的联动关系,给出了手部各节点位置的计算方法;最后基于OpenGL实现了三维虚拟人的手语动画.实验结果表明,基于关键帧手势以及不可能的手势的三维虚拟人的手语动画方法不仅解决了真实人手部动作的复杂性问题,又没有裁剪必要的手部动作,获得了理想的手语动画效果.     

基于VRML虚拟人的维文手语库的构建

维吾尔语的手语合成有助于改善维吾尔族聋哑人与听力正常人进行自然交流,也可以应用于计算机辅助维吾尔哑语教学、维文电视节目播放等方面。维文手语库是维吾尔语手语合成的基础。通过分析维吾尔手语的特点,采用关键帧插值技术来控制VRML虚拟人的手势动作,利用Visual C++和OpenGL环境实现了一个维吾尔文的手势编辑系统,通过手势运动数据驱动虚拟人来实时显示当前的手势状态。通过该系统,收集了常用的维吾尔语词汇及32个维吾尔字母的手势运动数据。     

基于骨骼蒙皮动画的虚拟人手势合成技术的研究

虚拟人手势合成技术在聋哑人的很多领域都有广泛应用。本文研究了虚拟人运动控制的方法以及骨骼蒙皮动画的原理。利用3dsmax、poser等建模软件对虚拟人模型进行创建,在XNA游戏引擎中,实现了对虚拟人手势等的控制。该种运动控制方式不受动作片段的影响,可以广泛应用于虚拟人运动控制中。     

手势数据驱动的头部运动合成方法

非手部手势是手语表达中不可缺少的一部分,头部运动的实现并与手势进行协同表达是其重要研究内容。对真人手语表演数据中的手势与头部动作之间的关系进行了深入研究,提取二者的动作特征,利用核典型相关分析方法(kernel canonical correlation analysis,KCCA)建立起手势与头部动作之间的预测关系模型。动画合成结果以及评价实验表明,KCCA方法能更好地刻画手势与头部动作的协调性,实现虚拟人行为动作合成的逼真性。     

基于VGG-Nets算法手势识别设计与实现

为提高正常人与聋哑人之间的交流效率,基于TensorFlow深度学习框架,采用Python、OpenCV视觉库、VGG-Nets网络模型算法,将手语的手势动作转换为文本、语音等形式,构造了正常人与聋哑人之间的无障碍交流平台。在功能上主要完成对静态手语的识别,并以此为基础实现手势动作信息到文本、语音等信息的转化。本系统可以使用摄像头主动采集手语信息,也可利用前端上传图片至后台;结合手语识别的固定流程识别手语,并在前端界面反馈出识别结果。本系统旨在通过机器学习算法实现手语动作识别,并将手语翻译成文字、语音等形式,可有效提升交流效率。     

面向广电节目的虚拟人手语合成显示平台研究

基于中国手语合成技术的虚拟人手语视频显示平台技术是一个全新的课题.为了满足广电新闻节目对手势运动流畅性要求,实现了一种上下文相关的手势运动平滑算法,该方法能够充分利用前后两帧的差异来实现手势运动的平滑过渡,其视觉感观效果较传统插值算法更加平滑自然;同时提出了一种基于统计和规则相结合的手势运动重定向算法,在统计方法的基础上针对不同骨架大小以及运动特性进行规则约束,使得标准模型手势运动数据应用到新模型上而不失其准确性;最后,通过扩展基本手语词表达形式并基于alpha融合技术实现了面向广电新闻节目的虚拟人手语合成显示平台并取得很好的结果.     

基于Leap Motion的手语识别算法优化

Leap Motion设备产生的数据在虚拟环境中可以进行手势识别。通过识别和跟踪用户的手来生成虚拟3D手部模型,从而获取手势信息。本文设计了一种基于隐马尔可夫模型(Hidden Markov Model, HMM)分类算法来学习从Leap Motion中所获取的手势信息的系统,根据手势特征的重要性赋予不同权值,可进一步提高分类准确率,实现手语信息的识别输入。测试结果表明,识别准确率为86.1%,手语打字输入识别速度为每分钟13.09个字符,可显著提高聋哑人与正常人之间沟通的便捷性。     

基于视觉的手势识别技术

近年来计算机已经成为人们日常生活的一部分,人们与计算机的交互也日益成为科研领域的热点。基于视觉的手势识别是实现新一代人机交互所不可缺少的一项关键技术,而手势识别的研究也可促进手语识别的发展,从而消除健全人与聋哑人之间的交流障碍,使他们能获得健全人的正常生活,帮忙他们参加社会的各项活动。文中介绍了手势识别方法的发展、手势识别的技术难点,具体阐述了基于视觉的手势识别系统原理和组成,手势的建模以及在手势识别中常用的技术方法。     

基于IMU的手语识别骨骼模型构建

手语识别属于手势识别的研究范畴。传统的基于数据手套的手语识别方法不能完整捕捉手语的所有要素,无法识别手部与肢体配合的手语动作。 惯性测量单元(IMU)由于体积小、成本低而被越来越多地应用到动作捕捉项目中。借鉴机器人运动学相关知识,提出了基于IMU的手语识别骨骼模型,该模型符合人体生物学特征。模型的构建步骤为首先进行骨骼的选取,然后进行尺寸标定。最后提出了标定模型尺寸的实验方法,使用IMU获得的动作集的数据可以进行求解。     

基于视觉的手势识别技术

近年来计算机已经成为人们日常生活的一部分,人们与计算机的交互也日益成为科研领域的热点.基于视觉的手势识别是实现新一代人机交互所不可缺少的一项关键技术,而手势识别的研究也可促进手语识别的发展,从而消除健全人与聋哑人之间的交流障碍,使他们能获得健全人的正常生活,帮忙他们参加社会的各项活动.文中介绍了手势识别方法的发展、手势识别的技术难点,具体阐述了基于视觉的手势识别系统原理和组成,手势的建模以及在手势识别中常用的技术方法.     

A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery

The advent of depth cameras has enabled mid-air interactions for shape modeling with bare hands. Typically, these interactions employ a finite set of pre-defined hand gestures to allow users to specify modeling operations in virtual space. However, human interactions in real world shaping processes (such as pottery or sculpting) are complex, iterative, and continuous. In this paper, we show that the expression of user intent in shaping processes can be derived from the geometry of contact between the hand and the manipulated object. Specifically, we describe the design and evaluation of a geometric interaction technique for bare-hand mid-air virtual pottery. We model the shaping of a pot as a gradual and progressive convergence of the pot’s profile to the shape of the user’s hand represented as a point-cloud (PCL). Thus, a user does not need to learn, know, or remember any gestures to interact with our system. Our choice of pottery simplifies the geometric representation, allowing us to systematically study how users use their hands and fingers to express the intent of deformation during a shaping process. Our evaluations demonstrate that it is possible to enable users to express their intent for shape deformation without the need for a fixed set of gestures for clutching and deforming a shape.     

The design of hand gestures for human–computer interaction: Lessons from sign language interpreters

The design and selection of 3D modeled hand gestures for human–computer interaction should follow principles of natural language combined with the need to optimize gesture contrast and recognition. The selection should also consider the discomfort and fatigue associated with distinct hand postures and motions, especially for common commands. Sign language interpreters have extensive and unique experience forming hand gestures and many suffer from hand pain while gesturing. Professional sign language interpreters (N=24) rated discomfort for hand gestures associated with 47 characters and words and 33 hand postures. Clear associations of discomfort with hand postures were identified. In a nominal logistic regression model, high discomfort was associated with gestures requiring a flexed wrist, discordant adjacent fingers, or extended fingers. These and other findings should be considered in the design of hand gestures to optimize the relationship between human cognitive and physical processes and computer gesture recognition systems for human–computer input.     

Hand Motion Gesture Frequency Properties and Multimodal Discourse Analysis

Gesture and speech are co-expressive and complementary channels of a single human language system. While speech carries the major load of symbolic presentation, gesture provides the imagistic content. We investigate the role of oscillatory/cyclical hand motions in ‘carrying’ this image content. We present our work on the extraction of hand motion oscillation frequencies of gestures that accompany speech. The key challenges are that such motions are characterized by non-stationary oscillations, and multiple frequencies may be simultaneously extant. Also, the duration of the oscillations may be extended over very few cycles. We apply the windowed Fourier transform and wavelet transform to detect and extract gesticulatory oscillations. We tested these against synthetic signals (stationary and non-stationary) and real data sequences of gesticulatory hand movements in natural discourse. Our results show that both filters functioned well for the synthetic signals. For the real data, the wavelet bandpass filter bank is better for detecting and extracting hand gesture oscillations. We relate the hand motion oscillatory gestures detected by wavelet analysis to speech in natural conversation and apply to multimodal language analysis. We demonstrate the ability of our algorithm to extract gesticulatory oscillations and show how oscillatory gestures reveal portions of the multimodal discourse structure.     

Modeling human hand movements,facial expressions,and articulation to synthesize and visualize gesture information

Information and mathematical models are proposed for the animation of sign language communication based on a virtual human model. A model is developed to fix sign language morphemes and is used to create a technology and software to generate, store, and reproduce gestures. Algorithmic solutions are proposed for the computation of human-like trajectories of hand and body movements in passing from a gesture to another and also facial expressions and articulation.     

基于虚拟人合成技术的中国手语合成方法

介绍了一种中国手语合成方法,实现了文本到中国手语的自动翻译,并使用虚拟人合成技术,实现了中国手语的合成与显示,以此帮助聋人与听力正常人之间实现自然交流.在该方法中,首先应用两只数据手套和3个6自由度位置跟踪器,基于运动跟踪的原理,记录真实人体演示每个手语词的运动数据,建立一个初始的手语词运动数据库.然后,应用一种基于控制点的人体运动编辑方法,对每个手语词的运动数据进行编辑与微调,最后得到一个高质量的手语词运动数据库.当给定一个文本句子时,应用人体运动合成方法,对每个手语词的手语运动片段进行拼接合成,最终生成一个完整的手语运动,并基于VRML的人体运动显示方法将合成的运动逼真地显示出来.基于该方法,在PC/Windows/VC6.0环境下实现了一个中国聋人手语合成系统.该系统采集了<中国手语>(含续集)中收录的5 596个手语词,可以合成一般生活与教学用语.经聋校的老师和学生确认,合成手语准确逼真,可以广泛应用于教学、电视、Internet 等多种大众媒体,帮助聋人参与其他听力正常人的活动,具有广泛的应用前景和重要的社会意义.     

基于双重模型的计算机手语动作的实现方法

首先给出了一个新的方法来建立手的模型.这个手的模型由两个模型组成:一个是实现运动的骨骼模型,其决定了运动姿态及运动约束;另一个是表现运动的皮肤(表面)模型,用来表现真实感.其次,根据该模型实现手语的动作,利用该方法实现手语动作的关键技术,包括手及手臂旋转变换的变换矩阵、有向拓扑结构描述骨骼模型、表现运动的皮肤模型的拓扑不变性、以及骨骼模型实现运动和皮肤模型表现运动的方法.该方法具有一定的通用性,为网络环境下实现异语种的手语表达与传输打下了基础.     

Gesture Spotting and Recognition for Human–Robot Interaction

Visual interpretation of gestures can be useful in accomplishing natural human-robot interaction (HRI). Previous HRI research focused on issues such as hand gestures, sign language, and command gesture recognition. Automatic recognition of whole-body gestures is required in order for HRI to operate naturally. This presents a challenging problem, because describing and modeling meaningful gesture patterns from whole-body gestures is a complex task. This paper presents a new method for recognition of whole-body key gestures in HRI. A human subject is first described by a set of features, encoding the angular relationship between a dozen body parts in 3-D. A feature vector is then mapped to a codeword of hidden Markov models. In order to spot key gestures accurately, a sophisticated method of designing a transition gesture model is proposed. To reduce the states of the transition gesture model, model reduction which merges similar states based on data-dependent statistics and relative entropy is used. The experimental results demonstrate that the proposed method can be efficient and effective in HRI, for automatic recognition of whole-body key gestures from motion sequences