基于HOG特征的稀疏编码手语识别方法研究

为提高手语识别方法的识别速度与识别率,提出一种基于HOG特征的稀疏编码手语识别方法.通过基于学习加权局部特征的监督、判别和面向事件的字典,将手语识别表达为稀疏表示问题.对每一类手语样本的HOG特征进行提取,再用LC-KSVD算法来学习面向事件和辨别的字典,以将样本数据传输到稀疏空间.鉴于不同类别样本之间的区别,采用提取...     

基于深度学习的连续手语语句识别算法

目前,关于连续手语语句识别的研究相对较少,原因在于难以有效地分割出手语词。该文利用卷积神经网络提取手语词的手型特征,同时利用轨迹归一化算法提取手语词的轨迹特征,并在此基础上完成长短期记忆网络的构建,从而为手语语句识别准备好手语词分类器。对于一个待识别的手语语句,采用基于右手心轨迹信息的分割算法来检测过渡动作。由过渡动作可以将语句分割为多个片段,考虑到某些过渡动作可能是手语词内部的动作,所以将若干个片段拼接成一个复合段,并按照层次遍历的次序对所有复合段运用手语词分类器进行识别。最后,采用跨段搜索的动态规划算法寻找最大后验概率的词汇序列,从而完成手语语句的识别。实验结果表明,该算法可以对47个常用手语词组成的语句做出识别,且具有较高的准确性和实时性。     

智能仿生双向手语翻译系统

设计了一套智能仿生双向手语翻译系统,该系统主要由STM32微处理器、LD3320非特定语音识别模块、SYN6288语音合成芯片等组成,能够实现语音与手势的双向翻译。其中语音转手势部分可通过语音识别模块获得指令,手语机器人根据指令完成语音转动作的翻译。手势转语音部分通过数据手套捕获手臂的动作和姿态,识别手语动作,控制手语机器人发出语音。该系统具有成本低、识别度高、使用方便等优势,具有良好的应用前景。     

基于关键帧的连续手语语句识别算法研究

目前,对于动态手语的识别大多只是针对手语词汇的,对连续的手语语句的识别研究以及相应成果较少,原因在于难以对其进行有效的分割。提出了一种基于加权关键帧的手语语句识别算法。关键帧可以看作是手语词汇的基本组成单元,根据关键帧即可得到相关词汇,并将其组成连续的手语语句,从而避免了对手语语句直接做分割的难点。借助于体感设备,首先提出了一种基于手语轨迹的自适应关键帧提取算法,然后根据关键帧包含的语义对其进行加权处理,最后设计了基于加权关键帧序列的识别算法,得到连续的手语语句。实验证明,设计的算法可以实现对连续手语语句的实时识别。     

基于三维手部骨架数据的连续手语识别

为有效地消除手语识别过程中背景、光照等干扰因素带来的视觉问题,采用低冗余的骨架数据表达手语信息,设计了一个端到端连续手语识别模型.首先,分别从帧内和帧间提取手型和轨迹特征,可以有效地降低原始样本的离散程度;其次,构建一系列并行的双路残差网络对手型和轨迹特征进行优化与融合,生成时空特征序列;最后,基于注意力机制的编码-解码网络实现时空特征序列到翻译文本的映射.使用Leap Motion收集建立了一个基于三维手部骨架数据的手语数据集LMSLR.实验结果表明,在LMSLR数据集和公共的CSL数据集上,该模型与大多数基于视频处理的模型相比具有较高的准确率和较小的计算量.     

基于稀疏降噪自编码的电能质量扰动识别研究

针对传统电网电能质量扰动识别中特征向量提取精度不足、扰动信号识别率不高的问题,提出使用稀疏降噪自编码模型构建扰动分类识别网络.首先对扰动信号样本做降噪腐蚀处理,并对误差函数添加稀疏惩罚项和权重衰减项.然后采用梯度下降法求解误差偏导函数方程,增强了整个自编码模型的数据特征提取能力.最后选用Logistic分类器对特征向量...     

基于二级匹配策略的实时动态手语识别

动态手语可以利用其轨迹与关键手型加以描述。大量的统计实验数据表明,大多数的常用手语通过轨迹曲线的匹配即可实现识别,因此,提出一种针对动态手语的分级匹配识别算法。首先利用体感设备获取手势轨迹,并根据轨迹的点密度分布设计了一种关键帧检测算法以提取手势的关键手型,结合轨迹的曲线特征,实现对动态手语的精确描述。然后利用优化的动态时间规整(DTW)算法完成对手语的一级匹配,即轨迹匹配。若此时可以得到识别结果,那么识别过程可以结束,否则进入二级匹配,即针对关键手型再做匹配识别,从而得到最终的识别结果。实验证明,所提算法不仅实时性好,识别的准确率也较高。     

非特定人手语识别进展及关键问题研究思路

非特定人手语识别是推动手语系统实用化所必须解决的问题.在非特定人手语识别研究中,训练数据的缺乏和非特定人手语数据的差异性矛盾给原有研究框架的有效性带来了挑战.提出了非特定人手语识别新的研究框架,并给出了解决问题的策略与思路.这些问题的解决将对中国手语识别及其他相关领域具有非常重要的意义.     

改进SSD算法在中国手语识别上的应用

基于计算机视觉的手语识别技术能为聋校双语教学带来很大的便利.近年来,随着深度学习技术的蓬勃发展,手语识别的准确率和速度有了极大的提高.与使用颜色标记和外界技术(如Kinect手心定位技术)的方法不同,提出一种改进的SSD(Single-Shot Multibox Detector)网络,对手势进行目标检测完成中国手语识...     

基于深度自编码的医疗命名实体识别模型

针对在医疗命名实体识别（MNER）问题中随着网络加深,基于深度学习的识别模型出现的识别精度与算力要求不平衡的问题,提出一种基于深度自编码的医疗命名实体识别模型CasSAttMNER。首先,使用编码与解码间深度差平衡策略,以经过蒸馏的Transformer语言模型RBT6作为编码器以减小编码深度以及降低对训练和应用上的算力要求;然后,使用双向长短期记忆（BiLSTM）网络和条件随机场（CRF）提出了级联式多任务双解码器,从而完成实体提及序列标注与实体类别判断;最后,基于自注意力机制在实体类别中增加实体提及过程抽取的隐解码信息,以此来优化模型设计。实验结果表明,CasSAttMNER在两个中文医疗实体数据集上的F值度量可分别达到0.943 9和0.945 7,较基线模型分别提高了3个百分点和8个百分点,验证了该模型更进一步地提升了解码器性能。     

基于Kinect的手语识别方法

为实现基于Kinect的手语识别,提出了一种利用有限状态机及动态时间规整(DTW)的动态手语识别方法。首先,利用Kinect技术得到人体深度图像和骨骼特征信息;然后利用手部分割算法得到手部深度图像,再选取识别正确率高的梯度方向直方图(HOG)特征算子来提取手部特征;最后加入有限状态机和DTW算法实现动态手语识别。实验结果表明:该方法能够实现对常用手语单词、句子的识别,识别准确率可达95%。     

A Chinese sign language recognition system based on SOFM/SRN/HMM

In sign language recognition (SLR), the major challenges now are developing methods that solve signer-independent continuous sign problems. In this paper, SOFM/HMM is first presented for modeling signer-independent isolated signs. The proposed method uses the self-organizing feature maps (SOFM) as different signers' feature extractor for continuous hidden Markov models (HMM) so as to transform input signs into significant and low-dimensional representations that can be well modeled by the emission probabilities of HMM. Based on these isolated sign models, a SOFM/SRN/HMM model is then proposed for signer-independent continuous SLR. This model applies the improved simple recurrent network (SRN) to segment continuous sign language in terms of transformed SOFM representations, and the outputs of SRN are taken as the HMM states in which the lattice Viterbi algorithm is employed to search the best matched word sequence. Experimental results demonstrate that the proposed system has better performance compared with conventional HMM system and obtains a word recognition rate of 82.9% over a 5113-sign vocabulary and an accuracy of 86.3% for signer-independent continuous SLR.     

Australian sign language recognition

This paper presents an automatic Australian sign language (Auslan) recognition system, which tracks multiple target objects (the face and hands) throughout an image sequence and extracts features for the recognition of sign phrases. Tracking is performed using correspondences of simple geometrical features between the target objects within the current and the previous frames. In signing, the face and a hand of a signer often overlap, thus the system needs to segment these for the purpose of feature extraction. Our system deals with the occlusion of the face and a hand by detecting the contour of the foreground moving object using a combination of motion cues and the snake algorithm. To represent signs, features that are invariant to scaling, 2D rotations and signing speed are used for recognition. The features represent the relative geometrical positioning and shapes of the target objects, as well as their directions of motion. These are used to recognise Auslan phrases using Hidden Markov Models. Experiments were conducted using 163 test sign phrases with varying grammatical formations. Using a known grammar, the system achieved over 97% recognition rate on a sentence level and 99% success rate at a word level.     

Large vocabulary sign language recognition based on fuzzy decision trees

The major difficulty for large vocabulary sign recognition lies in the huge search space due to a variety of recognized classes. How to reduce the recognition time without loss of accuracy is a challenging issue. In this paper, a fuzzy decision tree with heterogeneous classifiers is proposed for large vocabulary sign language recognition. As each sign feature has the different discrimination to gestures, the corresponding classifiers are presented for the hierarchical decision to sign language attributes. A one- or two- handed classifier and a hand-shaped classifier with little computational cost are first used to progressively eliminate many impossible candidates, and then, a self-organizing feature maps/hidden Markov model (SOFM/HMM) classifier in which SOFM being as an implicit different signers' feature extractor for continuous HMM, is proposed as a special component of a fuzzy decision tree to get the final results at the last nonleaf nodes that only include a few candidates. Experimental results on a large vocabulary of 5113-signs show that the proposed method dramatically reduces the recognition time by 11 times and also improves the recognition rate about 0.95% over single SOFM/HMM.     

Local Binary Pattern based features for sign language recognition

In this paper we focus on appearance features particularly the Local Binary Patterns describing the manual component of Sign Language. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

Local binary pattern based features for sign language recognition

In this paper we focus on appearance features describing the manual component of Sign Language particularly the Local Binary Patterns. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

Understanding vision-based continuous sign language recognition

Multimedia Tools and Applications - Real-time sign language translation systems, that convert continuous sign sequences to text/speech, will facilitate communication between the deaf-mute community...     

Real-time American sign language recognition using desk andwearable computer based video

We present two real-time hidden Markov model-based systems for recognizing sentence-level continuous American sign language (ASL) using a single camera to track the user's unadorned hands. The first system observes the user from a desk mounted camera and achieves 92 percent word accuracy. The second system mounts the camera in a cap worn by the user and achieves 98 percent accuracy (97 percent with an unrestricted grammar). Both experiments use a 40-word lexicon     

Non-manual cues in automatic sign language recognition

Present work deals with the incorporation of non-manual cues in automatic sign language recognition. More specifically, eye gaze, head pose, and facial expressions are discussed in relation to their grammatical and syntactic function and means of including them in the recognition phase are investigated. Computer vision issues related to extracting facial features, eye gaze, and head pose cues are presented and classification approaches for incorporating these non-manual cues into the overall Sign Language recognition architecture are introduced.