Local Binary Pattern based features for sign language recognition

In this paper we focus on appearance features particularly the Local Binary Patterns describing the manual component of Sign Language. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

Local binary pattern based features for sign language recognition

In this paper we focus on appearance features describing the manual component of Sign Language particularly the Local Binary Patterns. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

An efficient traffic sign recognition based on graph embedding features

Neural Computing and Applications - Traffic sign recognition (TSR) is one of the significant modules of an intelligent transportation system. It instantly assists the drivers to efficiently...     

基于自编码的中国手语识别研究

针对动态手语上下文联系强的特点,采用LSTM(Long Short-Term Memory)循环神经网络用于识别,同时,利用自编码实现动态手语的无监督学习。该方法将数据手套作为采集设备,在获取手语信息后,通过编码器、解码器生成手语的重构矢量。在实验过程中,将采集的数据集进行模型的训练,并与监督学习的结果进行比较。实验结果表明,该方法能够有效进行手语识别的无监督学习。     

基于Kinect的手语识别方法

为实现基于Kinect的手语识别,提出了一种利用有限状态机及动态时间规整(DTW)的动态手语识别方法。首先,利用Kinect技术得到人体深度图像和骨骼特征信息;然后利用手部分割算法得到手部深度图像,再选取识别正确率高的梯度方向直方图(HOG)特征算子来提取手部特征;最后加入有限状态机和DTW算法实现动态手语识别。实验结果表明:该方法能够实现对常用手语单词、句子的识别,识别准确率可达95%。     

连续手语识别中的文本纠正和补全方法

针对基于视频的连续手语识别的文本结果存在语义模糊、语序混乱的问题,提出一种两步法将连续手语识别结果的手语文本转化为通顺、可懂的汉语文本.第一步,基于自然手语规则以及N元语言模型(N-gram)对连续手语识别的结果进行文本调序;第二步,利用汉语通用量词数据集训练双向长短期记忆(Bi-LSTM)网络模型,以解决手语语法无量...     

Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language

Because of using traditional hand-sign segmentation and classification algorithm,many diversities of Bangla language including joint-letters,dependent vowels etc.and representing 51 Bangla written characters by using only 36 hand-signs,continuous hand-sign-spelled Bangla sign language(BdSL)recognition is challenging.This paper presents a Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language which consists of two phases.First phase is designed for hand-sign classification and the second phase is designed for Bangla language modeling algorithm(BLMA)for automatic recognition of hand-sign-spelled Bangla sign language.In first phase,we have proposed two step classifiers for hand-sign classification using normalized outer boundary vector(NOBV)and window-grid vector(WGV)by calculating maximum inter correlation coefficient(ICC)between test feature vector and pre-trained feature vectors.At first,the system classifies hand-signs using NOBV.If classification score does not satisfy specific threshold then another classifier based on WGV is used.The system is trained using 5,200 images and tested using another(5,200×6)images of 52 hand-signs from 10 signers in 6 different challenging environments achieving mean accuracy of 95.83%for classification with the computational cost of 39.972 milliseconds per frame.In the Second Phase,we have proposed Bangla language modeling algorithm(BLMA)which discovers allhidden charactersbased onrecognized charactersfrom 52 hand-signs of BdSL to make any Bangla words,composite numerals and sentences in BdSL with no training,only based on the result of first phase.To the best of our knowledge,the proposed system is the first system in BdSL designed on automatic recognition of hand-sign-spelled BdSL for large lexicon.The system is tested for BLMA using hand-sign-spelled 500 words,100 composite numerals and 80 sentences in BdSL achieving mean accuracy of 93.50%,95.50%and 90.50%respectively.     

Arabic sign language continuous sentences recognition using PCNN and graph matching

Many previous researchers have tried developing sign languages recognition systems in general and Arabic sign language specifically. They succeeded to achieve acceptable results for isolated gestures level, but none of them investigated the recognition of connected sequence of gestures. This paper focuses on how to recognize real-time connected sequence of gestures using graph-matching technique, also how the continuous input gestures are segmented and classified. Graphs are a general and powerful data structure useful for the representation of various objects and concepts. This work is a component of a real-time Arabic Sign Language Recognition system that applied pulse-coupled neural network for static posture recognition in its first phase. This work can be adapted and applied to different sign languages and other recognition problems.     

形状标记图和Gabor小波的交通标志识别

交通标志识别为智能车辆行驶提供了有价值的道路环境信息.提出一种结合形状标记图和Gabor波的交通标志识别方法,交通标志识别过程如下:1)变换图像的RGB像素值来增强交通标志主特征颜色(红,蓝,黄)区域并进行分割,用形态学操作消除噪声点的影响;2)提取感兴趣区域的标记图作为其形状特征,用Euclidean距离来对其进行初分类;3)对交通标志感兴趣区域的灰度图像进行Gabor小波变换,获得其不同角度和尺度的小波图像,用二维独立分量分析法提取其主特征,并送入线性支持向量机来判断感兴趣区域所属的交通标志类型.实验结果表明,提出的算法能够稳定、有效地检测和识别智能车辆行驶环境中的多类交通标志.     

基于Gabor和改进LDA的人耳识别

针对人耳识别中无法避免的小样本问题,提出了基于Gabor特征和改进LDA(ILDA)的识别算法。该算法首先提取人耳局部Gabor特征,然后重新定义Fisher准则和类内分散度矩阵,再将高维空间映射到低维后寻找最优投影方向,最后利用训练样本与测试样本特征投影值的欧氏距离进行分类识别。与传统方法相比,新算法能有效解决人耳识别中的小样本问题,获得较高的识别准确率。     

Efficient fall activity recognition by combining shape and motion features

Computational Visual Media - This paper presents a vision-based system for recognizing when elderly adults fall. A fall is characterized by shape deformation and high motion. We represent shape...     

Large vocabulary sign language recognition based on fuzzy decision trees

The major difficulty for large vocabulary sign recognition lies in the huge search space due to a variety of recognized classes. How to reduce the recognition time without loss of accuracy is a challenging issue. In this paper, a fuzzy decision tree with heterogeneous classifiers is proposed for large vocabulary sign language recognition. As each sign feature has the different discrimination to gestures, the corresponding classifiers are presented for the hierarchical decision to sign language attributes. A one- or two- handed classifier and a hand-shaped classifier with little computational cost are first used to progressively eliminate many impossible candidates, and then, a self-organizing feature maps/hidden Markov model (SOFM/HMM) classifier in which SOFM being as an implicit different signers' feature extractor for continuous HMM, is proposed as a special component of a fuzzy decision tree to get the final results at the last nonleaf nodes that only include a few candidates. Experimental results on a large vocabulary of 5113-signs show that the proposed method dramatically reduces the recognition time by 11 times and also improves the recognition rate about 0.95% over single SOFM/HMM.     

基于分块自适应融合特征的交通标志识别

交通标志由外部轮廓和内部指示符号组成,HOG特征可较好描述图像轮廓但易受噪声影响,而LBP特征对图像细节刻画好,提出基于分块HOG-LBP自适应融合特征的交通标志识别方法。通过分块计算梯度直方图得到的权重系数,来判断该块是属于轮廓还是内部指示,对前者选择HOG权重大,后者选择LBP特征权重大,将自适应串行融合后的特征送入支持向量机识别。仿真实验结果表明,该算法对标准交通标志识别率可达到100%,对含模糊、残缺、遮挡等非标准交通标志也达到了76%。     

基于关键帧的多级分类手语识别研究*

提出了一种基于关键帧识别的多级分类的手语识别方法,该方法采用HDR(多层判别回归)/DTW（动态时间规正）模板匹配多级分类方法。根据手语表达由多帧构成的特点,采用SIFT(尺度不变特征变换)算法定位获取手语词汇的关键帧,并提取其特征向量;根据手语词汇的关键帧采用HDR方法缩小搜索范围,然后采用DTW比较待识别的手语词特征与该范围内每一个手语词进行匹配比较,计算概率最大的为识别结果。这种方法在相同识别率的情况下比HMM识别方法速度提高近8.2%,解决了模板匹配法在大词汇量面前识别率快速下降的问题。     

基于形状特征的植物叶片在线识别方法

针对传统植物识别方法工作任务量大,效率低下以及难以保证数据客观性的问题,提出了一种基于形状特征的植物叶片识别算法,并开发了一款C/S模式的植物叶片在线识别Android应用。叶片图像经预处理后,提取叶片的轮廓凸包顶点比、轮廓曲率方差等形状特征,采用KNN-SVM对叶片进行分类识别。实验结果表明,相比于一些已有识别算法,该算法可以达到更高的识别率;该Android应用稳定可靠,可以满足用户的需求。     

A non-linear model of shape and motion for tracking finger spelt American sign language

This work presents a piecewise linear approximation to non-linear Point Distribution Models for modelling the human hand. The work utilises the natural segmentation of shape space, inherent to the technique, to apply temporal constraints, which can be used with CONDENSATION to support multiple hypotheses and discontinuous jumps within shape space. This paper presents a novel method by which the one-state transitions of the English Language are projected into shape space for tracking and model prediction using an HMM like approach. The paper demonstrates that this model of motion provides superior results to that of other tracking approaches.     

基于YOLO v3算法改进的交通标志识别算法

针对目前交通标志识别任务在使用深度学习算法时存在模型参数量大、实时性较差和准确率较低的问题,提出了基于YOLO v3改进的交通标志识别算法。该算法首先将深度可分离卷积引入YOLO v3算法的特征提取层,将卷积过程分解为深度卷积、逐点卷积两部分,实现通道内卷积与通道间卷积之间的分离,从而保证了在较高识别准确率的基础上极大地减少了算法模型参数数量以及计算量。其次,在损失函数设计上使用广义交并比（GIoU）损失替换均方误差（MSE）损失,将评测标准量化为损失,解决了MSE损失存在的优化不一致和尺度敏感的问题,同时将Focal损失加入到损失函数以解决正负样本严重不均衡的问题,通过降低大量简单背景类的权重使得算法更专注于检测前景类。将该算法应用于交通标志任务中的结果表明,在TT100K数据集上,该算法的平均精度均值（mAP）指标达到了89%,相较于YOLO v3算法提升了6.6个百分点,且其参数量仅为原始YOLO v3算法的1/5左右,每秒帧数（FPS）亦比YOLO v3算法提升了60%。该算法在极大地减少模型参数量和计算量的同时,提高了检测速度和检测精度。     

基于特征的江面轮船识别算法*

本文提出了一种江面轮船目标的特征识别方法。首先对江面轮船图像进行预处理,然后通过二维小波变换,提取出边缘轮廓,将目标物体与背景分离开来。结合本文提出的FE（Feature Extract）算法提取出轮船图像的四个特征,根据这些特征建造一个知识库,通过选取适当的知识,采用产生式规则对目标物体进行判别,排除干扰目标,从而识别出轮船目标。最后本文从图片库中抽取几张图片进行实验,相比于之前的单特征方法和Adboost方法可以看出,该方法在识别率上要高于单特征方法,在识别速度上要快于Adboost方法。     

Towards subject independent continuous sign language recognition: A segment and merge approach

This paper presents a segment-based probabilistic approach to robustly recognize continuous sign language sentences. The recognition strategy is based on a two-layer conditional random field (CRF) model, where the lower layer processes the component channels and provides outputs to the upper layer for sign recognition. The continuously signed sentences are first segmented, and the sub-segments are labeled SIGN or ME (movement epenthesis) by a Bayesian network (BN) which fuses the outputs of independent CRF and support vector machine (SVM) classifiers. The sub-segments labeled as ME are discarded and the remaining SIGN sub-segments are merged and recognized by the two-layer CRF classifier; for this we have proposed a new algorithm based on the semi-Markov CRF decoding scheme. With eight signers, we obtained a recall rate of 95.7% and a precision of 96.6% for unseen samples from seen signers, and a recall rate of 86.6% and a precision of 89.9% for unseen signers.