A belief-based sequential fusion approach for fusing manual signs and non-manual signals

Most of the research on sign language recognition concentrates on recognizing only manual signs (hand gestures and shapes), discarding a very important component: the non-manual signals (facial expressions and head/shoulder motion). We address the recognition of signs with both manual and non-manual components using a sequential belief-based fusion technique. The manual components, which carry information of primary importance, are utilized in the first stage. The second stage, which makes use of non-manual components, is only employed if there is hesitation in the decision of the first stage. We employ belief formalism both to model the hesitation and to determine the sign clusters within which the discrimination takes place in the second stage. We have implemented this technique in a sign tutor application. Our results on the eNTERFACE’06 ASL database show an improvement over the baseline system which uses parallel or feature fusion of manual and non-manual features: we achieve an accuracy of 81.6%.     

基于深度图像信息的手语识别算法

为了实现手语视频中手语字母的准确识别,提出了一种基于DI_CamShift和手语视觉单词（SLVW）的手语识别算法。首先采用Kinect获取手语字母手势视频及其深度信息;然后通过计算获得深度图像中手语手势的主轴方向角和质心位置,计算搜索窗口对手势跟踪;进而使用基于深度积分图像的Ostu算法分割手势并提取其尺度不变特征转换(SIFT)特征;最后构建SLVW词包并用支持向量机(SVM)进行识别。单个手语字母最好识别率为99.67%,平均识别率96.47%     

采用深度图像信息和SLVW的手语识别

为了实现手语视频中手语字母的准确识别,提出了一种基于DI_CamShift和SLVW的算法。该方法将Kinect作为手语视频采集设备,在获取彩色视频的同时得到其深度信息;计算深度图像中手语手势的主轴方向角和质心位置,通过调整搜索窗口对手势进行准确跟踪;使用基于深度积分图像的Ostu算法分割手势,并提取其SIFT特征;构建了SLVW词包作为手语特征,并用SVM进行识别。通过实验验证该算法,其单个手语字母最好识别率为99.87%,平均识别率96.21%。     

一种用于交通标识分类的形状识别算法

交通标识分类是交通标识识别系统的基础环节,而交通标识形状识别是交通标识分类的核心部分.对交通标识进行了研究,将交通标识分为禁令标识、警告标识和指示标识3大类分别进行分析,提出了一种利用边缘走势统计特征反映目标形状特征的新算法,并将其与BP神经网络相结合用于交通标识形状的识别.首先利用颜色信息实现交通标识区域分割,随后记...     

基于高稳定SURF特征的交通标志识别

针对光照变化和部分遮挡这两种情形,提出一种基于多帧视频图像的高稳定特征的交通标志识别方法。利用有交通标志的多帧视频图像的SURF特征建立bag of SURFs特征向量集,与标准交通标志图像的模板特征向量集匹配,采用权值计分策略的最高得分确定交通标志的识别结果。对三种情形下的公开视频图像集进行了实验并与最新方法进行对比分析,结果表明新方法的交通标志识别效果具有明显的优越性,是在光照变化和部分遮挡情形下一种有效的交通标志识别方法。     

基于词根的中国手语识别方法

迄今为止，手语识别面临的最大问题是如何解决词汇集易扩充的连续识别，提出一种大词汇量连续中国手语识别方法，将词根作为识别基元，由于基元的数目是有限的，因此基于HMM的手语信号的训练和识别变得比较容易处理，可以实现更大词汇量的识别。除此之外，所提方法还有利于实现手势语和手指语的混合识别。从中国手语中共整理现2400多个词根，为每个词根建一个并行的HMM模型，对各数据流的HMM模型进行聚集，确定出手识别的基元。根据这些基元对手妫刻苦骊，并建立了树状搜索网络，使用状态垄点上高斯密度函数聚类、语言模型和N－Best方法提高系统的速度和精度。对5119个手语词做了实验，连续语句的识别率可在90％以上。     

Local Binary Pattern based features for sign language recognition

In this paper we focus on appearance features particularly the Local Binary Patterns describing the manual component of Sign Language. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

Local binary pattern based features for sign language recognition

In this paper we focus on appearance features describing the manual component of Sign Language particularly the Local Binary Patterns. We compare the performance of these features with geometric moments describing the trajectory and shape of hands. Since the non-manual component is also very important for sign recognition we localize facial landmarks via Active Shape Model combined with Landmark detector that increases the robustness of model fitting. We test the recognition performance of individual features and their combinations on a database consisting of 11 signers and 23 signs with several repetitions. Local Binary Patterns outperform the geometric moments. When the features are combined we achieve a recognition rate up to 99.75% for signer dependent tests and 57.54% for signer independent tests.     

交通标志识别特征提取研究综述

交通标志识别(TSR)是智能交通系统(ITS)的一个重要研究方向,而特征提取是交 通标志识别研究中的重点。聚焦交通标志识别的特征提取,综述了常见的人工特征(颜色直方图、 尺度不变特征变换特征、局部二值模式特征、方向梯度直方图特征、Haar-like 特征、Gabor 小 波特征、Canny 特征等)和深度特征(提取自 AlexNet,VGG16,Inception 等),并在同一数据集 (GTSRB)上提取多种特征,采用相同分类器,通过相同评价指标体系进行定量比较与分析,并 以图表方式,针对不同特征和不同交通标志类别,进行直观的性能比较研究,以期为交通标志 识别时特征向量的选择和深入研究提供参考。     

Detection and classification of road signs in natural environments

An automatic road sign recognition system first locates road signs within images captured by an imaging sensor on-board of a vehicle, and then identifies the detected road signs. This paper presents an automatic neural-network-based road sign recognition system. First, a study of the existing road sign recognition research is presented. In this study, the issues associated with automatic road sign recognition are described, the existing methods developed to tackle the road sign recognition problem are reviewed, and a comparison of the features of these methods is given. Second, the developed road sign recognition system is described. The system is capable of analysing live colour road scene images, detecting multiple road signs within each image, and classifying the type of road signs detected. The system consists of two modules: detection and classification. The detection module segments the input image in the hue-saturation-intensity colour space, and then detects road signs using a Multi-layer Perceptron neural-network. The classification module determines the type of detected road signs using a series of one to one architectural Multi-layer Perceptron neural networks. Two sets of classifiers are trained using the Resillient-Backpropagation and Scaled-Conjugate-Gradient algorithms. The two modules of the system are evaluated individually first. Then the system is tested as a whole. The experimental results demonstrate that the system is capable of achieving an average recognition hit-rate of 95.96% using the scaled-conjugate-gradient trained classifiers.     

Exploiting street-level panoramic images for large-scale automated surveying of traffic signs

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.     

基于手语视觉单词特征的手语字母识别研究

为有效识别手语字母,提出一种手语视觉单词(SLVW)的识别方法。采用Kinect获取手语字母视频及其深度信息,在深度图像中,通过计算获得手语手势的主轴方向角和质心位置以调整搜索窗口,利用基于深度图像信息的DI_CamShift方法对手势进行跟踪,进而使用基于深度积分图像的Ostu方法分割手势,并提取其尺度不变特征变换数据。将局部特征描述子表示的图像小区域量化生成SLVW,统计一幅手语图像中的视觉单词频率,用词包模型表示手语字母,并用支持向量机进行识别。实验结果表明,该方法不受颜色、光照和阴影的干扰,具有较高的识别准确性和鲁棒性,对复杂背景手语视频中的30个手语字母的平均识别率达到96.21%。     

视觉位置识别中代表地点的标识牌算法

针对视觉位置识别中因检索全局图片而带来大量的时间消耗情况,以及不同地点的视觉图像存在相似和同一地点从不同视角看起来会不尽相同而导致感知混淆的问题.本文提出一种基于显著性算法提取候选对象并生成代表地点的标识牌算法.该方法对在位置识别系统中每个地点其对应的视频序列段上的关键帧使用显著性算法,生成大量的视觉显著的候选对象,并用对这些候选对象有效计算其之间的评价函数,再使用层次聚类算法计算出每一段序列上具有代表性的对象,最后将这些对象组合成具有代表视频序列的标识牌.使用标识牌代表地点的方式,插入位置识别系统中搜索地点对应的大量图像集的前一个步骤中,以此来缩小搜索范围,避免感知混淆所带来的全局搜索不确定的困惑.     

Understanding gestures with systematic variations in movement dynamics

Sign language communication includes not only lexical sign gestures but also grammatical processes which represent inflections through systematic variations in sign appearance. We present a new approach to analyse these inflections by modelling the systematic variations as parallel channels of information with independent feature sets. A Bayesian network framework is used to combine the channel outputs and infer both the basic lexical meaning and inflection categories. Experiments using a simulated vocabulary of six basic signs and five different inflections (a total of 20 distinct gestures) obtained from multiple test subjects yielded 85.0% recognition accuracy. We also propose an adaptation scheme to extend a trained system to recognize gestures from a new person by using only a small set of data from the new person. This scheme yielded 88.5% recognition accuracy for the new person while the unadapted system yielded only 52.6% accuracy.     

Coupled grouping and matching for sign and gesture recognition

Matching an image sequence to a model is a core problem in gesture or sign recognition. In this paper, we consider such a matching problem, without requiring a perfect segmentation of the scene. Instead of requiring that low- and mid-level processes produce near-perfect segmentation, we take into account that such processes can only produce uncertain information and use an intermediate grouping module to generate multiple candidates. From the set of low-level image primitives, such as constant color region patches found in each image, a ranked set of salient, overlapping, groups of these primitives are formed, based on low-level cues such as region shape, proximity, or color. These groups corresponds to underlying object parts of interest, such as the hands. The sequence of these frame-wise group hypotheses are then matched to a model by casting it into a minimization problem. We show the coupling of these hypotheses with both non-statistical matching (match to sample-based modeling of signs) and statistical matching (match to HMM models) are possible. Our algorithm not only produces a matching score, but also selects the best group in each image frame, i.e. recognition and final segmentation of the scene are coupled. In addition, there is no need for tracking of features across sequences, which is known to be a hard task. We demonstrate our method using data from sign language recognition and gesture recognition, we compare our results with the ground truth hand groups, and achieved less than 5% performance loss for both two models. We also tested our algorithm on a sports video dataset that has moving background.     

面向大词汇量的实时连续中国手语识别系统

目前手语识别研究的难点之一在于如何实现大词汇量的连续语句识别，针对这个问题，该文提出了几个有效的方法，包括修正转移矩阵，状态结点的捆绑，快速匹配，在搜索路径中加入词跳转的估计参数等。利用上述技术，该文实现了一个基于数据手套和位置跟踪器的大词汇量的连续的中国手语实时识别系统，对中国手语辞典中收录的5100个词以及一批连续语句作实验，实验结果表明，文中所介绍的技术在提高系统识别速度和准确率方面都很有效。     

Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models

The major challenges that sign language recognition (SLR) now faces are developing methods that solve large-vocabulary continuous sign problems. In this paper, transition-movement models (TMMs) are proposed to handle transition parts between two adjacent signs in large-vocabulary continuous SLR. For tackling mass transition movements arisen from a large vocabulary size, a temporal clustering algorithm improved from k-means by using dynamic time warping as its distance measure is proposed to dynamically cluster them; then, an iterative segmentation algorithm for automatically segmenting transition parts from continuous sentences and training these TMMs through a bootstrap process is presented. The clustered TMMs due to their excellent generalization are very suitable for large-vocabulary continuous SLR. Lastly, TMMs together with sign models are viewed as candidates of the Viterbi search algorithm for recognizing continuous sign language. Experiments demonstrate that continuous SLR based on TMMs has good performance over a large vocabulary of 5113 Chinese signs and obtains an average accuracy of 91.9%