基于手势数据分析的地方手语动画合成方法研究*

为了在现有的手势数据基础上合成新的地方手语手势,采取对这些手势数据进行运动相似性分段和动态聚类自动获取具有时序特征的基本手形类数据以及结合手臂状态数据的方法,最后生成新的地方手语手势动画数据。这不仅节省了重新采集数据需要的昂贵设备和大量工作,而且也能达到手势表达准确性的目的。     

Computer animated visual translation from natural language to sign language

Computer animation and visualization can facilitate communication between the hearing impaired and those with normal speaking capabilities. This paper presents a model of a system that is capable of translating text from a natural language into animated sign language. Techniques have been developed to analyse language and transform it into sign language in a systematic way. A hand motion coding method as applied to the hand motion representation, and control has also been investigated. Two translation examples are also given to demonstrate the practicality of the system.     

Educational resources and implementation of a Greek sign language synthesis architecture

In this paper, we present how creation and dynamic synthesis of linguistic resources of Greek Sign Language (GSL) may serve to support development and provide content to an educational multitask platform for the teaching of GSL in early elementary school classes. The presented system utilizes standard virtual character (VC) animation technologies for the synthesis of sign sequences/streams, exploiting digital linguistic resources of both lexicon and grammar of GSL. Input to the system is written Greek text, which is transformed into GSL and animated on screen. To achieve this, a syntactic parser decodes the structural patterns of written Greek and matches them into equivalent patterns of GSL, which are then signed by a VC. The adopted notation system for the representation of GSL phonology incorporated in the system’s lexical knowledge database, is Hamburg Notation System (HamNoSys). For the implementation of the virtual signer tool, the definition of the VC follows the h-anim standard and is implemented in a web browser using a standard VRML plug-in.     

Motion synthesis and editing for the generation of new sign language content

Machine Translation - Existing work on the animation of signing avatars often relies on pure procedural techniques or on the playback of Motion Capture (MoCap) data. While the first solution...     

Study of sign segmentation in the text of Chinese sign language

Universal Access in the Information Society - The natural language processing (NLP) of sign language aims to make human sign language “understandable” to computers. In achieving this...     

Machine translation from text to sign language: a systematic review

Universal Access in the Information Society - An equal opportunity for all is the basic right of every human being. The deaf society of the world needs to have access to all the information just...     

Automatic translation of Arabic text-to-Arabic sign language

Universal Access in the Information Society - Arabic sign language (ArSL) is a full natural language that is used by the deaf in Arab countries to communicate in their community. Unfamiliarity with...     

Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language

Because of using traditional hand-sign segmentation and classification algorithm,many diversities of Bangla language including joint-letters,dependent vowels etc.and representing 51 Bangla written characters by using only 36 hand-signs,continuous hand-sign-spelled Bangla sign language(BdSL)recognition is challenging.This paper presents a Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language which consists of two phases.First phase is designed for hand-sign classification and the second phase is designed for Bangla language modeling algorithm(BLMA)for automatic recognition of hand-sign-spelled Bangla sign language.In first phase,we have proposed two step classifiers for hand-sign classification using normalized outer boundary vector(NOBV)and window-grid vector(WGV)by calculating maximum inter correlation coefficient(ICC)between test feature vector and pre-trained feature vectors.At first,the system classifies hand-signs using NOBV.If classification score does not satisfy specific threshold then another classifier based on WGV is used.The system is trained using 5,200 images and tested using another(5,200×6)images of 52 hand-signs from 10 signers in 6 different challenging environments achieving mean accuracy of 95.83%for classification with the computational cost of 39.972 milliseconds per frame.In the Second Phase,we have proposed Bangla language modeling algorithm(BLMA)which discovers allhidden charactersbased onrecognized charactersfrom 52 hand-signs of BdSL to make any Bangla words,composite numerals and sentences in BdSL with no training,only based on the result of first phase.To the best of our knowledge,the proposed system is the first system in BdSL designed on automatic recognition of hand-sign-spelled BdSL for large lexicon.The system is tested for BLMA using hand-sign-spelled 500 words,100 composite numerals and 80 sentences in BdSL achieving mean accuracy of 93.50%,95.50%and 90.50%respectively.     

Computation of the structured stability radius via matrix sign function

Robustness of perturbed state space models of the form is considered, where B, C are given matrices, A is an asymptotically stable matrix and D is the unknown perturbation matrix. An efficient algorithm to compute the complex structured stability radius, which is based on the properties of the matrix sign function, is presented. A comparison with previous algorithms shows the efficiency of the new algorithm     

The application of structured learning in natural language processing

We propose a structured learning approach, max-margin structure (MMS), which is targeted at natural language processing (NLP) tasks. The architecture of our approach is shown to capture structural aspects of the problem domains, leading to demonstrable performance improvements on two NLP tasks: part-of-speech tagging and statistical machine translation (SMT). We present a perceptron-based online learning algorithm to train the model and demonstrate desirable computational scaling behavior over traditional optimisation methods.     

Performance analysis of Otsu thresholding for sign language segmentation

Multimedia Tools and Applications - Sign language recognition system generally consists of three main processes, which are segmentation, modelling, and classification. Image segmentation plays a...     

手语识别中基于HMM的区分性训练方法

传统的隐马尔科夫模型（HMM）的训练方法基于统计概率的最大似然准则（MLE），在训练样本数目足够大的情况下，这种方法在理论上可以得到最优的结果．在手语识别研究中，采集足够大的训练样本十分困难．区分性训练可以很好地弥补由于训练样本的缺乏以及手语模型之间的近似而造成的识别系统的缺陷．最大交互信息准则（MMIE）作为区分性训练准则的一种已经被广泛的应用于语音识别领域．文中通过合理的构建手语识别中的竞争模型和易混集。提出了MMIE准则的改进形式，并将其应用于特定人与非特定人手语识别．实验证明，使用改进的MMIE准则对识别系统性能有很大的提高．     

Synthesis of sign language co-articulation based on key frames

Multimedia Tools and Applications - Co-articulation is a language phenomenon. In sign language (SL), it takes the form of impact among adjacent signs, which results in variations of signs from...     

基于关键帧的多级分类手语识别研究*

提出了一种基于关键帧识别的多级分类的手语识别方法,该方法采用HDR(多层判别回归)/DTW（动态时间规正）模板匹配多级分类方法。根据手语表达由多帧构成的特点,采用SIFT(尺度不变特征变换)算法定位获取手语词汇的关键帧,并提取其特征向量;根据手语词汇的关键帧采用HDR方法缩小搜索范围,然后采用DTW比较待识别的手语词特征与该范围内每一个手语词进行匹配比较,计算概率最大的为识别结果。这种方法在相同识别率的情况下比HMM识别方法速度提高近8.2%,解决了模板匹配法在大词汇量面前识别率快速下降的问题。     

Improving realism in automated fingerspelling of American sign language

Machine Translation - Fingerspelling is a process of communicating letters of a spoken language alphabet using a person’s hand or hands. Portraying animations of fingerspelling has proved...     

Optimized H.264 compression of sign language video

In TV broadcasting deaf people are not able to get information from the audio content. In public television, some programs may be accompanied by a sign language interpreter as a part of the broadcasted signal. As a supplementary service, it would enable more programs to be accessible with a sign language interpreter to assist in comprehension. To be able to transmit such data flow separately, we define the parameters of compression of sign language interpreter image to ensure intelligibility and quality while maintaining low bitrate. This paper deals with specific video compression of Czech sign language interpreter based on regions of interest implemented to the ×264 open source library. The results of this approach are verified in subjective tests with the deaf and hearing evaluators. The experiments examine the intelligibility of sign language expressions containing minimal pairs for different levels of image compression and also evaluate the subjective quality of the final image.