Hidden Markov modeling using a dominant state sequence with application to speech recognition

Approximate maximum likelihood (ML) hidden Markov modeling using the most likely state sequence (MLSS) is examined and compared with the exact ML approach that considers all possible state sequences. It is shown that for any hidden Markov model (HMM), the difference between the approximate and the exact normalized likelihood functions cannot exceed the logarithm of the number of states divided by the dimension of the output vectors (frame length), which is negligible for typically used values of vector dimension (128–256) and number of states (2–30). Furthermore, for Gaussian HMMs and a given observation sequence, the MLSS is typically the sequence of nearest neighbor states in the Itakura-Saito sense, and the posterior probability of any state sequence which departs from the MLSS in a single time instant decays exponentially with the frame length. Hence, for a sufficiently large frame length the exact and approximate ML approaches provide similar model estimates and likelihood values. The results and their implications on speech recognition are demonstrated in a set of experiments.     

A Hierarchical Self-orginising Map Model for Sequence Recognition

This paper presents an analysis of an original hierarchical neural model on a complex sequence - the complete sixteenth fourpart fugue in G minor of the Well-Tempered Clavier (vol 1) of J. S. Bach. The model makes an effective use of context information, through its hierarchical topology and embedded time integrators, and that enables it to keep a very good account of past events. The model performs sequence classification and discrimination efficiently. It has application in domains which require pattern recognition, or particulary, which demand recognising either a set if sequences of vectors in time, or sub-sequences into a unique and large sequence of vectors in time. Received: 16 March 1999, Received in revised form: 30 July 1999, Accepted: 27 September 1999     

Speech recognition with continuous-parameter hidden Markov models

The acoustic modeling problem in automatic speech recognition is examined from an information-theoretic point of view. This problem is to design a speech-recognition system which can extract from the speech waveform as much information as possible about the corresponding word sequence. The information extraction process is broken down into two steps: a signal-processing step which converts a speech waveform into a sequence of information-bearing acoustic feature vectors, and a step which models such a sequence. We are primarily concerned with the use of hidden Markov models to model sequences of feature vectors which lie in a continuous space. We explore the trade-off between packing information into such sequences and being able to model them accurately. The difficulty of developing accurate models of continuous-parameter sequences is addressed by investigating a method of parameter estimation which is designed to cope with inaccurate modeling assumptions.     

基于语义向量与OCSVM的工控网络异常行为识别

为克服基于漏洞库等传统安全防护策略的短板,实现对未知攻击行为的识别和预警.使用时间窗划分和深度包检测技术,将端到端的通信内容转化为控制行为序列.根据工控协议的语义特性,采用语义向量模型将行为序列转化为统一维度的特征向量.基于单类支持向量机（OCSVM）仅使用正常行为样本构造的异常识别模型,克服了无法从生产环境中获得异常样本的困难.对于所仿真出的多种异常行为序列,模型识别的平均准确率能够达到93%以上.     

Unsupervised Learning of Multiple Motifs in Biopolymers Using Expectation Maximization

The MEME algorithm extends the expectation maximization (EM) algorithm for identifying motifs in unaligned biopolymer sequences. The aim of MEME is to discover new motifs in a set of biopolymer sequences where little or nothing is known in advance about any motifs that may be present. MEME innovations expand the range of problems which can be solved using EM and increase the chance of finding good solutions. First, subsequences which actually occur in the biopolymer sequences are used as starting points for the EM algorithm to increase the probability of finding globally optimal motifs. Second, the assumption that each sequence contains exactly one occurrence of the shared motif is removed. This allows multiple appearances of a motif to occur in any sequence and permits the algorithm to ignore sequences with no appearance of the shared motif, increasing its resistance to noisy data. Third, a method for probabilistically erasing shared motifs after they are found is incorporated so that several distinct motifs can be found in the same set of sequences, both when different motifs appear in different sequences and when a single sequence may contain multiple motifs. Experiments show that MEME can discover both the CRP and LexA binding sites from a set of sequences which contain one or both sites, and that MEME can discover both the –10 and –35 promoter regions in a set of E. coli sequences.     

Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker

Sign language (SL), which is a highly visual–spatial, linguistically complete, and natural language, is the main mode of communication among deaf people. Described in this paper are two different American Sign Language (ASL) word recognition systems developed using artificial neural networks (ANN) to translate the ASL words into English. Feature vectors of signing words taken at five time instants were used in the first system, while histograms of feature vectors of signing words were used in the second system. The systems use a sensory glove, Cyberglove™, and a Flock of Birds^® 3-D motion tracker to extract the gesture features. The finger joint angle data obtained from strain gauges in the sensory glove define the hand shape, and the data from the tracker describe the trajectory of hand movement. In both systems, the data from these devices were processed by two neural networks: a velocity network and a word recognition network. The velocity network uses hand speed to determine the duration of words. Signs are defined by feature vectors such as hand shape, hand location, orientation, movement, bounding box, and distance. The second network was used as a classifier to convert ASL signs into words based on features or histograms of these features. We trained and tested our ANN models with 60 ASL words for a different number of samples. These methods were compared with each other. Our test results show that the accuracy of recognition of these two systems is 92% and 95%, respectively.     

The identification of nonlinear models for process control using tailored “plant-friendly” input sequences

This paper considers certain practical aspects of the identification of nonlinear empirical models for chemical process dynamics. The primary focus is the identification of second-order Volterra models using input sequences that offer the following three advantages: (1) they are “plant friendly;” (2) they simplify the required computations; (3) they can emphasize certain model parameters over others. To provide a quantitative basis for discussing the first of these advantages, this paper defines a friendliness index f that relates to the number of changes that occur in the sequence. For convenience, this paper also considers an additional nonlinear model structure: the Volterra–Laguerre model. To illustrate the practical utility of the input sequences considered here, second-order Volterra and Volterra–Laguerre models are developed that approximate the dynamics of a first-principles model of methyl methacrylate polymerization.     

Counter-Propagation Neural Networks for Molecular Sequence Classification: Supervised LVQ and Dynamic Node Allocation

A modified counter-propagation (CP) algorithm with supervised learning vector quantizer (LVQ) and dynamic node allocation has been developed for rapid classification of molecular sequences. The molecular sequences were encoded into neural input vectors using an n–gram hashing method for word extraction and a singular value decomposition (SVD) method for vector compression. The neural networks used were three-layered, forward-only CP networks that performed nearest neighbor classification. Several factors affecting the CP performance were evaluated, including weight initialization, Kohonen layer dimensioning, winner selection and weight update mechanisms. The performance of the modified CP network was compared with the back-propagation (BP) neural network and the k–nearest neighbor method. The major advantages of the CP network are its training and classification speed and its capability to extract statistical properties of the input data. The combined BP and CP networks can classify nucleic acid or protein sequences with a close to 100% accuracy at a rate of about one order of magnitude faster than other currently available methods.     

基于拉普拉斯脸和隐马尔可夫的视频人脸识别

提出了一种基于拉普拉斯脸和隐马尔可夫模型的视频人脸识别方法。在训练过程中，采用拉普拉斯脸方法将每一视频序列中的人脸图像映射到拉普拉斯空间，将降维后的特征作为观测值，通过隐马尔可夫模型得到每一训练视频的统计特性和时间动态特性。在识别过程中，用每一个训练视频的隐马尔可夫模型来分析测试视频的时间动态特性，计算出每一训练模型产生该序列的概率，概率最大值所对应的模型就是待识别序列所属的类别。实验结果表明，该方法能够很好地进行视频人脸识别。     

Exploring alternatives for transition verification

When an implementation under test (IUT) is state-based, and its expected abstract behavior is given in terms of a finite state machine (FSM), a checking sequence generated from a specification FSM and applied to an IUT for testing can provide us with high-level confidence in the correct functional behavior of our implementation. One of the issues here is to generate efficient checking sequences in terms of their lengths. As a major characteristics, a checking sequence must contain all β-sequences for transition verification. In this paper, we discuss the possibility of reducing the lengths of checking sequences by making use of the invertible transitions in the specification FSM to increase the choice of β-sequences to be considered for checking sequence generation. We present a sufficient condition for adopting alternative β-sequences and illustrate typical ways of incorporating these alternative β-sequences into existing methods for checking sequence generation to reduce the lengths. Compared to the direct use of three existing methods, our experiments show that most of the time the saving gained by adopting alternative β-sequences falls in the range of 10–40%.     

带有关节权重的DTW 动作识别算法研究

大多数动作仅包含部分关节的运动,现有方法未对运动剧烈的关节与几乎不参与运 动的关节进行区分,一定程度上降低了动作识别精度。针对这个问题,提出一种自适应关节权重 计算方法。结合动态时间规整(DTW)方法,利用获得的关节权重进行动作识别。首先对分类动作 序列进行分段,每段动作序列中运动较剧烈的关节选择分配更高权重,其余关节平均分配权重; 然后提取特征向量,计算两段动作序列的DTW 距离;最后采用K 近邻方法进行动作识别。实验 结果表明,该算法的总体分类识别准确率较高,且对于较相似的动作也能获得较好的识别结果。     

B-spline neural network design using improved differential evolution for identification of an experimental nonlinear process

B-Spline Neural Network (BSNN), a type of basis function neural network, is trained by gradient-based methods which may fall into local minima during the learning procedure. To overcome the limitations encountered by gradient-based optimization methods, we propose differential evolution (DE) – an evolutionary computation methodology – which can provide a stochastic search to adjust the control points of a BSNN. In this paper, we propose six DE approaches using chaotic sequences based on logistic mapping to train a BSNN. Chaos describes the complex behavior of a nonlinear deterministic system. The application of chaotic sequences instead of random sequences in DE is a powerful strategy to diversify the DE population and improve the DE's performance in preventing premature convergence to local minima. The numerical results presented here indicate that chaotic DE was effective for building a good BSNN model for the nonlinear identification of an experimental nonlinear yo–yo motion control system.     

A Temporal Logic for Proving Properties of Topologically General Executions

We present a generalization of the temporal propositional logic of linear time which is useful for stating and proving properties of the generic execution sequence of a parallel program or a non-deterministic program. The formal system we present is exactly that same as the third of three logics presented by Lehmann and Shelah (Information and Control53, 165–198 (1982)), but we give it a different semantics. The models are tree models of arbitrary size similar to those used in branching time temporal logic. The formulation we use allows us to state properties of the “co-meagre” family of paths, where the term “co-meagre” refers to a set whose complement is of the first category in Baire's classification looking at the set of paths in the model as a metric space. Our system is decidable, sound, and, complete for models of arbitrary size, but it has the finite model property; namely, every sentence having a model has a finite model.     

Image Registration Using Wavelet-Based Motion Model

An image registration algorithm is developed to estimate dense motion vectors between two images using the coarse-to-fine wavelet-based motion model. This motion model is described by a linear combination of hierarchical basis functions proposed by Cai and Wang (SIAM Numer. Anal., 33(3):937–970, 1996). The coarser-scale basis function has larger support while the finer-scale basis function has smaller support. With these variable supports in full resolution, the basis functions serve as large-to-small windows so that the global and local information can be incorporated concurrently for image matching, especially for recovering motion vectors containing large displacements. To evaluate the accuracy of the wavelet-based method, two sets of test images were experimented using both the wavelet-based method and a leading pyramid spline-based method by Szeliski et al. (International Journal of Computer Vision, 22(3):199–218, 1996). One set of test images, taken from Barron et al. (International Journal of Computer Vision, 12:43–77, 1994), contains small displacements. The other set exhibits low texture or spatial aliasing after image blurring and contains large displacements. The experimental results showed that our wavelet-based method produced better motion estimates with error distributions having a smaller mean and smaller standard deviation.     

Recognition of loop parallelisms by simulated execution

Conclusions The method of recognition of loop parallelisms based on simulation of loop execution has been described, for the sake of simplicity, within the scope of simple loop structure analysis. At the same time, the method can be effectively extended to loops of arbitrary structures including loop nests. The method allows to vary the number of analyzed passes from N₁ to N₁×N₂×...×N_j, where N_i is the number of iterations of a loop of an imbedding i. Reduction of a nest of loops to a loop of the form (1) can be carried out as follows.With a minimum size of the analyzed passes, internal loops (if necessary) are treated as being unwound into a linear sequence; with a maximum size of passes, the heading of the nest of loops is represented as DO KI=J, N, M, where K, I, J, N, and M are vectors, and a run through the values of vector I is simulated.For parallel programs executed in MINIMAX-type systems, branches are implemented in individual elementary machines. In allocating loop passes to different branches it is necessary to minimize exchange interactions taking place between machines. The simulation method makes it possible in such cases to disclose information allowing to make rational decisions.Translated from Kibernetika, No. 3, pp. 28–33, May–June, 1981.     

Statistical Recognition of Multivariate Non-Gaussian Patterns

Statistical recognition of multivariate patterns is investigated for the case in which the attribute vectors of patterns have non-Gaussian distributions and only the moments of these distributions are known. A recognition approach based on the approximation of pattern distributions by Gram–Charlier series and use of the Bayes decision rule is developed. New decision rules are designed. A computer-aided modeling experiment is described.     

An undecidable problem for countable markov chains

A nondeterministic defense system is considered for a linearly ordered sequences of nodes of a countable Markov chain with an environment. We prove undecidability of the problem of existence of a sequence of signals from attacking subsystems that render the central nodes completely defenseless.Translated from Kibernetika, No. 2, pp. 1–6, 15, March–April, 1991.