Paraphrastic language models

Natural languages are known for their expressive richness. Many sentences can be used to represent the same underlying meaning. Only modelling the observed surface word sequence can result in poor context coverage and generalization, for example, when using n-gram language models (LMs). This paper proposes a novel form of language model, the paraphrastic LM, that addresses these issues. A phrase level paraphrase model statistically learned from standard text data with no semantic annotation is used to generate multiple paraphrase variants. LM probabilities are then estimated by maximizing their marginal probability. Multi-level language models estimated at both the word level and the phrase level are combined. An efficient weighted finite state transducer (WFST) based paraphrase generation approach is also presented. Significant error rate reductions of 0.5–0.6% absolute were obtained over the baseline n-gram LMs on two state-of-the-art recognition tasks for English conversational telephone speech and Mandarin Chinese broadcast speech using a paraphrastic multi-level LM modelling both word and phrase sequences. When it is further combined with word and phrase level feed-forward neural network LMs, a significant error rate reduction of 0.9% absolute (9% relative) and 0.5% absolute (5% relative) were obtained over the baseline n-gram and neural network LMs respectively.     

Fuzzy models of language structures

Statistical distribution of language structures reflect important regularities controlling informational and psycho-physiological processes, which accompany the generation of verbal language or printed texts. In this paper, fuzzy quantitative models of language statistics are constructed. The suggested models are based on the assumption about a super-position of two kinds of uncertainties: probabilistic and possibilistic. The realization of this super-position in statistical distributions is achieved by the splitting procedure of the probability measure. In this way, the fuzzy versions of generalized binomial, Fucks', and Zipf-Mandelbrot's distributions are constructed describing the probabilistic and possibilistic organization of language at any level: morphological, syntactic, or phonological.     

A definition of measures over language space

As an attempt to associate a real number with a language, entropies of languages are computed by Banerji, Kuich, and others. As mappings from languages to real numbers, in this paper, measures over languages are presented. These measures satisfy additivity while entropies do not. Two kinds of measures, p-measure and ω-measure, are defined, and the computing method of these measures is shown for regular languages and context-free languages. Some properties of these measures are applied to show the nonregularity of several languages.     

SPARDL模型的Event-B解释

针对由周期行为和模式转换机制组成的实时系统提出的SPARDL需求建模语言,详细阐明了其对应的SPARDL模型的Event-B解释。通过Event-B来解释SPARDL的语义,同时提出一种基于SPARDL模型特征的精化框架用于Event-B模型的开发。最后,通过案例研究的分析展示用Event-B对SPARDL模型建模和验证的方法的有效性。     

A linguistic ontology of space for natural language processing

We present a detailed semantics for linguistic spatial expressions supportive of computational processing that draws substantially on the principles and tools of ontological engineering and formal ontology. We cover language concerned with space, actions in space and spatial relationships and develop an ontological organization that relates such expressions to general classes of fixed semantic import. The result is given as an extension of a linguistic ontology, the Generalized Upper Model, an organization which has been used for over a decade in natural language processing applications. We describe the general nature and features of this ontology and show how we have extended it for working particularly with space. Treaitng the semantics of natural language expressions concerning space in this way offers a substantial simplification of the general problem of relating natural spatial language to its contextualized interpretation. Example specifications based on natural language examples are presented, as well as an evaluation of the ontology's coverage, consistency, predictive power, and applicability.     

Somun: entity-centric summarization incorporating pre-trained language models

Neural Computing and Applications - Text summarization resolves the issue of capturing essential information from a large volume of text data. Existing methods either depend on the end-to-end...     

Neural network language models for off-line handwriting recognition

Unconstrained off-line continuous handwritten text recognition is a very challenging task which has been recently addressed by different promising techniques. This work presents our latest contribution to this task, integrating neural network language models in the decoding process of three state-of-the-art systems: one based on bidirectional recurrent neural networks, another based on hybrid hidden Markov models and, finally, a combination of both. Experimental results obtained on the IAM off-line database demonstrate that consistent word error rate reductions can be achieved with neural network language models when compared with statistical N-gram language models on the three tested systems. The best word error rate, 16.1%, reported with ROVER combination of systems using neural network language models significantly outperforms current benchmark results for the IAM database.     

Continuous speech recognition using linear dynamic models

Hidden Markov models (HMMs) with Gaussian mixture distributions rely on an assumption that speech features are temporally uncorrelated, and often assume a diagonal covariance matrix where correlations between feature vectors for adjacent frames are ignored. A Linear Dynamic Model (LDM) is a Markovian state-space model that also relies on hidden state modeling, but explicitly models the evolution of these hidden states using an autoregressive process. An LDM is capable of modeling higher order statistics and can exploit correlations of features in an efficient and parsimonious manner. In this paper, we present a hybrid LDM/HMM decoder architecture that postprocesses segmentations derived from the first pass of an HMM-based recognition. This smoothed trajectory model is complementary to existing HMM systems. An Expectation-Maximization (EM) approach for parameter estimation is presented. We demonstrate a 13 % relative WER reduction on the Aurora-4 clean evaluation set, and a 13 % relative WER reduction on the babble noise condition.     

Matching natural language patterns to graphical system models

We outline an approach to parsing based on system modelling. The underlying assumption, which determines the limits of the approach, is that a narrative natural language text constitutes a symbolic model of the system described, written for the purpose of communicating static and/or dynamic system aspects.     

Personalized text snippet extraction using statistical language models

In knowledge discovery in a text database, extracting and returning a subset of information highly relevant to a user's query is a critical task. In a broader sense, this is essentially identification of certain personalized patterns that drives such applications as Web search engine construction, customized text summarization and automated question answering. A related problem of text snippet extraction has been previously studied in information retrieval. In these studies, common strategies for extracting and presenting text snippets to meet user needs either process document fragments that have been delimitated a priori or use a sliding window of a fixed size to highlight the results. In this work, we argue that text snippet extraction can be generalized if the user's intention is better utilized. It overcomes the rigidness of existing approaches by dynamically returning more flexible start-end positions of text snippets, which are also semantically more coherent. This is achieved by constructing and using statistical language models which effectively capture the commonalities between a document and the user intention. Experiments indicate that our proposed solutions provide effective personalized information extraction services.     

Creating word-level language models for large-vocabulary handwriting recognition

We discuss development of a word-unigram language model for online handwriting recognition. First, we tokenize a text corpus into words, contrasting with tokenization methods designed for other purposes. Second, we select for our model a subset of the words found, discussing deviations from an N-most-frequent-words approach. From a 600-million-word corpus, we generated a 53,000-word model which eliminates 45% of word-recognition errors made by a character-level-model baseline system. We anticipate that our methods will be applicable to offline recognition as well, and to some extent to other recognizers, such as speech recognizers and video retrieval systems. Received: November 1, 2001 / Revised version: July 22, 2002     

Postprocessing statistical language models for handwritten Chinesecharacter recognizer

Two statistical language models have been investigated on their effectiveness in upgrading the accuracy of a Chinese character recognizer. The baseline model is one of lexical analytic nature which segments a sequence of character images according to the maximum matching of words with consideration of word binding forces. A model of bigram statistics of word-classes is then investigated and compared against the baseline model in terms of recognition rate improvement on the image recognizer. On the average, the baseline language model improves the recognition rate by about 7% while the bigram statistics model upgrades it by about 10%     

Phrase classes in two-level language models for ASR

In this work, we propose and compare two different approaches to a two-level language model. Both of them are based on phrase classes but they consider different ways of dealing with phrases into the classes. We provide a complete formulation consistent with the two approaches. The language models proposed were integrated into an Automatic Speech Recognition (ASR) system and evaluated in terms of Word Error Rate. Several series of experiments were carried out over a spontaneous human–machine dialogue corpus in Spanish, where users asked for information about long-distance trains by telephone. It can be extracted from the obtained results that the integration of phrases into classes when using the language models proposed leads to an improvement of the performance of an ASR system. Moreover, the obtained results seem to indicate that the history length with which the best performance is achieved is related to the features of the model itself. Thus, not all the models show the best results with the same value of history length.     

Continuous speech recognition and syntactic processing in Iranian Farsi language

In this paper, the architecture of the first Iranian Farsi continuous speech recognizer and syntactic processor is introduced. In this system, by extracting suitable features of speech signal (cepstral, delta-cepstral, energy and zero-crossing rate) and using a hydrid architecture of neural networks (a Self-Organizing Feature Map, SOFM, at the first stage and a Multi-Layer Perceptron, MLP, at the second stage) the Iranian Farsi phonemes are recognized. Then the string of phonemes are corrected, segmented and converted to formal text by using a non-stochastic method. For syntactic processing, the symbolic (by using artificial intelligence techniques) and connectionist (by using artificial neural networks) approaches are used to determine the correctness, position and the kind of syntactic errors in Iranian Farsi sentences, as well.     

State space models of remote manipulation tasks

A state variable formulation of the remote manipulation problem is presented, applicable to human-supervised or autonomous computer-manipulators. A discrete state vector, containing position variables for the manipulator and relevant objects, spans a quantized state space comprising many static configurations of objects and hand. A manipulation task is a desired new state. State transitions are assigned costs and are accomplished by commands: hand motions plus grasp, release, push, twist, etc. In control theory terms the problem is to find the cheapest control history (if any) from present to desired state. A method similar to dynamic programming is used to determine the optimal history. The system is capable of obstacle avoidance, grasp rendezvous, incorporation of new sensor data, remembering results of previous tasks, and so on.     

Robust stability in linear state space models

A new sufficient condition is presented for the stability of interval matrices based on Kharitonov's theorem (1978).