A novel objective function for improved phoneme recognition usingtime-delay neural networks

Single-speaker and multispeaker recognition results are presented for the voice-stop consonants /b,d,g/ using time-delay neural networks (TDNNs) with a number of enhancements, including a new objective function for training these networks. The new objective function, called the classification figure of merit (CFM), differs markedly from the traditional mean-squared-error (MSE) objective function and the related cross entropy (CE) objective function. Where the MSE and CE objective functions seek to minimize the difference between each output node and its ideal activation, the CFM function seeks to maximize the difference between the output activation of the node representing incorrect classifications. A simple arbitration mechanism is used with all three objective functions to achieve a median 30% reduction in the number of misclassifications when compared to TDNNs trained with the traditional MSE back-propagation objective function alone.     

A new approach to artificial neural networks

A novel approach to artificial neural networks is presented. The philosophy of this approach is based on two aspects: the design of task-specific networks, and a new neuron model with multiple synapses. The synapses' connective strengths are modified through selective and cumulative processes conducted by axo-axonic connections from a feedforward circuit. This new concept was applied to the position control of a planar two-link manipulator exhibiting excellent results on learning capability and generalization when compared with a conventional feedforward network. In the present paper, the example shows only a network developed from a neuronal reflexive circuit with some useful artifices, nevertheless without the intention of covering all possibilities devised.     

A new approach for automated parts recognition using time series analysisand neural networks

This paper presents a new approach for automated parts recognition. It is based on the use of the signature and autocorrelation functions for feature extraction and a neural network for the analysis of recognition. The signature represents the shapes of boundaries detected in digitized binary images of the parts. The autocorrelation coefficients computed from the signature are invariant to transformations such as scaling, translation and rotation of the parts. These unique extracted features are fed to the neural network. A multilayer perceptron with two hidden layers, along with a backpropagation learning algorithm, is used as a pattern classifier. In addition, the position information of the part for a robot with a vision system is described to permit grasping and pick-up. Experimental results indicate that the proposed approach is appropriate for the accurate and fast recognition and inspection of parts in automated manufacturing systems.     

A segmental non-parametric-based phoneme recognition approach at the acoustical level

Although Hidden Markov Models (HMMs) are still the mainstream approach towards speech recognition, their intrinsic limitations such as first-order Markov models in use or the assumption of independent and identically distributed frames lead to the extensive use of higher level linguistic information to produce satisfactory results. Therefore, researchers began investigating the incorporation of various discriminative techniques at the acoustical level to induce more discrimination between speech units. As is known, the k-nearest neighbour (k-NN) density estimation is discriminant by nature and is widely used in the pattern recognition field. However, its application to speech recognition has been limited to few experiments. In this paper, we introduce a new segmental k-NN-based phoneme recognition technique. In this approach, a group-delay-based method generates phoneme boundary hypotheses, and an approximate version of k-NN density estimation is used for the classification and scoring of variable-length segments. During the decoding, the construction of the phonetic graph starts from the best phoneme boundary setting and progresses through splitting and merging segments using the remaining boundary hypotheses and constraints such as phoneme duration and broad-class similarity information. To perform the k-NN search, we take advantage of a similarity search algorithm called Spatial Approximate Sample Hierarchy (SASH). One major advantage of the SASH algorithm is that its computational complexity is independent of the dimensionality of the data. This allows us to use high-dimensional feature vectors to represent phonemes. By using phonemes as units of speech, the search space is very limited and the decoding process fast. Evaluation of the proposed algorithm with the sole use of the best hypothesis for every segment and excluding phoneme transitional probabilities, context-based, and language model information results in an accuracy of 58.5% with correctness of 67.8% on the TIMIT test dataset.     

A neural networks approach to minority game

The minority game (MG) comes from the so-called “El Farol bar” problem by W.B. Arthur. The underlying idea is competition for limited resources and it can be applied to different fields such as: stock markets, alternative roads between two locations and in general problems in which the players in the “minority” win. Players in this game use a window of the global history for making their decisions, we propose a neural networks approach with learning algorithms in order to determine players strategies. We use three different algorithms to generate the sequence of minority decisions and consider the prediction power of a neural network that uses the Hebbian algorithm. The case of sequences randomly generated is also studied. Research supported by Local Project 2004–2006 (EX 40%) Università di Foggia. A. Sfrecola is a researcher financially supported by Dipartimento di Scienze Economiche, Matematiche e Statistiche, Università degli Studi di Foggia, Foggia, Italy.     

A neural predictive coding feature extraction scheme in DCT domain for phoneme recognition

Nonlinear feature extraction of speech signals has been the main concern of many researches in recent years. In this paper, feature extraction of phonemes using NPC (neural predictive coding) model is generalized to a combination of time and DCT domains. Two main ideas were proposed and evaluated in this paper. First, a frame-wise DCT-based NPC feature extractor is proposed to overcome the computational complexity deficiency of the system. The basis of this approach is the application of a DCT pre-feature extractor to remove unwanted additional data. In this approach, the extracted features are the output of the hidden layer. It is shown that the use of a pre-processing stage can improve both computational complexity efficiency and accuracy issues. At the second approach, we proposed a complementary role for DCT domain features in classic NPC modeling. This approach uses the signal residual of the predicted signal in the DCT domain. The experiments were conducted on voiced plosive phonemes of TIMIT database. Simulations showed that the performance of the combined method is good at the plosive phonemes. The achieved accuracy that was resulted from the proposed method was 70.3% recognition rate on /b/d/g/ phonemes, which is higher than the results of traditional NPC approaches.     

A new approach to applying feedforward neural networks to the prediction of musculoskeletal disorder risk

A new and improved method to feedforward neural network (FNN) development for application to data classification problems, such as the prediction of levels of low-back disorder (LBD) risk associated with industrial jobs, is presented. Background on FNN development for data classification is provided along with discussions of previous research and neighborhood (local) solution search methods for hard combinatorial problems. An analytical study is presented which compared prediction accuracy of a FNN based on an error-back propagation (EBP) algorithm with the accuracy of a FNN developed by considering results of local solution search (simulated annealing) for classifying industrial jobs as posing low or high risk for LBDs. The comparison demonstrated superior performance of the FNN generated using the new method. The architecture of this FNN included fewer input (predictor) variables and hidden neurons than the FNN developed based on the EBP algorithm. Independent variable selection methods and the phenomenon of 'overfitting' in FNN (and statistical model) generation for data classification are discussed. The results are supportive of the use of the new approach to FNN development for applications to musculoskeletal disorders and risk forecasting in other domains.     

A new approach to radial basis function-based polynomial neural networks: analysis and design

In this study, we introduce a new topology of radial basis function-based polynomial neural networks (RPNNs) that is based on a genetically optimized multi-layer perceptron with radial polynomial neurons (RPNs). This paper offers a comprehensive design methodology involving various mechanisms of optimization, especially fuzzy C-means (FCM) clustering and particle swarm optimization (PSO). In contrast to the typical architectures encountered in polynomial neural networks (PNNs), our main objective is to develop a topology and establish a comprehensive design strategy of RPNNs: (a) The architecture of the proposed network consists of radial polynomial neurons (RPN). These neurons are fully reflective of the structure encountered in numeric data, which are granulated with the aid of FCM clustering. RPN dwells on the concepts of a collection of radial basis function and the function-based nonlinear polynomial processing. (b) The PSO-based design procedure being applied to each layer of the RPNN leads to the selection of preferred nodes of the network whose local parameters (such as the number of input variables, a collection of the specific subset of input variables, the order of the polynomial, the number of clusters of FCM clustering, and a fuzzification coefficient of the FCM method) are properly adjusted. The performance of the RPNN is quantified through a series of experiments where we use several modeling benchmarks, namely a synthetic three-dimensional data and learning machine data (computer hardware data, abalone data, MPG data, and Boston housing data) already used in neuro-fuzzy modeling. A comparative analysis shows that the proposed RPNN exhibits higher accuracy in comparison with some previous models available in the literature.     

&thetas;-adaptive neural networks: a new approach to parameterestimation

A novel use of neural networks for parameter estimation in nonlinear systems is proposed. The approximating ability of the neural network is used to identify the relation between system variables and parameters of a dynamic system. Two different algorithms, a block estimation method and a recursive estimation method, are proposed. The block estimation method consists of the training of a neural network to approximate the mapping between the system response and the system parameters which in turn is used to identify the parameters of the nonlinear system. In the second method, the neural network is used to determine a recursive algorithm to update the parameter estimate. Both methods are useful for parameter estimation in systems where either the structure of the nonlinearities present are unknown or when the parameters occur nonlinearly. Analytical conditions under which successful estimation can be carried but and several illustrative examples verifying the behavior of the algorithms through simulations are presented.     

A new approach to solve SLAM challenges by relative map filter

In this paper we propose a new approach to solve some challenges in the simultaneous localization and mapping (SLAM) problem based on the relative map filter (RMF). This method assumes that the relative distances between the landmarks of relative map are estimated fully independently. This considerably reduces the computational complexity to average number of landmarks observed in each scan. To solve the ambiguity that may happen in finding the absolute locations of robot and landmarks, we have proposed two separate methods, the lowest position error (LPE) and minimum variance position estimator (MVPE). Another challenge in RMF is data association problem where we also propose an algorithm which works by using motion sensors without engaging in their cumulative error. To apply these methods, we switch successively between the absolute and relative positions of landmarks. Having a sufficient number of landmarks in the environment, our algorithm estimates the positions of robot and landmarks without using motion sensors and kinematics of robot. Motion sensors are only used for data association. The empirical studies on the proposed RMF-SLAM algorithm with the LPE or MVPE methods show a better accuracy in localization of robot and landmarks in comparison with the absolute map filter SLAM.     

Handwritten digit recognition by neural networks with single-layertraining

It is shown that neural network classifiers with single-layer training can be applied efficiently to complex real-world classification problems such as the recognition of handwritten digits. The STEPNET procedure, which decomposes the problem into simpler subproblems which can be solved by linear separators, is introduced. Provided appropriate data representations and learning rules are used, performance comparable to that obtained by more complex networks can be achieved. Results from two different databases are presented: an European database comprising 8700 isolated digits and a zip code database from the US Postal Service comprising 9000 segmented digits. A hardware implementation of the classifier is briefly described.     

A neural network approach to CSG-based 3-D object recognition

Describes the recognition subsystem of a computer vision system based on constructive solid geometry (CSG) representation scheme. Instead of using the conventional CSG trees to represent objects, the proposed system uses an equivalent representation scheme-precedence graphs-for object representation. Each node in the graph represents a primitive volume and each are between two nodes represents the relation between them. Object recognition is achieved by matching the scene precedence graph to the model precedence graph. A constraint satisfaction network is proposed to implement the matching process. The energy function associated with the network is used to enforce the matching constraints including match validity, primitive similarity, precedence graph preservation, and geometric structure preservation. The energy level is at its minimum only when the optimal match is reached. Experimental results on several range images are presented to demonstrate the proposed approach     

A new approach to recognition of deformed patterns

By introducing a distance between original and deformed patterns and an adequate cost function we derive a gradient dynamics for transformation functions that transform one pattern into another one. In its simplest form the cost function is lent from an elastic medium but in further improvements the corresponding Lamé constants are chosen pattern dependent. In the case of line patterns a smearing out by means of a Gaussian distribution is introduced and made time dependent. The numerical simulations show the feasibility of the present approach, for instance, for the recognition of handwritten characters or of faces showing different facial expressions.     

A Hopfield neural network approach to decentralized self-synchronizing sensor networks

Decentralized inference of a sensor network in the difficult case of a nonreciprocal nonlinear context is investigated by transforming the sensor network into a Hopfield neural network. Equilibrium states of the latter correspond to situations of global consensus in the sensor network, characterized by suitable regions (consensus regions) in the space of its parameters. The said transformation was recently proposed by the author and applied to the simple case of three sensors. The general case of more than three sensors is investigated in the present paper. A procedure is developed for determining the structure and the properties of the consensus regions.     

A reference model approach to stability analysis of neural networks

In this paper, a novel methodology called a reference model approach to stability analysis of neural networks is proposed. The core of the new approach is to study a neural network model with reference to other related models, so that different modeling approaches can be combinatively used and powerfully cross-fertilized. Focused on two representative neural network modeling approaches (the neuron state modeling approach and the local field modeling approach), we establish a rigorous theoretical basis on the feasibility and efficiency of the reference model approach. The new approach has been used to develop a series of new, generic stability theories for various neural network models. These results have been applied to several typical neural network systems including the Hopfield-type neural networks, the recurrent back-propagation neural networks, the BSB-type neural networks, the bound-constraints optimization neural networks, and the cellular neural networks. The results obtained unify, sharpen or generalize most of the existing stability assertions, and illustrate the feasibility and power of the new method.     

A new synthesis approach for feedback neural networks based on theperceptron training algorithm

In this paper, a new synthesis approach is developed for associative memories based on the perceptron training algorithm. The design (synthesis) problem of feedback neural networks for associative memories is formulated as a set of linear inequalities such that the use of perceptron training is evident. The perceptron training in the synthesis algorithms is guaranteed to converge for the design of neural networks without any constraints on the connection matrix. For neural networks with constraints on the diagonal elements of the connection matrix, results concerning the properties of such networks and concerning the existence of such a network design are established. For neural networks with sparsity and/or symmetry constraints on the connection matrix, design algorithms are presented. Applications of the present synthesis approach to the design of associative memories realized by means of other feedback neural network models are studied. To demonstrate the applicability of the present results and to compare the present synthesis approach with existing design methods, specific examples are considered.     

Nonlinear measures: a new approach to exponential stabilityanalysis for Hopfield-type neural networks

In this paper, a new concept called nonlinear measure is introduced to quantify stability of nonlinear systems in the way similar to the matrix measure for stability of linear systems. Based on the new concept, a novel approach for stability analysis of neural networks is developed. With this approach, a series of new sufficient conditions for global and local exponential stability of Hopfield type neural networks is presented, which generalizes those existing results. By means of the introduced nonlinear measure, the exponential convergence rate of the neural networks to stable equilibrium point is estimated, and, for local stability, the attraction region of the stable equilibrium point is characterized. The developed approach can be generalized to stability analysis of other general nonlinear systems.     

High speed paper currency recognition by neural networks.

In this paper a new technique is proposed to improve the recognition ability and the transaction speed to classify the Japanese and US paper currency. Two types of data sets, time series data and Fourier power spectra, are used in this study. In both cases, they are directly used as inputs to the neural network. Furthermore, we also refer a new evaluation method of recognition ability. Meanwhile, a technique is proposed to reduce the input scale of the neural network without preventing the growth of recognition. This technique uses only a subset of the original data set which is obtained using random masks. The recognition ability of using large data set and a reduced data set are discussed. In addition to that the results of using a reduced data set of the Fourier power spectra and the time series data are compared.     

A hierarchical approach to real-time activity recognition in body sensor networks

Real-time activity recognition in body sensor networks is an important and challenging task. In this paper, we propose a real-time, hierarchical model to recognize both simple gestures and complex activities using a wireless body sensor network. In this model, we first use a fast and lightweight algorithm to detect gestures at the sensor node level, and then propose a pattern based real-time algorithm to recognize complex, high-level activities at the portable device level. We evaluate our algorithms over a real-world dataset. The results show that the proposed system not only achieves good performance (an average utility of 0.81, an average accuracy of 82.87%, and an average real-time delay of 5.7 seconds), but also significantly reduces the network’s communication cost by 60.2%.     

A new approach to target recognition for LADAR data

We discuss target detection in LADAR intensity images. Thirteen features, eleven of which come from an asymmetric co-occurrence matrix, are extracted from region-of-interest windows in each image. Two methods of feature selection are applied to the extracted vectors. Random selection leads to a pair of selected features for a nearest-neighbor rule (1-nn) detector. Extended backpropagation leads to six selected features using a modified multilayered perceptron (MLP) network. The 1-nn detector achieves a test-error rate of about 16% at a false-alarm rate of 8%. The MLP has a test-error rate of about 12% with a false-alarm rate of 6%