A CMOS neural network for pattern association

The authors present an analog complementary metal-oxide semiconductor (CMOS) version of a model for pattern association, along with discussions of design philosophy, electrical results, and a chip architecture for a 512-element, feed-forward IC. They discuss hardware implementations of neural networks and the effect of limited interconnections. They then examine network design, processor-element design, and system operation     

Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network

This paper investigates the prediction of a Lorenz chaotic attractor having relatively high values of Lypunov's exponents. The characteristic of this time series is its rich chaotic behavior. For such dynamic reconstruction problem, regularized radial basis function (RBF) neural network (NN) models have been widely employed in the literature. However, author recommends using a two-layer multi-layer perceptron (MLP) NN-based recurrent model. When none of the available linear models have been able to learn the dynamics of this attractor, it is shown that the proposed NN-based auto regressive (AR) and auto regressive moving average (ARMA) models with regularization have not only learned the true trajectory of this attractor, but also performed much better in multi-step-ahead predictions. However, equivalent linear models seem to fail miserably in learning the dynamics of the time series, despite the low values of Akaike's final prediction error (FPE) estimate. Author proposes to employ the recurrent NN-based ARMA model with regularization which clearly outperforms all other models and thus, it is possible to obtain good results for prediction and reconstruction of the dynamics of the chaotic time series with NN-based models.     

Support-vector-based fuzzy neural network for pattern classification

Fuzzy neural networks (FNNs) for pattern classification usually use the backpropagation or C-cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data. However, such kinds of learning algorithms usually cannot minimize the empirical risk (training error) and expected risk (testing error) simultaneously, and thus cannot reach a good classification performance in the testing phase. To tackle this drawback, a support-vector-based fuzzy neural network (SVFNN) is proposed for pattern classification in this paper. The SVFNN combines the superior classification power of support vector machine (SVM) in high dimensional data spaces and the efficient human-like reasoning of FNN in handling uncertainty information. A learning algorithm consisting of three learning phases is developed to construct the SVFNN and train its parameters. In the first phase, the fuzzy rules and membership functions are automatically determined by the clustering principle. In the second phase, the parameters of FNN are calculated by the SVM with the proposed adaptive fuzzy kernel function. In the third phase, the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method. To investigate the effectiveness of the proposed SVFNN classification, it is applied to the Iris, Vehicle, Dna, Satimage, Ijcnn1 datasets from the UCI Repository, Statlog collection and IJCNN challenge 2001, respectively. Experimental results show that the proposed SVFNN for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions.     

A neural network for visual pattern recognition

A model of a neural network is presented that offers insight into the brain's complex mechanisms as well as design principles for information processors. The model has properties and abilities that most modern computers and pattern recognizers do not possess; pattern recognition, selective attention, segmentation, and associative recall. When a composite stimulus consisting of two or more patterns is presented, the model pays selective attention to each of the patterns one after the other, segments a pattern from the rest, and recognizes it separately in contrast to earlier models. This model has perfect associative recall, even for deformed patterns, without regard to their positions. It can be trained to recognize any set of patterns     

Recurrent neural network with attention mechanism for language model

Neural Computing and Applications - The rapid growth of the Internet promotes the growth of textual data, and people get the information they need from the amount of textual data to solve problems....     

Recurrent neural network for combined economic and emission dispatch

Recently, the combined economic and emission dispatch (CEED) problem, which aims to simultaneously decrease fuel cost and reduce environmental emissions of power systems, has been a widespread concern. To improve the utilization efficiency of primary energy, combined heat and power (CHP) units are likely to play an important role in the future. The goal of this study is to propose an approach to solve the CEED problems in a CHP system which consists of eight power generators (PGs), two CHP units and one heat only unit. Owing to the existence of power loss in power transmission line and the non-convex feasible region of CHP units, the proposed problem is a nonlinear, multi-constraints, non-convex multi-objectives (MO) optimization problem. To deal with it, a recurrent neural network (RNN) combined with a novel technique is developed. It means that the feasible region is separated into two convex regions by using two binary variables to search for different regions. In the frame of the neurodynamic optimization, existence and convergence of the dynamic model are analyzed. It shows that the convergence solution obtained by RNN is the optimal solution of CEED problem. Numerical simulation results show that the proposed algorithm can generate solutions efficiently.     

A neural network for linear matrix inequality problems

Gradient-type Hopfield networks have been widely used in optimization problems solving. The paper presents a novel application by developing a matrix oriented gradient approach to solve a class of linear matrix inequalities (LMIs), which are commonly encountered in the robust control system analysis and design. The solution process is parallel and distributed in neural computation. The proposed networks are proven to be stable in the large. Representative LMIs such as generalized Lyapunov matrix inequalities, simultaneous Lyapunov matrix inequalities, and algebraic Riccati matrix inequalities are considered. Several examples are provided to demonstrate the proposed results. To verify the proposed control scheme in real-time applications, a high-speed digital signal processor is used to emulate the neural-net-based control scheme.     

Hypothetical neural mechanism that may play a role in mental rotation: an attractor neural network model

We propose an attractor neural network (ANN) model that performs rotation-invariant pattern recognition in such a way that it can account for a neural mechanism being involved in the image transformation accompanying the experience of mental rotation. We compared the performance of our ANN model with the results of the chronometric psychophysical experiments of Cooper and Shepard (Cooper L A and Shepard R N 1973 Visual Information Processing (New York: Academic) pp 204-7) on discrimination of alphanumeric characters presented in various angular departures from their canonical upright position. Comparing the times required for pattern retrieval in its canonical upright position with the reaction times of human subjects, we found agreement in that (i) retrieval times for clockwise and anticlockwise departures of the same angular magnitude (up to 180 degrees) were not different, (ii) retrieval times increased with departure from upright and (iii) increased more sharply as departure from upright approached 180 degrees. The rotation-invariant retrieval of the activity pattern has been accomplished by means of the modified algorithm of Dotsenko (Dotsenko V S 1988 J. Phys. A: Math. Gen. 21 L783-7) proposed for translation-, rotation- and size-invariant pattern recognition, which uses relaxation of neuronal firing thresholds to guide the evolution of the ANN in state space towards the desired memory attractor. The dynamics of neuronal relaxation has been modified for storage and retrieval of low-activity patterns and the original gradient optimization of threshold dynamics has been replaced with optimization by simulated annealing.     

Recurrent neural network training with feedforward complexity

This paper presents a training method that is of no more than feedforward complexity for fully recurrent networks. The method is not approximate, but rather depends on an exact transformation that reveals an embedded feedforward structure in every recurrent network. It turns out that given any unambiguous training data set, such as samples of the state variables and their derivatives, we need only to train this embedded feedforward structure. The necessary recurrent network parameters are then obtained by an inverse transformation that consists only of linear operators. As an example of modeling a representative nonlinear dynamical system, the method is applied to learn Bessel's differential equation, thereby generating Bessel functions within, as well as outside the training set.     

Recurrent neural network approach for cyclic job shop scheduling problem

While cyclic scheduling is involved in numerous real-world applications, solving the derived problem is still of exponential complexity. This paper focuses specifically on modelling the manufacturing application as a cyclic job shop problem and we have developed an efficient neural network approach to minimise the cycle time of a schedule. Our approach introduces an interesting model for a manufacturing production, and it is also very efficient, adaptive and flexible enough to work with other techniques. Experimental results validated the approach and confirmed our hypotheses about the system model and the efficiency of neural networks for such a class of problems.     

A neural network model for monotone linear asymmetric variationalinequalities

A linear variational inequality is a uniform approach for some important problems in optimization and equilibrium problems. We give a neural network model for solving asymmetric linear variational inequalities. The model is based on a simple projection and contraction method. Computer simulation is performed for linear programming (LP) and linear complementarity problems (LCP). The test results for the LP problem demonstrate that our model converges significantly faster than the three existing neural network models examined in a comparative study paper.     

A higher-order neural network for distortion invariant pattern recognition

Recently, it is shown that a single layer, higher-order neural network is effective for scale, rotation and shift invariance and in the training process it requires only one example for one category and a very small number of iterations. However, there are problems that scale invariance doesn't hold precisely and it is not so effective for distortion of unknown patterns. In this paper we present an idea to realize the scale invariance precisely and suggest a method that is available to distorted patterns. The experimental results are presented to show the feasibility of our approach.     

The self-trapping attractor neural network. I. Analysis of a simple 1-D model

Attractor neural networks (ANNs) based on the Ising model are naturally fully connected and are homogeneous in structure. These features permit a deep understanding of the underlying mechanism, but limit the applicability of these models to the brain. A more biologically realistic model can be derived from an equally simple physical model by utilizing recurrent self-trapping inputs to supplement very sparse intranetwork interactions. This paper reports the analysis of a one-dimensional (1-D) ANN coupled to a second system that computes overlaps with a single stored memory. Results show that: 1) the 1-D self-trapping model is equivalent to an isolated ANN with both full connectivity of one strength and nearest neighbor synapses of an independent strength; 2) the dynamics of ANN and self-trapping updates are independent; 3) there is a critical synaptic noise level below which memory retrieval occurs; 4) the 1-D self-trapping model converges to a fully connected Hopfield model for zero strength nearest neighbor synapses, and has a greater magnitude memory overlap for nonzero strength nearest neighbor synapses; and (5) the mechanism of self-trapping is an iterative map on the mean overlap as a function of the reentrant input.     

Ordinal preference and inter-rater pattern recognition: Hopfield neural network vs. measures of association

In decision analysis, if the criterion is an ordinal rather than a cardinal one, a preferential solution depends on the inter-rater agreement. The Kendall coefficient of concordance W, the Friedman ranks statistic F _r , and the Page L statistic are often used to determine the association among M sets of rankings. However, they may get some anomalies because they all use the cardinal variable “variance” to judge the association. In order to correct the anomalies, we use the modified Hopfield neural network instead to determine the association among M sets of rankings. The results are not only to reduce the unidentified cases but also to solve the anomalies.     

Blind equalization of a noisy channel by linear neural network

In this paper, a new neural approach is introduced for the problem of blind equalization in digital communications. Necessary and sufficient conditions for blind equalization are proposed, which can be implemented by a two-layer linear neural network, in the hidden layer, the received signals are whitened, while the network outputs provide directly an estimation of the source symbols. We consider a stochastic approximate learning algorithm for each layer according to the property of the correlation matrices of the transmitted symbols. The proposed class of networks yield good results in simulation examples for the blind equalization of a three-ray multipath channel.     

A neural network approach for solving linear bilevel programming problem

A novel neural network approach is proposed for solving linear bilevel programming problem. The proposed neural network is proved to be Lyapunov stable and capable of generating optimal solution to the linear bilevel programming problem. The numerical result shows that the neural network approach is feasible and efficient.     

Cooperative coevolution of artificial neural network ensembles for pattern classification

This paper presents a cooperative coevolutive approach for designing neural network ensembles. Cooperative coevolution is a recent paradigm in evolutionary computation that allows the effective modeling of cooperative environments. Although theoretically, a single neural network with a sufficient number of neurons in the hidden layer would suffice to solve any problem, in practice many real-world problems are too hard to construct the appropriate network that solve them. In such problems, neural network ensembles are a successful alternative. Nevertheless, the design of neural network ensembles is a complex task. In this paper, we propose a general framework for designing neural network ensembles by means of cooperative coevolution. The proposed model has two main objectives: first, the improvement of the combination of the trained individual networks; second, the cooperative evolution of such networks, encouraging collaboration among them, instead of a separate training of each network. In order to favor the cooperation of the networks, each network is evaluated throughout the evolutionary process using a multiobjective method. For each network, different objectives are defined, considering not only its performance in the given problem, but also its cooperation with the rest of the networks. In addition, a population of ensembles is evolved, improving the combination of networks and obtaining subsets of networks to form ensembles that perform better than the combination of all the evolved networks. The proposed model is applied to ten real-world classification problems of a very different nature from the UCI machine learning repository and proben1 benchmark set. In all of them the performance of the model is better than the performance of standard ensembles in terms of generalization error. Moreover, the size of the obtained ensembles is also smaller.     

A high-performance neural network for solving linear and quadraticprogramming problems

Two classes of high-performance neural networks for solving linear and quadratic programming problems are given. We prove that the new system converges globally to the solutions of the linear and quadratic programming problems. In a neural network, network parameters are usually not specified. The proposed models can overcome numerical difficulty caused by neural networks with network parameters and obtain desired approximate solutions of the linear and quadratic programming problems.     

Robust neural network control system design for linear ultrasonic motor

Linear ultrasonic motor (LUSM) has much merit, such as high precision, fast control dynamics and large driving force, etc.; however, the dynamic characteristic of LUSM is nonlinear and the precise dynamic model of LUSM is difficult to obtain. To tackle this problem, this study presents a robust neural network control (RNNC) system for LUSM to track a reference trajectory with L ₂ robust tracking performance. The developed RNNC system is composed of a neural network controller and a robust controller. The neural network controller is the principal controller used to mimic an ideal controller and the robust controller is adopted to achieve L ₂ robust tracking performance. The developed RNNC system is then applied to control an LUSM. Experimental results show that the developed RNNC system can achieve favorable tracking performance with unknown of LUSM model.