A practical overview of neural networks

This paper overviews the myths and misconceptions that have surrounded neural networks in recent years. Focusing on backpropagation and the Hopfield network, we discuss the problems that have plagued practical application of these techniques, and review some of the recent progress made. Both real and perceived inadequacies of backpropagation are discussed, as well as the need for an understanding of statistics and of the problem domain in order to apply and assess the neural network properly. We consider alternatives or variants to backpropagation, which overcome some of its real limitations. The Hopfield network's poor performance on the traveling salesman problem in combinatorial optimization has colored its reception by engineers; we describe both new research in this area and promising results in other practical optimization applications. Overall, it is hoped, this paper will aid in a more balanced understanding of neural networks. They seem worthy of consideration in many applications, but they do not deserve the status of a panacea – nor are they as fraught with problems as would now seem to be implied.     

Optimising memory usage in n-tuple neural networks

The use of n-tuple or weightless neural networks as pattern recognition devices is well known (Aleksander and Stonham, 1979). They have some significant advantages over the more common and biologically plausible networks, such as multi-layer perceptrons; for example, n-tuple networks have been used for a variety of tasks, the most popular being real-time pattern recognition, and they can be implemented easily in hardware as they use standard random access memories.

In operation, a series of images of an object are shown to the network, each being processed suitably and effectively stored in a memory called a discriminator. Then, when another image is shown to the system, it is processed in a similar manner and the system reports whether it recognises the image; is the image sufficiently similar to one already taught?

If the system is to be able to recognise and discriminate between m-objects, then it must contain m-discriminators. This can require a great deal of memory.

This paper describes various ways in which memory requirements can be reduced, including a novel method for multiple discriminator n-tuple networks used for pattern recognition. By using this method, the memory normally required to handle m-objects can be used to recognise and discriminate between 2^m — 2 objects.     

Artificial neural networks in process estimation and control

In this contribution, the suitability of the artificial neural network methodology for solving some process engineering problems is discussed. First the concepts involved in the formulation of artificial neural networks are presented. Next the suitability of the technique to provide estimates of difficult to measure quality variables is demonstrated by application to industrial data. Measurements from established instruments are used as secondary variables for estimation of the “primary” quality variables. The advantage of using these estimates for feedback control is then demonstrated. The possibility of using neural network models directly within a model-based predictive control strategy is also considered, making use of an on-line optimization routine to determine the future inputs that will minimize the deviations between the desired and predicted outputs. Control is implemented in a receding horizon fashion. Application of the predictive controller to a nonlinear distillation system is used to indicate the potential of the neural network based control philosophy.     

Applications of neural networks for coordinate transformations in robotics

The use of artificial neural networks is investigated for application to trajectory control problems in robotics. The relative merits of position versus velocity control is considered and a control scheme is proposed in which neural networks are used as static maps (trained off-line) to compute the inverse of the manipulator Jacobian matrix. A proof of the stability of this approach is offered, assuming bounded errors in the static map. A representative two-link robot is investigated using an artificial neural network which has been trained to compute the components of the inverse of the Jacobian matrix. The controller is implemented in the laboratory and its performance compared to a similar controller with the analytical inverse Jacobian matrix.     

Servo controller design using neural networks

The dynamics of a physical plant may be difficult to express as concise mathematical equations. In practice there exist uncertainties that cannot be modeled with the system equations. Hence, robustness against system uncertainties is essential in a control system design. In this article, multilayered neural networks (MNNs) are used to compensate for model uncertainties of a dynamical system. Neural network models are used along with a classical linear servo controller derived from the linear state space equations. These models are trained so that system uncertainties are compensated. The design of a servo system indicates the enhanced performance of the neural-network-based servo controller as compared to the classical servo controller.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

Modeling spiking neural networks

A notation for the functional specification of a wide range of neural networks consisting of temporal or non-temporal neurons, is proposed. The notation is primarily a mathematical framework, but it can also be illustrated graphically and can be extended into a language in order to be automated. Its basic building blocks are processing entities, finer grained than neurons, connected by instant links, and as such they form sets of interacting entities resulting in bigger and more sophisticated structures. The hierarchical nature of the notation supports both top-down and bottom-up specification approaches. The use of the notation is evaluated by a detailed example of an integrated tangible agent consisting of sensors, a computational part, and actuators. A process from specification to both software and hardware implementation is proposed.     

Manufacturing features recognition using backpropagation neural networks

A backpropagation neural network (BPN) is applied to the problem of feature recognition from a boundary representation (B-rep) solid model to facilitate process planning of manufactured products. It is based on the use of the face complexity code to represent the features and a neural network for the analysis of the recognition. The face complexity code is a measure of the face complexity of a feature based on the convexity or concavity of the surrounding geometry. The codes for various features are fed to the network for analysis. A backpropagation network is implemented for recognition of features and tested on published results to measure its performance. Any two or more features having significant differences in face complexity codes were used as exemplars for training the network. A new feature presented to the network is associated with one of the existing clusters, if they are similar, or the network creates a new cluster, if otherwise. Experimental results show that the network was consistent in recognizing features, hence is appropriate for application to the problem of feature recognition in automated manufacturing environment.     

Controlling fast spring-legged locomotion with artificial neural networks

Controlling the model of an one-legged robot is investigated. The model consists merely of a mass less spring attached to a point mass. The motion of this system is characterised by repeated changes between ground contact and flight phases. It can be kept in motion by active control only. Robots that are suited for fast legged locomotion require different hardware layouts and control approaches in contrast to slow moving ones. The spring mass system is a simple model that describes this principle movement of a spring-legged robot. Multi-Layer-Perceptrons (MLPs), Radial Basis Functions (RBFs) and Self-Organising Motoric Maps (SOMMs) were used to implement neurocontrollers for such a movement system. They all prove to be suitable for control of the movement. This is also shown by an experiment where the environment of the spring-mass system is changed from even to uneven ground. The neurocontroller is performing well with this additional complexity without being trained for it.     

A review of neural networks for statistical process control

This paper aims to take stock of the recent research literature on application of Neural Networks (NNs) to the analysis of Shewhart's traditional Statistical Process Control (SPC) charts. First appearing in the late 1980s, most of the literature claims success, great or small, in applying NNs for SPC (NNSPC). These efforts are viewed in this paper as useful steps towards automatic on-line SPC for continuous improvement of quality and for real-time manufacturing process control. A standard NN approach that can parallel the universality of the traditional Shewhart charts has not yet been developed or adopted, although knowledge in this area is rapidly increasing. This paper attempts to provide a practical insight into the issues involved in application of NNs to SPC with the hope of advancing the use of NN techniques and facilitating their adoption as a new and useful aspect of SPC. First, a brief review of control chart analysis prior to the introduction of NN technology is presented. This is followed by an examination and classification of the NNSPC existing literature. Next, an extensive discussion of implementation issues with reference to significant research papers is presented. Finally, after summarising the survey, a set of general guidelines for future applications of NNs to SPC is outlined.     

Evaluation of correlation methods applying neural networks

This paper deals with a new perception of classic correlation methods on the basis of neural nets. We present and examine neural nets that evaluate the similarity between two arbitrary vectors in a process of measurement or pattern recognition. Thus, classifying patterns by means of feature vectors is feasible just as by correlation methods. Furthermore, we show that the difference correlation procedure and the squared-distance correlation procedure can be presented directly as special cases of the neural methods. Using an example of a typical recognition problem and Gaussian-distributed measuring errors, computer simulations have yielded that neural and correlation procedures are almost identical in behaviour regarding the error rates. Consequently, the neural procedures presented can be understood as a generalisation of correlation procedures.     

Block attractor in spatially organized neural networks

The model of attractor neural network on the small-world topology (local and random connectivity) is investigated. The synaptic weights are random, driving the network towards a disordered state for the neural activity. An ordered macroscopic neural state is induced by a bias in the network weight connections, and the network evolution when initialized in blocks of positive/negative activity is studied. The retrieval of the block-like structure is investigated. An application to the Hebbian learning of a pattern, carrying local information, is presented. The block and the global attractor compete according to the initial conditions and the change of stability from one to the other depends on the long-range character of the network connectivity, as shown with a flow-diagram analysis. Moreover, a larger number of blocks emerges with the network dilution.     

Design of an analytic constrained predictive controller using neural networks

This paper shows how the solution of the standard predictive control problem can be recast as a continuous function of the state, the reference signal, the noise and the disturbances, and hence can be approximated arbitrarily closely by a feed-forward neural network. The existence of such a continuous mapping eliminates the need for linear independency of the active constraints, and therefore the resulting analytic constrained predictive controller will combine constraint handling with speed while being applicable to fast and complex control systems with many constraints. The effectiveness of the proposed controller design methodology is shown for a simulation example of an elevator model and for a real-time laboratory inverted pendulum system.     

A neural network based adaptive robot controller

Neural network based adaptive controllers have been shown to achieve much improved accuracy compared with traditional adaptive controllers when applied to trajectory tracking in robot manipulators. This paper describes a new Recursive Prediction Error technique for estimating network parameters which is more computationally efficient. Results show that this neural controller suppresses disturbances accurately and achieves very small errors between commanded and actual trajectories.     

Traffic Engineering for wireless connectionless access networks supporting QoS-demanding media applications

This paper focuses on the problem of optimal QoS Traffic Engineering (TE) in Co-Channel Interference (CCI)-affected power-limited wireless access networks that support connectionless services. By exploiting the analytical tool offered by nonlinear optimization and following the emerging “Decomposition as Optimization” paradigm [1], the approach pursued in this paper allows to develop a resource allocation algorithm that is distributed, asynchronous, scalable and self-adaptive. Interestingly, the proposed algorithm enables each node of the network to distribute its outgoing traffic among all feasible next-hops in an optimal way, as measured by an assigned global cost function of general form. This optimal traffic distribution complies with several subjective as well as objective QoS requirements advanced by the supported media flows and involves only minimum information exchange between neighboring nodes. Furthermore, it allows for load-balanced multiple forwarding paths and it is able to self-perform optimal traffic re-distribution (i.e., re-routing) in the case of failure of the underlying wireless links. Finally, actual effectiveness of the overall proposed algorithm is numerically tested via performance comparisons against both DSDV-based single-path routing algorithms and interference-aware multipath routing algorithms.     

Integrated optimal topology design and shape optimization using neural networks

In this paper, neural network- and feature-based approaches are introduced to overcome current shortcomings in the automated integration of topology design and shape optimization. The topology optimization results are reconstructed in terms of features, which consist of attributes required for automation and integration in subsequent applications. Features are defined as cost-efficient simple shapes for manufacturing. A neural network-based image-processing technique is presented to match the arbitrarily shaped holes inside the structure with predefined features. The effectiveness of the proposed approach in integrating topology design and shape optimization is demonstrated with several experimental examples.     

Using neural networks to detect the bivariate process variance shifts pattern

Most of the research in statistical process control has been focused on monitoring the process mean. Typically, it is also important to detect variance changes as well. This paper presents a neural network-based approach for detecting bivariate process variance shifts. Some important implementation issues of neural networks are investigated, including analysis window size, number of training examples, sample size, training algorithm, etc. The performance of the neural network, in terms of the ARL and run length distribution, is compared with that of traditional multivariate control charts. Through rigorous evaluation and comparison, our research results show that the proposed neural network performs substantially better than the traditional generalized variance chart and might perform better than the adaptive sizes control charts in the case that the out-of-control covariance matrix is not known in advance.