Rotationally invariant pattern recognition by use of linear and nonlinear cascaded filters

We discuss the merits of using single-layer (linear and nonlinear) and multiple-layer (nonlinear) filters for rotationally invariant and noise-tolerant pattern recognition. The capability of each approach is considered with reference to a two-class, rotation-invariant, character recognition problem. The minimum average correlation energy (MACE) filter is a linear filter that is generally accepted to be optimal for detecting signals that are free from noise. Here it is found that an optimized MACE filter cannot differentiate between the characters E and F in a rotation-invariant manner. We have found, however, that this task is possible when a single optimized linear filter is used to achieve the required response when a nonlinear threshold function is included after the filter. We show that this structure can be cascaded to form a multiple-layer, cascaded filter and that the capability of such a system is enhanced by its increased noise tolerance in the character recognition problem. Finally, we show the capability of a two-layer cascade as a means to detect different species of bacteria in images obtained from a phase-contrast microscope.     

Statistical performance of cascaded linear shift-invariant processing

The cascaded correlator architecture comprises a series of traditional linear correlators separated by nonlinear threshold functions, trained with neural-network techniques. We investigate the shift-invariant classification performance of cascaded correlators in comparison with optimum Bayes classifiers. Inputs are formulated as randomly generated sample members of known statistical class distributions. It is shown that when the separability of true and false classes is varied in both the first and the second orders, the two-stage cascaded correlator shows performance similar to that of the optimum quadratic Bayes classifier throughout the studied range. It is shown that this is due to the similar decision boundaries implemented by the two nonlinear classifiers.     

Nondestructive elastostatic identification of unilateral cracks through BEM and neural networks

An inverse problem in nonlinear elastostatics is considered which concerns the identification of unilateral contact cracks by means of boundary measurements for given static loadings. Highly nonlinear structural behaviour like closed cracks can hardly be identified. In this case, the analysis of more than one loading cases is proposed and tested in this paper. The direct problem is modelled by using a direct multiregion boundary element formulation. The arising linear complementarity problem is solved explicitly by a pivoting (Lemke) technique. In view of the complexity of the inverse problem, a neural network based identification approach is adopted which uses feed-forward multilayer neural networks trained by back-propagation, error-driven supervised training. The applicability of the method is demonstrated by some numerical examples.     

Temperature-constrained cascade correlation networks

A novel neural network has been devised that combines the advantages of cascade correlation and computational temperature constraints. The combination of advantages yields a nonlinear calibration method that is easier to use, stable, and faster than back-propagation networks. Cascade correlation networks adjust only a single unit at a time, so they train very rapidly when compared to back-propagation networks. Cascade correlation networks determine their topology during training. In addition, the hidden units are not readjusted once they have been trained, so these networks are capable of incremental learning and caching. With the cascade architecture, temperature may be optimized for each hidden unit. Computational temperature is a parameter that controls the fuzziness of a hidden unit's output. The magnitude of the change in covariance with respect to temperature is maximized. This criterion avoids local minima, forces the hidden units to model larger variances in the data, and generates hidden units that furnish fuzzy logic. As a result, models built using temperature-constrained cascade correlation networks are better at interpolation or generalization of the design points. These properties are demonstrated for exemplary linear interpolations, a nonlinear interpolation, and chemical data sets for which the numbers of chlorine atoms in polychlorinated biphenyl molecules are predicted from mass spectra.     

Polynomial distance classifier correlation filter for pattern recognition

We introduce what is to our knowledge a new nonlinear shift-invariant classifier called the polynomial distance classifier correlation filter (PDCCF). The underlying theory extends the original linear distance classifier correlation filter [Appl. Opt. 35, 3127 (1996)] to include nonlinear functions of the input pattern. This new filter provides a framework (for combining different classification filters) that takes advantage of the individual filter strengths. In this new filter design, all filters are optimized jointly. We demonstrate the advantage of the new PDCCF method using simulated and real multi-class synthetic aperture radar images.     

Robustness analysis of radial base function and multi-layered feed-forward neural network models

In this paper, two popular types of neural network models (radial base function (RBF) and multi-layered feed-forward (MLF) networks) trained by the generalized delta rule, are tested on their robustness to random errors in input space. A method is proposed to estimate the sensitivity of network outputs to the amplitude of random errors in the input space, sampled from known normal distributions. An additional parameter can be extracted to give a general indication about the bias on the network predictions. The modelling performances of MLF and RBF neural networks have been tested on a variety of simulated function approximation problems. Since the results of the proposed validation method strongly depend on the configuration of the networks and the data used, little can be said about robustness as an intrinsic quality of the neural network model. However, given a data set where ‘pure’ errors from input and output space are specified, the method can be applied to select a neural network model which optimally approximates the nonlinear relations between objects in input and output space. The proposed method has been applied to a nonlinear modelling problem from industrial chemical practice. Since MLF and RBF networks are based on different concepts from biological neural processes, a brief theoretical introduction is given.     

利用结构化神经网络识别振动系统非线性特性

本文提出了振动系统非线性特性识别的结构化神经网络方法。与传统的前馈神经网络不同的是，该法把系统分为线性和非线性两部分，学习得到的神经网络可以单独识别出系统非线性模型，而不是线性与非线性综合在一起的模型。本文将其应用于振动系统非线性特性的识别。实例表明该方法是可行的。     

Neural Networks in Applied Statistics

Artificial neural networks are computer algorithms or computer programs derived in part from attempts to model the activity of nerve cells. They have been applied to pattern recognition, classification, and optimization problems in the physical and chemical sciences, as well as in other fields. We introduce the principles of the multilayer feedforward network that is among the most commonly used neural networks in practical problems. The relevance of neural network models for the applied statistician is considered using a time series prediction problem as an example. The multilayer feedforward neural network uses a nonlinear function of the predictors to obtain predictions for future time series values. We illustrate the considerations involved in specifying a neural network model and evaluate the accuracy of neural network models relative to the accuracy obtained using other computer-intensive, nonmodel-based techniques.     

Face recognition by using feature-specific imaging

We present a face-recognition system based on the optical measurement of linear features. We describe a polarization-based optical system that computes linear projections of an incident irradiance distribution. We quantify the fundamental limitations of optical feature measurement. We find that higher feature fidelity can be obtained by feature-specific imaging than by postprocessing a conventional image. We present feature-fidelity results for wavelet, principal component, and Fisher features. We study face recognition by using a k-nearest neighbors classifier and two different feed-forward neural networks. Each image block is reduced to either a one- or a two-dimensional feature space for input to these recognition algorithms. As high as 99% recognition has been achieved with one-dimensional wavelet feature projections and 100% has been achieved with two-dimensional projections. A 95-fold increase in noise tolerance by use of feature-specific imaging has been demonstrated for an example of the face-recognition problem. An optical experiment is performed to validate these results.     

Neural Networks-Based Real-Time Determination of the Laser Beam Spatial Profile and Vibrational-to-Translational Relaxation Time Within Pulsed Photoacoustics

This paper concerns with the possibilities of computational intelligence application for simultaneous determination of the laser beam spatial profile and vibrational-to-translational relaxation time of the polyatomic molecules in gases by pulsed photoacoustics. Results regarding the application of neural computing through the use of feed-forward multilayer perception networks are presented. Feed-forward multilayer perception networks are trained in an offline batch training regime to estimate simultaneously, and in real-time, the laser beam spatial profile (profile shape class) and the vibrational-to-translational relaxation time from given (theoretical) photoacoustic signals. The proposed method significantly shortens the time required for the simultaneous determination of the laser beam spatial profile and relaxation time and has the advantage of accurately calculating the aforementioned quantities.     

Distance-classifier correlation filters for multiclass target recognition

We describe a correlation-based distance-classifier scheme for the recognition and the classification of multiple classes. The underlying theory uses shift-invariant filters to compute distances between the input image and ideal references under an optimum transformation. The original distance-classifier correlation filter was developed for a two-class problem. We introduce a distance-classifier correlation filter that simultaneously considers multiple classes, and we show that the earlier two-class formulation is a special case of the classifier presented. Initial results are presented to demonstrate the discrimination- and distortion-tolerance capabilities of the proposed filter.     

Virtual Instruments Based on Stacked Neural Networks to Improve Product Quality Monitoring in a Refinery

In this paper, a virtual instrument for the estimation of octane number in the gasoline produced by refineries is introduced. The instrument was designed with the aim of replacing measuring hardware during maintenance operations. The virtual instrument is based on a nonlinear moving average model, implemented by using multilayer perceptron neural networks. Stacking approaches are adopted to improve the estimation performance of the instrument. Classical linear algorithms of model aggregation are compared in the paper with a nonlinear strategy, based on the neural combination of a set of first-level neural estimators. The validity of the proposed approach is verified by comparison with the performance of both linear and nonlinear modeling techniques. The designed virtual instrument has been implemented by a large refinery in Sicily, which supplied the data used during the design phase     

Remote sensing image segmentation using a Kalman filter-trained neural network

This article describes the application of a neural network to the segmentation of remote sensing images of multispectral SPOT and fully polarimetric SAR data. The structure of the network is a modified multilayer perceptron and is trained by the Kalman filter theory. The internal activity of the network is a nonlinear function, while the function at output layer is linearized through the use of a polynomial basis function, thus allowing us employ the theory of Kalman filtering as the learning rule. The network is therefore called the dynamic learning (DL) neural network. It is found that, when applied to SPOT and SAR data, the DL neural network gives a good segmentation results, while the learning rate is very promising compared to the standard backpropagation network and other fast-learning networks. In particular, for polarimetric SAR data, optimum polarizations for discriminating between different terrains are automatically built in through the use of the Kalman filter technique. The suitability and effectiveness of the proposed DL neural network to the segmentation of remote sensing images is demonstrated. © 1996 John Wiley & Sons, Inc.     

The Utilization of Artificial Neural Networks for Multisensor System Integration in Navigation and Positioning Instruments

Inertial-navigation system (INS) and global position system (GPS) technologies have been widely applied in many positioning and navigation applications. INS determines the position and the attitude of a moving vehicle in real time by processing the measurements of three-axis gyroscopes and three-axis accelerometers mounted along three mutually orthogonal directions. GPS, on the other hand, provides the position and the velocity through the processing of the code and the carrier signals of at least four satellites. Each system has its own unique characteristics and limitations. Therefore, the integration of the two systems offers several advantages and overcomes each of their drawbacks. The integration of INS and GPS is usually implemented utilizing the Kalman filter, which represents one of the best solutions for INS/GPS integration. However, the Kalman filter performs adequately only under certain predefined dynamic models. Alternatively, this paper suggests an INS/GPS integration method based on artificial neural networks (ANN) to fuse uncompensated INS measurements and differential GPS (DGPS) measurements. The proposed method suggests two different architectures: the position update architecture (PUA) and the position and velocity PUA (PVUA). Both architectures were developed utilizing multilayer feed-forward neural networks with a conjugate gradient training algorithm.     

Analysis of Knowledge Representations in Cascade Correlation Networks

Feed-forward neural network models approximate nonlinear functions connecting inputs to outputs. The cascade correlation (CC) learning algorithm allows networks to grow dynamically starting from the simplest network topology to solve increasingly more difficult problems. It has been demonstrated that the CC network can solve a wide range of problems including those for which other kinds of networks (e.g., back-propagation networks) have been found to fail. In this paper we show the mechanism and characteristics of nonlinear function learning and representations in CC networks, their generalization capabilities, the effects of environmental bias, etc., using a variety of knowledge representation analysis tools.

     

Evaluation of neural network models with generalized sensitivity analysis

A sensitivity analysis method for discovering characteristic features of the input data using neural network classification models has been devised. The sensitivity is the gradient of the neural network model response function, and because neural network models are nonlinear, the gradient depends on the point where it is evaluated. Two criteria are used for measuring the sensitivity. The first criterion calculates the sensitivity or gradient of the neural network output with respect to the average of the objects that comprise each class. The second criterion measures the average sensitivity of the class objects. The sensitivity analysis was applied to temperature-constrained cascade correlation network models and evaluated with sets of synthetic data and experimental mobility spectra. The neural network models were built using temperature-constrained cascade correlation networks (TCCCNs). A weight constraint was devised for the output units of the network models. This method implements weight decay with conjugate gradient training and yields more sensitive neural network models. Temperature-constrained hidden units furnish more sensitive network models than networks without constraints. By comparing the sensitivities of the class mean input and the mean sensitivity for all the inputs of a class, the individual input variables may be assessed for linearity. If these two sensitivities for an input variable differ by a constant factor, then that variable is modeled by a simple linear relationship. If the two sensitivities vary by a nonconstant scale factor, then the variable is modeled by higher order functions in the network. The sensitivity method was used to diagnose errors in the training data, and the test for linearity indicated a TCCCN architecture that had better predictability.     

电液伺服系统的神经网络建模方法研究

针对电液伺服系统固有的流量-压力特性等非线性因素使得采用传递函数等传统方法难以获得电液伺服系统的精确模型的问题,详细研究了电液伺服系统的神经网络建模方法.研究了两种最常见的神经网络,即多层感知器神经网络和径向基函数神经网络,采用5种典型学习算法构造了3种多层感知器神经网络和2种径向基函数神经网络,并结合自动定深电液伺服系统的工程实例,详细分析了这5种神经网络在电液伺服系统中的建模性能.研究结果表明,采用正交最小二乘算法的径向基函数神经网络最适合电液伺服系统的建模.     

Optical wavelet matched filter

A shift-invariant optical continuous wavelet transform is used for pattern recognition. We propose an Voptical wavelet matched filter that performs optical wavelet transforms for edge enhancement and the correlation between two wavelet transforms in a single step. This new bandpass matched filter shows improved discrimination capability with respect to the conventional matched filter and improved signal-to-noise ratio with respect to the phase-only matched filter. The wavelet matched filter provides flexibility of an adaptive choice of the scale factors of the wavelets that permit the selection of size and orientation of the smoothing function used in edge enhancement and the optimization of the performance of the filter. Optical experimental results are shown.     

Modeling the oxidation behavior of hot pressed nanocrystalline Cr–33Nb alloys

The oxidation behavior of hot pressed nanocrystalline Cr–33Nb alloys was modeled using a feed-forward multilayer Perceptron artificial neural network model. It was found that the artificial neural networks model is an applicable method for prediction of the oxidation behavior of hot pressed nanocrystalline Cr–33Nb alloys. The optimum number of the neurons and hidden layers to do this simulation was 16 and 16, respectively.     

Artificial neural network processing of stripping analysis responses for identifying and quantifying heavy metals in the presence of intermetallic compound formation

Feed-forward neural networks have been trained to identify and quantify heavy metals in mixtures under conditions where there were significant complications due to intermetallic compound formation. The networks were shown to be capable of (i) correlating voltammetric responses with individual heavy metals in complex mixtures, (ii) determining the relationship between responses and concentrations (including nonlinear relationships due to overlapping peaks and intermetallic compound formation), and (iii) rapidly determining concentrations of individual components from mixtures once trained. Using simulated data, modeled after complex interactions experimentally observed in samples containing Cu and Zn, it has been demonstrated that networks containing two layers of neurons (a nonlinear hidden layer and a linear output layer) can be trained to calculate concentrations under a variety of complicated situations. These include, but are not limited to, cases where the response of the intermetallic compound formed is observed as a shoulder of one of the pure metals and cases where the response of the intermetallic compound formed is not observed in the potential window. In addition, the network described above was trained to simultaneously determine concentrations of four metals (Cu, Pb, Cd, and Zn) in a concentration range where all responses were complicated by intermetallic compound formation (1-500 ppb).