Reinforcement and backpropagation training for an optical neural network using self-lensing effects

The optical bench training of an optical feedforward neural network, developed by the authors, is presented. The network uses an optical nonlinear material for neuron processing and a trainable applied optical pattern as the network weights. The nonlinear material, with the applied weight pattern, modulates the phase front of a forward propagating information beam by dynamically altering the index of refraction profile of the material. To verify that the network can be trained in real time, six logic gates were trained using a reinforcement training paradigm. More importantly, to demonstrate optical backpropagation, three gates were trained via optical error backpropagation. The output error is optically backpropagated, detected with a CCD camera, and the weight pattern is updated and stored on a computer. The obtained results lay the ground work for the implementation of multilayer neural networks that are trained using optical error backpropagation and are able to solve more complex problems.     

A neural network based on an inexpensive eight-bit microcontroller

In this paper, a neural network is trained and validated using a low end and inexpensive microcontroller. The well-known backpropagation algorithm is implemented to train a neural network model. Both the training and the validation parts are shown through an alphanumeric liquid crystal display. A chemical process was chosen as a realistic nonlinear system to demonstrate the feasibility, and the performance of the results found using the microcontroller. A comparison was made between the microcontroller and the computer results.     

Creating efficient nonlinear neural network process models that allow model interpretation

The KBANN (knowledge-based artificial neural networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. This idea is extended by presenting the MANNIDENT (multivariable artificial neural network identification) algorithm by which the mathematical equations of linear process models determine the topology and initial weights of a network, which is further trained using backpropagation. This method is applied to the task of modelling a non-isothermal CSTR in which a first-order exothermic reaction is occurring. This method produces statistically significant gains in accuracy over both a standard neural network approach and a linear model. Furthermore, using the approximate linear model to initialize the weights of the network produces statistically less variation in accuracy. By structuring the neural network according to the approximate linear model, the model can be readily interpreted.     

Simultaneous recurrent neural network trained with non-recurrent backpropagation algorithm for static optimisation

This paper explores feasibility of employing the non-recurrent backpropagation training algorithm for a recurrent neural network, Simultaneous Recurrent Neural network, for static optimisation. A simplifying observation that maps the recurrent network dynamics, which is configured to operate in relaxation mode as a static optimizer, to feedforward network dynamics is leveraged to facilitate application of a non-recurrent training algorithm such as the standard backpropagation and its variants. A simulation study that aims to assess feasibility, optimizing potential, and computational efficiency of training the Simultaneous Recurrent Neural network with non-recurrent backpropagation is conducted. A comparative computational complexity analysis between the Simultaneous Recurrent Neural network trained with non-recurrent backpropagation algorithm and the same network trained with the recurrent backpropagation algorithm is performed. Simulation results demonstrate that it is feasible to apply the non-recurrent backpropagation to train the Simultaneous Recurrent Neural network. The optimality and computational complexity analysis fails to demonstrate any advantage on behalf of the non-recurrent backpropagation versus the recurrent backpropagation for the optimisation problem considered. However, considerable future potential that is yet to be explored exists given that computationally efficient versions of the backpropagation training algorithm, namely quasi-Newton and conjugate gradient descent among others, are also applicable for the neural network proposed for static optimisation in this paper.     

基于NN/HMM混合模型的汉语地名识别系统

文章介绍了一个基于NN/HMM混合模型的汉语地名识别系统,该系统能自动判别并拒识词表之外的词。文中训练的基于HMM的模型,包括关键词模型、填充模型和“反关键词”模型。笔者对识别器的输出结果进行验证,把基于HMM的统计特征送到神经网络处理,由网络的输出来判断是否为词表之外的词。该文在实验中建立了一个基于传统N-Best方法的基准模型并试验了三种不同的网络拓扑结构,包括前馈后向传播网络、Elman后向传播网络以及可训练级联前导后向传播网络。实验结果表明前馈后向传播网络的性能最好,与基准模型比较平均错误率下降54.4%。     

Execution of a remote sensing application on a custom neurocomputer

A radial basis function neural network was successfully applied to an area which is relatively new for neural networks: a remote sensing application that provides estimates of water vapor content, an important parameter for climate modeling. The neural network provided results which are up to 32% better than had been previously obtained using conventional statistical methods on the same data. These results have implications for improved short-term weather forecasting and for long-term global climate modeling. The neural network approach is compared with the past and present operating algorithms at the National Oceanic and Atmospheric Administration. The radial basis function network's performance is compared with sigmoidal backpropagation network. Low-power electronic implementations of the neural methodology were explored to demonstrate the feasibility of placing the network on a remote sensing platform. This would permit processing the raw sensor data into information on the platform, eliminating the need to store the raw data, and helping to contain the expected explosion of climate data.     

Grammatical analysis of Hindi sentences using connectionist approach

Abstract: This paper presents a simple connectionist approach to parsing of a subset of sentences in the Hindi language, using Rule based Connectionist Networks (RBCN) as suggested by Fu in 1993. The basic grammar rules representing Kernel Hindi sentences have been used to determine the initial topology of the RBCN. The RBCN is based on a multilayer perceptron, trained using the backpropagation algorithm. The terminal symbols defined in the language structure are mapped onto the input nodes, the non-terminals onto hidden nodes and the start symbol onto the single output node of the network structure. The training instances are sentences of arbitrary, but fixed maximum length and fixed word order. A neural network based recognizer is used to perform grammaticality determination and parse tree generation of a given sentence. The network is exposed to both positive and negative training instances, derived from a simple context-free-grammar (CFG), during the training phase. The trained network recognizes seen sentences (sentences present in the training set) with 98–100% accuracy. Since a neural net based recognizer is trainable in nature, it can be trained to recognize any other CFG, simply by changing the training set. This results in reducing programming effort involved in parser development, as compared to that of the conventional AI approach. The parsing time is also reduced to a great extent as compared to that of a conventional parser, as a result of the inherent parallelism exhibited by neural net architecture.     

Neural networks for the EMOBOT robot control architecture

Within the EMOBOT approach to adaptive behaviour, the task of learning to control the behaviour is one of the most interesting challenges. Learned action selection between classically implemented control mechanisms, with respect to internal values and sensor readings, provides a way to modulate a variety of behavioural capabilities. To demonstrate the potential of the learning emotional controller, we chose a 10-5-12 MLP to implement the , controller of the EMOBOT. Since no teacher vector is available for the chosen task, the neural network is trained with a reinforcement strategy. The emotion-value-dependent reinforcement signal, together with the output of the network, is the basis with which to compute an artificial teacher vector. Then, the established gradient descent method (backpropagation of error) is applied to train the neural network. First results obtained by extensive simulations show that a still unrevealed richness in behaviour can be realised when using the neural-network-based learning emotional controller.     

Synthesis of optimal feedback controller by neural networks

In nonlinear optimal control problems, open-loop solutions from a fixed initial condition are much easier to compute than closed-loop solutions which do not depend on initial conditions. Two methods of using neural networks to approximate the optimal feedback controller are discussed. The indirect method uses a neural network to interpolate the whole field of extremals obtained from open-loop calculation. The direct method directly trains a neural network such that a general nonlinear optimal control performance index is minimized. The novelty of the modified backpropagation training is the requirement of the jacobian matrix of the neural network function. Simulation studies show that the closed-loop solution can be made to be arbitrarily close to the optimal open-loop solution with initial conditions chosen from a nontrivial subset of the state space.     

Fuzzy neural networks for estimation and fuzzy controller design: simulation study for a pulp batch digester

A structural implementation of a fuzzy inference system through connectionist network based on MLP with logical neurons connected through binary and numerical weights is considered. The resulting fuzzy neural network is trained using classical backpropagation to learn the rules of inference of a fuzzy system, by adjustment of the numerical weights. For controller design, training is carried out off line in a closed loop simulation. Rules for the fuzzy logic controller are extracted from the network by interpreting the consequence weights as measure of confidence of the underlying rule. The framework is used in a simulation study for estimation and control of a pulp batch digester. The controlled variable, the Kappa number, a measure of lignin content in the pulp, which is not measurable is estimated through temperature and liquor concentration using the fuzzy neural network. On the other hand a fuzzy neural network is trained to control the Kappa number and rules are extracted from the trained network to construct a fuzzy logic controller.     

用于过程建模的BP网络的M训练法及其改进

以Sigmoid为传递函数的BP网络在过程系统工程领域已经得到了广泛的应用 ,但是一般的GDR训练算法在极小点附近易发生振荡 ,收敛速度慢。本文提出了人工神经网络M法训练的新途径 ,并且通过不同算例和工业实际数据建模应用证实了M算法的收敛速度大约是GDR算法的 5 - 10倍左右 ,有效地提高了网络训练的速度和训练效率     

Prediction and optimization of depth of penetration for stainless steel gas tungsten arc welded plates using artificial neural networks and simulated annealing algorithm

The quality of a weld joint is highly influenced by depth of penetration. Hence, accurate prediction and maximization of depth of penetration is highly essential to ensure a good-quality joint. This paper highlights the development of neural network model for predicting depth of penetration and optimizing the process parameters for maximizing depth of penetration using simulated annealing algorithm. The process parameters chosen for the study are welding current, welding speed, gas flow rate and welding gun angle. The chosen output parameter was depth of penetration. The experiments were conducted based on design of experiments using fractional factorial with 125 runs. Using the experimental data, feed-forward backpropagation neural network model was developed and trained using Levenberg–Marquardt algorithm. It was found that ANN model based on network 4-15-1 predicted depth of penetration more accurately. A mathematical model was also developed correlating the process parameters with depth of penetration for doing optimization. A source code was developed in MATLAB to do the optimization. The optimized process parameters gave a value of 3.778 mm for depth of penetration.     

Multilayer neural networks for reduced-rank approximation

This paper is developed in two parts. First, the authors formulate the solution to the general reduced-rank linear approximation problem relaxing the invertibility assumption of the input autocorrelation matrix used by previous authors. The authors' treatment unifies linear regression, Wiener filtering, full rank approximation, auto-association networks, SVD and principal component analysis (PCA) as special cases. The authors' analysis also shows that two-layer linear neural networks with reduced number of hidden units, trained with the least-squares error criterion, produce weights that correspond to the generalized singular value decomposition of the input-teacher cross-correlation matrix and the input data matrix. As a corollary the linear two-layer backpropagation model with reduced hidden layer extracts an arbitrary linear combination of the generalized singular vector components. Second, the authors investigate artificial neural network models for the solution of the related generalized eigenvalue problem. By introducing and utilizing the extended concept of deflation (originally proposed for the standard eigenvalue problem) the authors are able to find that a sequential version of linear BP can extract the exact generalized eigenvector components. The advantage of this approach is that it's easier to update the model structure by adding one more unit or pruning one or more units when the application requires it. An alternative approach for extracting the exact components is to use a set of lateral connections among the hidden units trained in such a way as to enforce orthogonality among the upper- and lower-layer weights. The authors call this the lateral orthogonalization network (LON) and show via theoretical analysis-and verify via simulation-that the network extracts the desired components. The advantage of the LON-based model is that it can be applied in a parallel fashion so that the components are extracted concurrently. Finally, the authors show the application of their results to the solution of the identification problem of systems whose excitation has a non-invertible autocorrelation matrix. Previous identification methods usually rely on the invertibility assumption of the input autocorrelation, therefore they can not be applied to this case.     

APPRAISAL OF LIQUEFACTION POTENTIAL USING NEURAL NETWORK AND NEURO FUZZY APPROACH

In this study, standard penetration test dependent bore-log charts of different boreholes were collected for selected locations in order to prepare the datasets. Datasets were applied to the Idriss and Boulanger method to evaluate liquefaction potential. Complete datasets were used for development of neural network and neuro-fuzzy models. Feed forward backpropagation algorithm with a multilayer perceptron network is utilized to analyze the liquefaction occurrence in different locations. To meet the objective, 159 sets of geotechnical data were collected, out of which 133 datasets were used for development of models and 26 datasets were used for validation. Neural network models were trained with six input vectors by optimum numbers of hidden layers, epoch, and suitable transfer functions. Neuro-fuzzy models have been developed using the Takagi–Sugeno–Kang reliant approach. The predicted values of liquefaction potential by artificial neural networks and neuro-fuzzy models were compared with an empirical method (i.e., Idriss and Boulanger method). The compared values of liquefaction potential obtained by neural network and neuro-fuzzy models shows that trained artificial neural network models' prediction capability are better than that of neuro-fuzzy models.     

Towards the Optimal Learning Rate for Backpropagation

A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm.     

Fuzzy assisted learning in backpropagation neural networks

This paper reports on studies to overcome difficulties associated with setting the learning rates of backpropagation neural networks by using fuzzy logic. Building on previous research, a fuzzy control system is designed which is capable of dynamically adjusting the individual learning rates of both hidden and output neurons, and the momentum term within a back-propagation network. Results show that the fuzzy controller not only eliminates the effort of configuring a global learning rate, but also increases the rate of convergence in comparison with a conventional backpropagation network. Comparative studies are presented for a number of different network configurations. The paper also presents a brief overview of fuzzy logic and backpropagation learning, highlighting how the two paradigms can enhance each other.     

Dynamics identification and control of nonlinear MIMO coupled plant using supervised neural gas and comparison with recurrent neural controller

The dynamics identification and subsequent control of a nonlinear system is not a trivial issue. The application of a neural gas network that is trained with a supervised batch version of the algorithm can produce identification models in a robust way. In this paper, the neural model identifies each local transfer function, demonstrating that the local linear approximation can be done. Moreover, other parameters are analyzed in order to obtain a correct modeling. Furthermore, the algorithm is applied to control a nonlinear multi-input multi-output system composed of tanks. In addition, this plant is a coupled system where the manipulated input variables are influencing all the output variables. The aim of the work is to demonstrate that the supervised neural gas algorithm is able to obtain linear models to be used in a state space design scenario to control nonlinear coupled systems and guarantee a robust control method. The results are compared with the common approach of using a recurrent neural controller trained with a dynamic backpropagation algorithm. Regarding the steady-state errors in disturbance rejection, reference tracking and sensitivity to simple process changes, the proposed approach shows an interesting application to control nonlinear plants.

     

基于稀疏诱导特征选择神经网络的储层预测方法研究

常规储层预测方法对地震属性之间的隐含关系挖掘不充分、地震属性种类繁多难以选择.针对以上问题,为提高储层岩性的分类预测精度,提出一种结合特征选择与神经网络的储层预测方法.以DenseNet与SENet为基础,使用正则惩罚项进行网络输入层稀疏化,得到每个输入节点权重,进一步使用ReLU激活函数构建特征选择层,实现地震属性的...     

A fuzzy logic system for the detection and recognition ofhandwritten street numbers

Fuzzy logic is applied to the problem of locating and reading street numbers in digital images of handwritten mail. A fuzzy rule-based system is defined that uses uncertain information provided by image processing and neural network-based character recognition modules to generate multiple hypotheses with associated confidence values for the location of the street number in an image of a handwritten address. The results of a blind test of the resultant system are presented to demonstrate the value of this new approach. The results are compared to those obtained using a neural network trained with backpropagation. The fuzzy logic system achieved higher performance rates     

Use of neural networks for condition rating of jointed concrete pavements

This paper discusses the use of backpropagation neural networks as a management tool for the maintenance of jointed concrete pavement. The backpropagation algorithm is applied to model the condition rating scheme adopted by Oregon State Department of Transportation. The backpropagation technique was successful in accurately capturing the nonlinear characteristics of the condition rating model. A large training set of actual pavement condition cases was used to train the network. The training was terminated when the average training error reached 0.022. A set of 6802 cases was used to test the generalization ability of the system. The trained network was able to accurately determine the correct condition ratings with an average testing error of 0.024. Finally, a statistical hypothesis test was conducted to demonstrate the system's fault-tolerance and generalization properties.