Back-propagation learning in expert networks

Expert networks are event-driven, acyclic networks of neural objects derived from expert systems. The neural objects process information through a nonlinear combining function that is different from, and more complex than, typical neural network node processors. The authors develop back-propagation learning for acyclic, event-driven networks in general and derive a specific algorithm for learning in EMYCIN-derived expert networks. The algorithm combines back-propagation learning with other features of expert networks, including calculation of gradients of the nonlinear combining functions and the hypercube nature of the knowledge space. It offers automation of the knowledge acquisition task for certainty factors, often the most difficult part of knowledge extraction. Results of testing the learning algorithm with a medium-scale (97-node) expert network are presented.     

Parallel, self-organizing, hierarchical neural networks

A new neural-network architecture called the parallel, self-organizing, hierarchical neural network (PSHNN) is presented. The new architecture involves a number of stages in which each stage can be a particular neural network (SNN). At the end of each stage, error detection is carried out, and a number of input vectors are rejected. Between two stages there is a nonlinear transformation of input vectors rejected by the previous stage. The new architecture has many desirable properties, such as optimized system complexity (in the sense of minimized self-organizing number of stages), high classification accuracy, minimized learning and recall times, and truly parallel architectures in which all stages operate simultaneously without waiting for data from other stages during testing. The experiments performed indicated the superiority of the new architecture over multilayered networks with back-propagation training.     

Short-term potentiation effect on pattern recall in sparsely coded neural network

It has been shown in studies of biological synaptic plasticity that synaptic efficacy can change in a very short time window, compared to the time scale associated with typical neural events. This time scale is small enough to possibly have an effect on pattern recall processes in neural networks. We study properties of a neural network which uses a cyclic Hebb rule. Then we add the short term potentiation of synapses in the recall phase. We show that this approach preserves the ability of the network to recognize the patterns stored by the network and that the network does not respond to other patterns at the same time. We show that this approach dramatically increases the capacity of the network at the cost of a longer pattern recall process. We discuss that the network possesses two types of recall. The fast recall does not need synaptic plasticity to recognize a pattern, while the slower recall utilizes synaptic plasticity. This is something that we all experience in our daily lives: some memories can be recalled promptly whereas recollection of other memories requires much more time.     

BP算法的改进及用模拟电路实现的神经网络分类器

基于用模拟电路实现神经网络分类器的目的，对多层静态前馈神经网络的ＢＰ算法做了改进，采用线性限幅函数代替Ｓｉｇｍｏｉｄ函数作为神经元的激活函数，给出了改进的ＢＰ算法。对该算法性能的实验研究表明：这种改进算法不但方便了用线性模拟集成运算放大电路实现神经网络，而且具有学习速度快，映射能力强等优点。根据本文算法设计的神经网络分类器，无论是计算机仿真，还是模拟电路实现，都得到了比较高的识别率。     

A reference model approach to stability analysis of neural networks

In this paper, a novel methodology called a reference model approach to stability analysis of neural networks is proposed. The core of the new approach is to study a neural network model with reference to other related models, so that different modeling approaches can be combinatively used and powerfully cross-fertilized. Focused on two representative neural network modeling approaches (the neuron state modeling approach and the local field modeling approach), we establish a rigorous theoretical basis on the feasibility and efficiency of the reference model approach. The new approach has been used to develop a series of new, generic stability theories for various neural network models. These results have been applied to several typical neural network systems including the Hopfield-type neural networks, the recurrent back-propagation neural networks, the BSB-type neural networks, the bound-constraints optimization neural networks, and the cellular neural networks. The results obtained unify, sharpen or generalize most of the existing stability assertions, and illustrate the feasibility and power of the new method.     

Bioreactor temperature profile controller using inverse neural network (INN) for production of ethanol

This paper presents the use of inverse neural networks (INN) for temperature control of a biochemical reactor and its effect on ethanol production. The process model is derived indicating the relationship between temperature, pH and dissolved oxygen. Using fundamental model obtained data sets; an inverse neural network has been trained using the back-propagation learning algorithm. Two types of temperature profile are used to compare the performance of the INN and conventional PID controllers. These controllers have been simulated in MATLAB for a quantitative comparison. The results obtained by the neural network based INN controller and by the PID controller are presented and compared. There is an improvement in the performance of INN controller in settling time and dead time and steady state error over the PID controller.     

Fuzzy neural computing of coffee and tainted-water data from an electronic nose

In this paper we compare the ability of a fuzzy neural network and a common back-propagation network to classify odour samples that were obtained by an electronic nose employing semiconducting oxide conductometric gas sensors. Two different sample sets have been analysed: first, the aroma of three blends of commercial coffee, and secondly, the headspace of six different tainted-water samples. The two experimental data sets provide an excellent opportunity to test the ability of a fuzzy neural network due to the high level of sensor variability often experienced with this type of sensor. Results are presented on the application of three-layer fuzzy neural networks to electronic nose data. They demonstrate a considerable improvement in performance compared to a common back-propagation network.     

基于神经网络的飞控系统传感器故障诊断的研究

在标准逆传播神经网络的基础上，提出了一种改进的逆传播神经网络，用来消除标准逆传播神经网络的缺陷，并且设计了一个主神经网络和三个分布神经网络的结构，通过神经网络的在线学习，得到需要的参数估计。用它与传感器实际测得的实际值比较，可判断出阶跃故障，并且给出了仿真实例。     

Application of Feed-forward Artificial Neural Networks to the Identification of Defective Analog Integrated Circuits

This paper presents a new approach for detecting defects in analog integrated circuits using a feed-forward neural network trained by the resilient error back-propagation method. A feed-forward neural network has been used for detecting faults in a simple analog CMOS circuit by representing the differences observed in power supply current of fault-free and faulty circuits. The identification of defects was performed in time and frequency domains, followed by a comparison of results achieved in both domains. We show that resilient back-propagation neural networks can be a very efficient and versatile approach for identifying defective analog circuits. Moreover, this approach is not limited to the supply current analysis, because it also offers monitoring of other circuit parameters. The type of defects detected by the resilient backpropagation neural networks, as well as other possible applications of this approach, are discussed.     

A kind of BP neural network algorithm based on grey interval

In order to improve the learning ability of a forward neural network, in this article, we incorporate the feedback back-propagation (FBBP) and grey system theory to consider the learning and training of a neural network new perspective. By reducing the input grey degree we optimise the input of the neural network to make it more rational for learning and training of neural networks. Simulation results verified the efficiency of the proposed algorithm by comparing its performance with that of FBBP and classic back-propagation (BP). The results showed that the proposed algorithm has the characteristics of fast training and strong ability of generalisation and it is an effective learning method.     

基于伪并行混合遗传算法的神经网络优化

在分析并行多物种遗传算法应用于神经网络拓扑结构的设计和学习之后,提出一种伪并行遗传（PPGA-MBP）混合算法,结合改进的BP算法对多层前馈神经网络的拓扑结构进行优化。算法编码采用基于实数的层次混合方式,允许两个不同结构的网络个体交叉生成有效子个体。利用该算法对N-Parity问题进行了实验仿真,并对算法中评价函数各部分系数和种群规模对算法的影响进行了分析。实验证明取得了明显的优化效果,提高了神经网络的自适应能力和泛化能力,具有全局快速收敛的性能。     

Neural network-based colonoscopic diagnosis using on-line learning and differential evolution

In this paper, on-line training of neural networks is investigated in the context of computer-assisted colonoscopic diagnosis. A memory-based adaptation of the learning rate for the on-line back-propagation (BP) is proposed and used to seed an on-line evolution process that applies a differential evolution (DE) strategy to (re-) adapt the neural network to modified environmental conditions. Our approach looks at on-line training from the perspective of tracking the changing location of an approximate solution of a pattern-based, and thus, dynamically changing, error function. The proposed hybrid strategy is compared with other standard training methods that have traditionally been used for training neural networks off-line. Results in interpreting colonoscopy images and frames of video sequences are promising and suggest that networks trained with this strategy detect malignant regions of interest with accuracy.     

Phase equilibria prediction of solid solute in supercritical carbon dioxide with and without a cosolvent: The use of artificial neural network

In this study, a back-propagation multi-layer neural network was developed to predict the solubility of solid solute in supercritical carbon dioxide with and without cosolvent. The solubility of anthracene in CO₂ with cosolvents, acetone, ethanol and cyclohexane were employed as model systems to investigate the supercritical carbon dioxide behaviour in ternary systems over a wide range of temperatures. The back-propagation neural network operated in a supervised learning mode. A number of networks were trained and tested with different network parameters using training and testing data sets. To establish the network applicability, a validating data set was used and the predictability of the network was statistically evaluated. Statistical estimations showed that the neural network predictions had an excellent agreement with experimental data. The calculated average relative deviation (ARD) and the root mean squared error (RMSD) for tested ANNs data points were 5.45% and 0.74%, respectively. A minimum number of data points have been employed to train the ANN. The predicted ARD and RMSD for the employed ternary systems were 7.83% and 0.07%, respectively. The results obtained in this work indicate that ANN is a superior technique with high level of accuracy for prediction of solubility of solid solute in ternary systems.     

基于BP神经网络的病毒检测方法

提出了一种基于BP神经网络的病毒检测方法,该方法成功地把BP神经网络的理论引入计算机病毒的检测中。该方法比传统的病毒检测技术更有效地对系统信息和文件系统进行语法分析,快速地诊断出被感染病毒以及病毒类型。     

The training of neural networks to model manufacturing processes

Neural networks have been increasingly used in various areas of manufacturing. Modelling of manufacturing processes, to allow experimentation on the model, is one of the areas in which successful applications have been reported. Most literature in this area is focused on network results. This paper concentrates on methods for training neural networks to model complex manufacturing processes. It summarises the use of neural network for process modelling in the past decade and provides some detailed guidelines for network training. A case study of a complex forming process is used to demonstrate a real implementation case in industry, and the issues arising from this case are discussed.     

Neural projection techniques for the visual inspection of network traffic

A crucial aspect in network monitoring for security purposes is the visual inspection of the traffic pattern, mainly aimed to provide the network manager with a synthetic and intuitive representation of the current situation. Towards that end, neural projection techniques can map high-dimensional data into a low-dimensional space adaptively, for the user-friendly visualization of monitored network traffic. This work proposes two projection methods, namely, cooperative maximum likelihood Hebbian learning and auto-associative back-propagation networks, for the visual inspection of network traffic. This set of methods may be seen as a complementary tool in network security as it allows the visual inspection and comprehension of the traffic data internal structure. The proposed methods have been evaluated in two complementary and practical network-security scenarios: the on-line processing of network traffic at packet level, and the off-line processing of connection records, e.g. for post-mortem analysis or batch investigation. The empirical verification of the projection methods involved two experimental domains derived from the standard corpora for evaluation of computer network intrusion detection: the MIT Lincoln Laboratory DARPA dataset.     

并联混合神经网络模型及应用研究

单一神经网络难以对复杂模型做出准确的预测,提出了一种并联型混合神经网络模型用于对复杂的系统进行预测,该模型由径向基函数网络、BP网络和控制模块组成。控制模块用于线性映射层,将两种单一神经网络的输出结合并得到最终的输出结果。详细地给出了混合模型的预测方法：首先,利用改进算法分别训练径向基函数网络和BP网络;其次,采用自适应遗传算法优化线性映射层以获得更好的预测精度;最后,利用两个实例比较单一神经网络和提出的混合网络的预测性能。实验表明,混合神经网络在预测精度上比单一网络具有更优的性能,同时,该混合模型为复杂系统提供了一种通用的预测工具。     

RBFNN versus FFNN for daily river flow forecasting at Johor River,Malaysia

Streamflow forecasting can have a significant economic impact, as this can help in water resources management and in providing protection from water scarcities and possible flood damage. Artificial neural network (ANN) had been successfully used as a tool to model various nonlinear relations, and the method is appropriate for modeling the complex nature of hydrological systems. They are relatively fast and flexible and are able to extract the relation between the inputs and outputs of a process without knowledge of the underlying physics. In this study, two types of ANN, namely feed-forward back-propagation neural network (FFNN) and radial basis function neural network (RBFNN), have been examined. Those models were developed for daily streamflow forecasting at Johor River, Malaysia, for the period (1999–2008). Comprehensive comparison analyses were carried out to evaluate the performance of the proposed static neural networks. The results demonstrate that RBFNN model is superior to the FFNN forecasting model, and RBFNN can be successfully applied and provides high accuracy and reliability for daily streamflow forecasting.     

Query based model learning and stable tracking of a robot arm using radial basis function network

In the context of a robot manipulator, a generalized neural emulator over the complete workspace is very difficult to obtain because of dimensionally insufficient training data. A query based learning algorithm is proposed in this paper that can generate new examples where control inputs are independent of states of the system. This algorithm is centered around the concept of network inversion using an extended Kalman filtering based algorithm. This is a novel idea since robot manipulator is an open loop unstable system and generation of control input independent of state is a research issue for neural model identification. Two trajectory independent stable control schemes have been designed using the neural emulator. One of the control schemes uses forward-inverse-modeling approach to update the controller parameters adaptively following Lyapunov function synthesis technique. The proposed scheme is trajectory independent unlike the back-propagation scheme. The second type of controller predicts the minimum variance estimate of control action using recall process (network inversion) and the control law is derived following a Lyapunov function synthesis approach so that the closed loop system consisting of controller and neural emulator remains stable. The simulation experiments show that the model validation approach is efficient and the proposed control schemes guarantee stable accurate tracking.     

A parallel formulation of back-propagation learning on distributed memory multiprocessors

This paper presents a mapping scheme for parallel pipelined execution of the Back-propagation Learning Algorithm on distributed memory multiprocessors. The proposed implementation exhibits inter-layer or pipelined parallelism, unique to the multilayer neural networks. Simple algorithms have been presented, which allow the data transfer involved in both recall and learning phases of the back-propagation algorithm to be carried out with a small communication overhead. The effectiveness of the mapping scheme has been illustrated, by estimating the speedup of the proposed implementation on an array of T-805 transputers.