Fusion of artificial neural networks for learning capability enhancement: Application to medical image classification

Artificial neural network (ANN) is one of the commonly used tools for computational applications. The specific advantages of ANN are high accuracy, less convergence time, less computational complexity, and so forth. However, all these merits are not available in the same ANN. Even though back propagation neural (BPN) networks are accurate, their computational complexity is significantly high. BPN networks are also not stable. On the other hand, Hopfield neural network (HNN) is better than BPN in terms of computational efficiency. But the accuracy of HNN is low. In this work, a modified ANN is proposed to overcome this specific problem. The modified ANN is a fusion of BPN and HNN. The technical concepts of BPN and HNN are mixed in the training algorithm of the proposed back propagation‐Hopfield network (BPHN). The objective of this fusion is to improve the performance of conventional ANN. Magnetic resonance brain image classification experiments are used to analyse the proposed BPHN. Experimental results have suggested improvement in the learning process of the proposed BPHN. A comparative analysis with the conventional networks is performed to validate the performance of the proposed approach.     

一种基于FNN的高速网络拥塞控制策略

以ATM（asynchronous transfer mode)为研究对旬,同种基于模糊神经网络（fuzzy neural network,简称FNN）的流量预测和拥塞控制策略,拥塞控制是高速网络（如ATM）研究中的关键问题之一,传统的基于BP神经网络的流量预测方法因其收敛速度较慢且具有较大的误差,影响了拥塞控制效果,而模糊神经网络由于具有处理不确定性问题和很强的学习能力,很好地解决这一问题,最后通过仿真,比较和分析了基于BP神经网络和基于FNN方法和性能,证明此方法是有效的。     

Performance comparison of artificial neural network models for daily rainfall prediction

With an aim to predict rainfall one-day in advance, this paper adopted different neural network models such as feed forward back propagation neural network (BPN), cascade-forward back propagation neural network (CBPN), distributed time delay neural network (DTDNN) and nonlinear autoregressive exogenous network (NARX), and compared their forecasting capabilities. The study deals with two data sets, one containing daily rainfall, temperature and humidity data of Nilgiris and the other containing only daily rainfall data from 14 rain gauge stations located in and around Coonoor (a taluk of Nilgiris). Based on the performance analysis, NARX network outperformed all the other networks. Though there is no major difference in the performances of BPN, CBPN and DTDNN, yet BPN performed considerably well confirming its prediction capabilities. Levenberg Marquardt proved to be the most effective weight updating technique when compared to different gradient descent approaches. Sensitivity analysis was instrumental in identifying the key predictors.     

An efficient document classification model using an improved back propagation neural network and singular value decomposition

This paper proposed a new improved method for back propagation neural network, and used an efficient method to reduce the dimension and improve the performance. The traditional back propagation neural network (BPNN) has the drawbacks of slow learning and is easy to trap into a local minimum, and it will lead to a poor performance and efficiency. In this paper, we propose the learning phase evaluation back propagation neural network (LPEBP) to improve the traditional BPNN. We adopt a singular value decomposition (SVD) technique to reduce the dimension and construct the latent semantics between terms. Experimental results show that the LPEBP is much faster than the traditional BPNN. It also enhances the performance of the traditional BPNN. The SVD technique cannot only greatly reduce the high dimensionality but also enhance the performance. So SVD is to further improve the document classification systems precisely and efficiently.     

基于结构学习和迭代自映射的自联想记忆模型

传统的人工神经元网络连接结构是固定的,是对权值的学习.提出一种基于生理神经元特征的人工神经元模型,并在以此为单元构成的用于实现自联想记忆的神经网络上进行对结构的学习.学习算法以设定神经元的输入/输出感受野、调整突触和轴突末梢的连接、并行的自投影迭代为特征.给出了此网络模型的矩阵描述和实验结果.     

Self-adaptive extreme learning machine

In order to overcome the disadvantage of the traditional algorithm for SLFN (single-hidden layer feedforward neural network), an improved algorithm for SLFN, called extreme learning machine (ELM), is proposed by Huang et al. However, ELM is sensitive to the neuron number in hidden layer and its selection is a difficult-to-solve problem. In this paper, a self-adaptive mechanism is introduced into the ELM. Herein, a new variant of ELM, called self-adaptive extreme learning machine (SaELM), is proposed. SaELM is a self-adaptive learning algorithm that can always select the best neuron number in hidden layer to form the neural networks. There is no need to adjust any parameters in the training process. In order to prove the performance of the SaELM, it is used to solve the Italian wine and iris classification problems. Through the comparisons between SaELM and the traditional back propagation, basic ELM and general regression neural network, the results have proven that SaELM has a faster learning speed and better generalization performance when solving the classification problem.     

Optoelectronic implementation of pulsed neurons and neural networks using bispin-devices

Optoelectronic spiking neuron that is based on bispin-device is described. The neuron has separate optical inputs for excitatory and inhibitory signals, which are represented with pulses of single polarity. Experimental data, which demonstrates similarity in form of output pulses and set of functions of the suggested neuron and a biological one is given. An example of hardware implementation of optoelectronic pulsed neural network (PNN) that is based on proposed neurons is described. Main elements of the neural network are a line of pulsed neurons and a connection array, part of which is made as a spatial light modulator (SLM) with memory. Usage of SLM allows modification of weights of connections in the learning process of the network. It is possible to create adaptive (capable of additional learning and relearning) optoelectronic PNNs with about 2000 neurons.     

A fuzzy back propagation network for output time prediction in a wafer fab

Output time prediction is a critical task to a wafer fab. Three major techniques commonly applied to predict the output time of a wafer lot include multiple-factor linear regression, production simulation, and back propagation networks (BPNs). Among them, a back propagation network has been shown to be the most promising approach for practical applications, considering both prediction accuracy and efficiency. In this study, a fuzzy back propagation network (FBPN) is constructed to incorporate production-control expert judgments in enhancing the performance of an existing crisp back propagation network. Parameters chosen as inputs to the FBPN are no longer considered as of equal importance, but some production-control experts are requested to express their opinions about the importance of each input parameter in predicting the output time with linguistic terms, which can be converted into pre-specified fuzzy numbers, aggregated, and then multiplied to the normalized value of the corresponding input parameter when fed into the FBPN. Subsequently, the arithmetic for triangular fuzzy numbers is applied to deal with all calculations involved in network learning. A defuzzification operation is finally performed to obtain a crisp output time forecast to enhance the practicability of the forecast. A fab production simulator is applied to generate test examples. The results of five test cases showed the FBPN outperformed the crisp BPN in the efficiency respect, including starting with a much smaller initial root mean square error (RMSE) and requiring much fewer epochs for convergence. In the respect of prediction accuracy measured with the minimal RMSE, the performance of the FBPN was slightly better than that of the BPN. In addition, our experiments also showed that it is possible to significantly reduce the RMSE if the fuzzy-valued output is treated as a weighted interval forecast for the actual cycle time.     

Static, Dynamic, and Hybrid Neural Networks in Forecasting Inflation

The back-propagation neural network (BPN) model has been the most popular form of artificial neural network model used for forecasting, particularly in economics and finance. It is a static (feed-forward) model which has a learning process in both hidden and output layers. In this paper we compare the performance of the BPN model with that of two other neural network models, viz., the radial basis function network (RBFN) model and the recurrent neural network (RNN) model, in the context of forecasting inflation. The RBFN model is a hybrid model with a learning process that is much faster than the BPN model and that is able to generate almost the same results as the BPN model. The RNN model is a dynamic model which allows feedback from other layers to the input layer, enabling it to capture the dynamic behavior of the series. The results of the ANN models are also compared with those of the econometric time series models.     

Incorporating the FCM–BPN approach with nonlinear programming for internal due date assignment in a wafer fabrication plant

To enhance the performance of the internal due date assignment in a wafer fab even further, this study incorporated the fuzzy c-means–back propagation network (FCM–BPN) approach with a nonlinear programming model. In the proposed methodology, the jobs are first classified into several categories by fuzzy c-means. Then, an individual back propagation network is constructed for each category to predict the completion time of the jobs. Subsequently, an individual nonlinear programming model is constructed for each back propagation network to adjust the connection weights in the back propagation network, allowing us to determine the internal due dates of the jobs in the category. The nonlinear programming model is finally converted into a goal programming problem that can be solved with existing optimization software. According to the experimental results, the proposed methodology outperforms the baseline multiple linear regression (MLR) approach by 24% in predicting the job completion/cycle times. In addition, the proposed methodology also guarantees that all jobs can be finished before the established internal due dates, without adding too large a fudge factor, and without sacrificing the accuracy of the completion/cycle time forecasts.     

基于改进GA-Elman的无线智能传播损耗预测方法

在5G移动通信系统商用落地的背景下,设计准确、高效的信道估计方法对无线网络性能优化具有重要意义。基于改进GA-Elman算法,提出一种新的无线智能传播损耗预测方法。对Elman神经网络中的连接权值、阈值和隐藏神经元进行实数编码,在隐藏神经元编码中加入二进制控制基因,同时利用自适应遗传算法对权值、阈值和隐藏神经元数量进行优化,解决网络易陷入局部极小值和神经元数目难以确定的问题,从而提高预测性能。仿真结果表明,与仅优化连接权值及阈值的GA-Elman神经网络和标准Elman神经网络相比,该方法具有较高的预测精度。     

Automatic thesaurus construction for spam filtering using revised back propagation neural network

Email has become one of the fastest and most economical forms of communication. Email is also one of the most ubiquitous and pervasive applications used on a daily basis by millions of people worldwide. However, the increase in email users has resulted in a dramatic increase in spam emails during the past few years. This paper proposes a new spam filtering system using revised back propagation (RBP) neural network and automatic thesaurus construction. The conventional back propagation (BP) neural network has slow learning speed and is prone to trap into a local minimum, so it will lead to poor performance and efficiency. The authors present in this paper the RBP neural network to overcome the limitations of the conventional BP neural network. A well constructed thesaurus has been recognized as a valuable tool in the effective operation of text classification, it can also overcome the problems in keyword-based spam filters which ignore the relationship between words. The authors conduct the experiments on Ling-Spam corpus. Experimental results show that the proposed spam filtering system is able to achieve higher performance, especially for the combination of RBP neural network and automatic thesaurus construction.     

Comparison between back propagation neural network and regression models for the estimation of pigment content in rice leaves and panicles using hyperspectral data

Two nitrogen experiments on rice were conducted in 2002, and the reflectances (350 to 2500 nm) and pigment contents (chlorophylls a and b, total chlorophylls and carotenoids) for leaf and panicle samples at different growth stages were measured in the laboratory. After performing an outlier analysis, the number of samples were 843 for leaves and 188 for panicles. Absorption features at 430, 460, 470, 640 and 660 nm for different pigments, and the relative reflectance of the green peak around 550 nm calculated by the continuum‐removed method, as well as the red edge position (REP) of rice leaves and panicles were selected as the independent variables, and measured pigment contents were selected as the dependent variables. Then, back propagation neural network (BPN) models, a kind of artificial neuron network (ANN), and multivariate linear regression models (MLR) were trained and tested. The main objective of this study was to compare the predictive ability of the ANN models to that of the MLR models in estimating the content of pigments in rice leaves and panicles. Results showed that all BPN models gave higher coefficients of determination (R²) and lower absolute errors (ABSEs) and root mean squared errors (RMSEs) than the corresponding MLR models, in both calibration and validation tests. Further significance tests by paired t tests and bootstrapping algorithms indicated that most of the BPN models outperformed the MLR models. When trained by combination data that did not meet the assumption of normal distribution, the BPN models appeared to not only have a better learning ability, but also had a more accurate predictive power than the MLR models. The estimation of leaf pigments was more accurate than that of panicle pigments, independent of which model was used.     

自适应神经网络学习方法研究

本文从连接权值、网络的拓扑结构、网络的学习参数以及神经元的激活特性等不同方面分别讨论了人工神经网络的学习问题，并就当前流行的ＢＰ模型提出了具体实现方法。实验表明，这些方法对于加快网络的收敛速度，优化网络的拓扑结构等方面有着显著成效，本文所述内容为ＡＮＮ学习算法的改进与设计提供了示例，途径和思想总结。     

Fast learning network: a novel artificial neural network with a fast learning speed

This paper proposes a novel artificial neural network called fast learning network (FLN). In FLN, input weights and hidden layer biases are randomly generated, and the weight values of the connection between the output layer and the input layer and the weight values connecting the output node and the input nodes are analytically determined based on least squares methods. In order to test the FLN validity, it is applied to nine regression applications, and experimental results show that, compared with support vector machine, back propagation, extreme learning machine, the FLN with much more compact networks can achieve very good generalization performance and stability at a very fast training speed and a quick reaction of the trained network to new observations. In addition, in order to further test the FLN validity, it is applied to model the thermal efficiency and NO _x emissions of a 330 WM coal-fired boiler and achieves very good prediction precision and generalization ability at a high learning speed.     

An implementation of backpropagation learning on GF11, a large SIMD parallel computer

Current connectionist simulations require huge computational resources. We describe a neural network simulator for the IBM GF11, an experimental SIMD machine with 566 processors and a peak arithmetic performance of 11 Gigaflops. We present our parallel implementation of the backpropagation learning algorithm, techniques for increasing efficiency performance measurements on the NETTALK text-to-speech benchmark, and a performance model for the simulator. Our simulator currently runs the backpropagation learning algorithm at 900 million connections per second, where each “connection per second” includes both a forward and a backward pass. This figure was obtained on the machine when only 356 processors were working; with all 566 processors operational, our simulator will run at over one billion connections per second. We conclude that the GF11 is well-suited to neural network simulation, and we analyze our use of the machine to determine which features are the most important for high performance.     

TurSOM: A paradigm bridging Turing's unorganized machines and self-organizing maps demonstrating dual self-organization

Self-organization is a widely used technique in unsupervised learning and data analysis, largely exemplified by k-means clustering, self-organizing maps (SOM) and adaptive resonance theory.In this paper we present a new algorithm: TurSOM, inspired by Turing's unorganized machines and Kohonen's SOM. Turing's unorganized machines are an early model of neural networks characterized by self-organizing connections, as opposed to self-organizing neurons in SOM.TurSOM introduces three new mechanisms to facilitate both neuron and connection self-organization. These mechanisms are: a connection learning rate, connection reorganization, and a neuron responsibility radius.TurSOM is implemented in a 1-dimensional network (i.e. chain of neurons) to exemplify the theoretical implications of these features. In this paper we demonstrate that TurSOM is superior to the classical SOM algorithm in several ways: (1) speed until convergence; (2) independent clusters; and (3) tangle-free networks.     

基于连接自组织发育的稀疏跨越-侧抑制神经网络设计

针对跨越——侧抑制神经网络（Span-lateral inhibition neural network,S-LINN）的结构调整及参数学习问题,结合生物神经系统中神经元的稀疏连接特性,依据儿童及青少年智力发展水平与大脑皮层发育之间的相互关系,提出以小世界网络连接模式进行初始稀疏化的连接自组织发育稀疏跨越——侧抑制神经网络设计方法.定义网络连接稀疏度及神经元输出贡献率,设计网络连接增长——修剪规则,根据智力超常组皮层发育与智力水平的对应关系调整和控制网络连接权值,动态调整网络连接实现网络智力的自组织发育.通过非线性动力学系统辨识及函数逼近基准问题的求解,证明在同等连接复杂度的情况下,稀疏连接的跨越——侧抑制神经网络具有更好的泛化能力.     

Mining the fuzzy control rules of aeration in a Submerged Biofilm Wastewater Treatment Process

This paper presents a special rule base extraction analysis for optimal design of an integrated neural-fuzzy process controller using an “impact assessment approach.” It sheds light on how to avoid some unreasonable fuzzy control rules by screening inappropriate fuzzy operators and reducing over fitting issues simultaneously when tuning parameter values for these prescribed fuzzy control rules. To mitigate the design efforts, the self-learning ability embedded in the neural networks model was emphasized for improving the rule extraction performance. An aeration unit in an Aerated Submerged Biofilm Wastewater Treatment Process (ASBWTP) was picked up to support the derivation of a solid fuzzy control rule base. Four different fuzzy operators were compared against one other in terms of their actual performance of automated knowledge acquisition in the system based on a partial or full rule base prescribed. Research findings suggest that using bounded difference fuzzy operator (O_b) in connection with back propagation neural networks (BPN) algorithm would be the best choice to build up this feedforward fuzzy controller design.     

Neural networks for advanced control of robot manipulators

Presents an approach and a systematic design methodology to adaptive motion control based on neural networks (NNs) for high-performance robot manipulators, for which stability conditions and performance evaluation are given. The neurocontroller includes a linear combination of a set of off-line trained NNs, and an update law of the linear combination coefficients to adjust robot dynamics and payload uncertain parameters. A procedure is presented to select the learning conditions for each NN in the bank. The proposed scheme, based on fixed NNs, is computationally more efficient than the case of using the learning capabilities of the neural network to be adapted, as that used in feedback architectures that need to propagate back control errors through the model to adjust the neurocontroller. A practical stability result for the neurocontrol system is given. That is, we prove that the control error converges asymptotically to a neighborhood of zero, whose size is evaluated and depends on the approximation error of the NN bank and the design parameters of the controller. In addition, a robust adaptive controller to NN learning errors is proposed, using a sign or saturation switching function in the control law, which leads to global asymptotic stability and zero convergence of control errors. Simulation results showing the practical feasibility and performance of the proposed approach to robotics are given.