An educational tool for artificial neural networks

Artificial neural networks are some kind of data processing systems, which try to simulate features of the human brain and its learning process. So, they are widely used by researchers to solve different problems in optimization, classification, pattern recognition, associative memory and control. In this paper, an educational tool, which can be used to work on different kinds of neural network models and learn fundamentals of the artificial neural network, is described. At this point, the whole tool environment provides an advanced system to ensure mentioned functions. The developed system supports using MLP, LVQ and SOM models and related learning algorithms. It employs some visual, interactive tools, which enable users to compose their own neural networks and work on the developed networks easily. By using these tools, users can also understand and learn working mechanism of a typical artificial neural network, using features of different models and related learning algorithms.     

基于Agent和数据切片的分布式神经网络协同学习研究

针对目前神经网络在处理类似生物信息数据库这类较大规模数据时,遇到的大规模数据处理耗时过长、内存资源不足等问题．在分析当前神经网络分布式学习的基础上,提出了一种新的基于Agent和切片思想的分布式神经网络协同训练算法．通过对训练样本和训练过程的有效切分,整个样本集的学习被分配到一个分布式神经网络集群环境中进行协同训练,同时通过竞争筛选机制,使得学习性能较好的训练个体能有效地在神经网络群中迁移,以获得较多的资源进行学习．理论分析论证了该方法不仅能有效提高神经网络向目标解收敛的成功率,同时也具有较高的并行计算性能,以加快向目标解逼近的速度．最后,该方法被应用到了蛋白质二级结构预测这一生物信息学领域的问题上．结果显示,该分布式学习算法不仅能有效地处理大规模样本集的学习,同时也改进了训练得到的神经网络性能．     

Comparing performances of backpropagation and genetic algorithms in the data classification

Artificial neural networks (ANN) have a wide ranging usage area in the data classification problems. Backpropagation algorithm is classical technique used in the training of the artificial neural networks. Since this algorithm has many disadvantages, the training of the neural networks has been implemented with the binary and real-coded genetic algorithms. These algorithms can be used for the solutions of the classification problems. The real-coded genetic algorithm has been compared with other training methods in the few works. It is known that the comparison of the approaches is as important as proposing a new classification approach. For this reason, in this study, a large-scale comparison of performances of the neural network training methods is examined on the data classification datasets. The experimental comparison contains different real classification data taken from the literature and a simulation study. A comparative analysis on the real data sets and simulation data shows that the real-coded genetic algorithm may offer efficient alternative to traditional training methods for the classification problem.     

Evolution of an artificial neural network based autonomous landvehicle controller

This paper presents an evolutionary method for creating an artificial neural network based autonomous land vehicle controller. The evolved controllers perform better in unseen situations than those trained with an error backpropagation learning algorithm designed for this task. In this paper, an overview of the previous connectionist based approaches to this task is given, and the evolutionary algorithms used in this study are described in detail. Methods for reducing the high computational costs of training artificial neural networks with evolutionary algorithms are explored. Error metrics specific to the task of autonomous vehicle control are introduced; the evolutionary algorithms guided by these error metrics reveal improved performance over those guided by the standard sum-squared error metric. Finally, techniques for integrating evolutionary search and error backpropagation are presented. The evolved networks are designed to control Carnegie Mellon University's NAVLAB vehicles in road following tasks.     

Pre-synaptic lateral inhibition provides a better architecture for self-organizing neural networks

Unsupervised learning is an important ability of the brain and of many artificial neural networks. A large variety of unsupervised learning algorithms have been proposed. This paper takes a different approach in considering the architecture of the neural network rather than the learning algorithm. It is shown that a self-organizing neural network architecture using pre-synaptic lateral inhibition enables a single learning algorithm to find distributed, local, and topological representations as appropriate to the structure of the input data received. It is argued that such an architecture not only has computational advantages but is a better model of cortical self-organization.     

脉冲神经网络:模型、学习算法与应用

脉冲神经网络是目前最具有生物解释性的人工神经网络,是类脑智能领域的核心组成部分.首先介绍各类常用的脉冲神经元模型以及前馈和循环型脉冲神经网络结构;然后介绍脉冲神经网络的时间编码方式,在此基础上,系统地介绍脉冲神经网络的学习算法,包括无监督学习和监督学习算法,其中监督学习算法按照梯度下降算法、结合STDP规则的算法和基于脉冲序列卷积核的算法3大类别分别展开详细介绍和总结;接着列举脉冲神经网络在控制领域、模式识别领域和类脑智能研究领域的应用,并在此基础上介绍各国脑计划中,脉冲神经网络与神经形态处理器相结合的案例;最后分析脉冲神经网络目前所存在的困难和挑战.     

Using the optimization layer-by-layer learning algorithm on local-recurrent-global-feedforward networks in financial time series predictions

In the field of time series prediction, neural networks are widely used and they have been proven useful and practical. To improve the prediction ability and to reduce the time consumption of neural networks, neural networks are usually developed by researchers and practitioners from learning algorithms, network architectures, etc. A local-recurrent-global-feedforward (LRGF) network using a learning algorithm called the optimization layer-by-layer (OLL) method is proposed. In addition, two representative LRGF networks are introduced: the finite impulse response (FIR) network and the FGS network (proposed by Frasconi, Gori and Soda), and a comparative simulation predicting several financial time series using both methods is performed. According to the results of the simulation, the FGS-OLL method gives better predicting performance than the FIR-OLL method.     

基于遗传算法优化投影寻踪技术的神经网络集成模型及其应用

首先利用遗传算法优化的投影寻踪技术对神经网络学习矩阵降维,再利用Bagging技术和不同的神经网络学习算法生成集成个体,并再次用遗传算法进化的投影寻踪技术对神经网络个体集成.建立基于遗传算法优化的投影寻踪技术神经网络集成模型,通过上证指数开盘价、收盘价进行实例分析,计算结果表明该方法具有较好的学习能力和泛化能力,在股市预测中预测精度高、稳定性好.     

小波及混沌学习神经网络在短期电力负荷预测中的应用

该文提出了采用小波和神经网络混合模型进行电力系统短期负荷预测方法。首先基于小波多分辨率分析方法将负荷序列分解成具有不同频率特征的序列。然后,根据分解后的各个分量的特点构造不同的神经网络模型对各分量分别进行预测。神经网络算法采用混沌学习算法,与传统BP算法相比,该算法利用混沌轨道的游动性使系统能够跳出局域极值的束缚而寻求全局最优点,这样克服了BP学习算法所存在的本质问题,可以加快网络学习速度和提高学习精度。最后对各分量预测信号进行重构得到最终预测结果。在构建网络模型时,该文考虑了气候因素的影响,并把它作为网络的一组输入点。实验结果表明基于这一方法的负荷预测系统具有较好的精度及稳定性。     

基于神经网络结构学习的知识求精方法

知识求精是知识获取中必不可少的步骤．已有的用于知识求精的ＫＢＡＮＮ（ｋｎｏｗ ｌｅｄｇｅ ｂａｓｅｄ ａｒｔｉｆｉｃｉａｌｎｅｕｒａｌｎｅｔｗ ｏｒｋ）方法,主要局限性是训练时不能改变网络的拓扑结构．文中提出了一种基于神经网络结构学习的知识求精方法,首先将一组规则集转化为初始神经网络,然后用训练样本和结构学习算法训练初始神经网络,并提取求精的规则知识．网络拓扑结构的改变是通过训练时采用基于动态增加隐含节点和网络删除的结构学习算法实现的．大量实例表明该方法是有效的     

Heuristic principles for the design of artificial neural networks

Artificial neural networks were used to support applications across a variety of business and scientific disciplines during the past years. Artificial neural network applications are frequently viewed as black boxes which mystically determine complex patterns in data. Contrary to this popular view, neural network designers typically perform extensive knowledge engineering and incorporate a significant amount of domain knowledge into artificial neural networks. This paper details heuristics that utilize domain knowledge to produce an artificial neural network with optimal output performance. The effect of using the heuristics on neural network performance is illustrated by examining several applied artificial neural network systems. Identification of an optimal performance artificial neural network requires that a full factorial design with respect to the quantity of input nodes, hidden nodes, hidden layers, and learning algorithm be performed. The heuristic methods discussed in this paper produce optimal or near-optimal performance artificial neural networks using only a fraction of the time needed for a full factorial design.     

A learning automata-based algorithm for determination of the number of hidden units for three-layer neural networks

There is no method to determine the optimal topology for multi-layer neural networks for a given problem. Usually the designer selects a topology for the network and then trains it. Since determination of the optimal topology of neural networks belongs to class of NP-hard problems, most of the existing algorithms for determination of the topology are approximate. These algorithms could be classified into four main groups: pruning algorithms, constructive algorithms, hybrid algorithms and evolutionary algorithms. These algorithms can produce near optimal solutions. Most of these algorithms use hill-climbing method and may be stuck at local minima. In this article, we first introduce a learning automaton and study its behaviour and then present an algorithm based on the proposed learning automaton, called survival algorithm, for determination of the number of hidden units of three layers neural networks. The survival algorithm uses learning automata as a global search method to increase the probability of obtaining the optimal topology. The algorithm considers the problem of optimization of the topology of neural networks as object partitioning rather than searching or parameter optimization as in existing algorithms. In survival algorithm, the training begins with a large network, and then by adding and deleting hidden units, a near optimal topology will be obtained. The algorithm has been tested on a number of problems and shown through simulations that networks generated are near optimal.     

Intelligent computing for numerical treatment of nonlinear prey–predator models

In this study, a new computing paradigm is presented for evaluation of dynamics of nonlinear prey–predator mathematical model by exploiting the strengths of integrated intelligent mechanism through artificial neural networks, genetic algorithms and interior-point algorithm. In the scheme, artificial neural network based differential equation models of the system are constructed and optimization of the networks is performed with effective global search ability of genetic algorithm and its hybridization with interior-point algorithm for rapid local search. The proposed technique is applied to variants of nonlinear prey–predator models by taking different rating factors and comparison with Adams numerical solver certify the correctness for each scenario. The statistical studies have been conducted to authenticate the accuracy and convergence of the design methodology in terms of mean absolute error, root mean squared error and Nash-Sutcliffe efficiency performance indices.     

Assessment of the performances of multilayer perceptron neural networks in comparison with recurrent neural networks and two statistical methods for diagnosing coronary artery disease

Abstract: We aimed to examine the diagnostic performances of multilayer perceptron neural networks (MLPNNs) for predicting coronary artery disease and to compare them with different types of artificial neural network methods, namely recurrent neural networks (RNNs) and two statistical methods (quadratic discriminant analysis (QDA) and logistic regression (LR)). MLPNNs were trained with backpropagation, quick propagation, delta-bar-delta and extended delta-bar-delta algorithms as classifiers; the RNN was trained with the Levenberg–Marquardt algorithm; LR and QDA were used for predicting coronary artery disease. Coronary artery disease was classified with accuracy rates varying from 79.9% to 83.9% by MLPNNs. Even though MLPNNs achieved higher accuracy rates than the statistical methods, LR (73.2%) and QDA (58.4%), their performances were lower compared to the RNN (84.7%). Among the four different types of training algorithms that trained MLPNNs, quick propagation achieved the highest accuracy rate; however, it was lower than the RNN trained with the Levenberg–Marquardt algorithm. RNNs, which demonstrated 84.7% accuracy and 86.5% positive predictive rates, may be a helpful tool in medical decision making for diagnosis of coronary artery disease.     

Artificial Neural Network Approach to Data Matrix Laser Direct Part Marking

Certain applications have recently appeared in industry where a traditional bar code printed on a label will not survive because the item to be tracked has to be exposed to harsh environments. Laser direct-part marking is a manufacturing process used to create permanent marks on a substrate that could help to alleviate this problem. In this research, artificial neural networks were employed to model the laser direct-part marking process of Data Matrix symbols on carbon steel substrates. Several experiments were conducted to study the laser direct-part marking process and to generate data to serve as training, validation and testing data sets in the artificial neural networks modeling process. Two performance measures, mean squared error and correlation coefficient, were utilized to assess the performance of the artificial neural network models. Single-output artificial neural network models corresponding to four performance measures specific to the Data Matrix bar code symbology were found to have good learning and predicting capabilities. The single-output artificial neural network models were compared to equivalent multiple linear regression models for validation purposes. The prediction capability of the single-output artificial neural network models with respect to laser direct-part marking of Data Matrix symbols on carbon steel substrates was superior to that of the multiple linear regression models.     

基于深度学习的网络流量预测研究综述

精准地预判网络流量变化趋势可以帮助运营商准确预估网络的使用情况,合理分配并高效利用网络资源,以满足日益增长且多样化的用户需求。以深度学习算法在网络流量预测领域的进展为线索,阐述了网络流量预测的评价指标和目前公开的网络流量数据集及应用,具体分析了网络流量预测中常用的深度信念网络、卷积神经网络、循环神经网络和长短时记忆网络共四种深度学习方法,并重点介绍了近年来针对不同问题所提出的改进神经网络模型,总结了各模型特点及应用场景。最后对网络流量预测未来发展进行了展望。     

图神经网络前沿进展与应用

图结构数据是现实生活中广泛存在的一类数据形式.宏观上的互联网、知识图谱、社交网络数据,微观上的蛋白质、化合物分子等都可以用图结构来建模和表示.由于图结构数据的复杂性和异质性,对图结构数据的分析和处理一直是研究界的难点和重点.图神经网络(Graph Neural Network,GNN)是近年来出现的一种利用深度学习直接对图结构数据进行学习的框架,其优异的性能引起了学者高度的关注和深入的探索.通过在图中的节点和边上制定一定的策略,GNN将图结构数据转化为规范而标准的表示,并输入到多种不同的神经网络中进行训练,在节点分类、边信息传播和图聚类等任务上取得优良的效果.与其他图学习算法相比较,GNN能够学习到图结构数据中的节点以及边的内在规律和更加深层次的语义特征.由于具有对图结构数据强大的非线性拟合能力,因此在不同领域的图相关问题上,GNN都表现出更高的准确率和更好的鲁棒性.本文在现有GNN研究的基础上,首先概述了GNN的出现历程,并介绍了相关概念和定义.之后本文着重讨论和对比了GNN中的各种算法框架,包括核心思想、任务划分、学习方式、优缺点、适用范围、实现成本等.此外,本文对GNN算法在多个不同领域下的应用场景进行了详细的阐述,将GNN与其他图学习算法的优缺点作了联系和比较.针对存在的一些问题和挑战,本文勾画了GNN的未来方向和发展趋势,最后对全文进行了全面而细致的总结.     

Use of artificial neural networks for estimation of agricultural wheel traction force in soil bin

This paper presents an application of artificial neural networks (ANNs) for the prediction of traction force using readily available datasets experimentally obtained from a soil bin utilizing single-wheel tester. Aiming this, firstly the tests were carried out using two soil textures and two tire types as affected by velocity, slippage, tire inflation pressure, and wheel load. On this basis, the potential of neural modeling was assessed with multilayered perceptron networks using various training algorithms among which, backpropagation algorithm was compared to backpropagation with declining learning rate factor algorithm due to their primarily yielded superior performance. The results divulged that the latter one could better achieve the aim of study in terms of performance criteria. Furthermore, it was inferred that ANNs could reliably provide a promising tool for prediction of traction force and its modeling.     

A globally convergent learning algorithm for PCA neural networks

Principal component analysis (PCA) by neural networks is one of the most frequently used feature extracting methods. To process huge data sets, many learning algorithms based on neural networks for PCA have been proposed. However, traditional algorithms are not globally convergent. In this paper, a new PCA learning algorithm based on cascade recursive least square (CRLS) neural network is proposed. This algorithm can guarantee the network weight vector converges to an eigenvector associated with the largest eigenvalue of the input covariance matrix globally. A rigorous mathematical proof is given. Simulation results show the effectiveness of the algorithm.     

Combining seasonal time series ARIMA method and neural networks with genetic algorithms for predicting the production value of the mechanical industry in Taiwan

Supplying industrial firms with an accurate method of forecasting the production value of the mechanical industry to facilitate decision makers in precise planning is highly desirable. Numerous methods, including the autoregressive integrated-moving average (ARIMA) model and artificial neural networks can make accurate forecasts based on historical data. The seasonal ARIMA (SARIMA) model and artificial neural networks can also handle data involving trends and seasonality. Although neural networks can make predictions, deciding the most appropriate input data, network structure and learning parameters are difficult. Therefore, this article presents a hybrid forecasting method that combines the SARIMA model and neural networks with genetic algorithms. Analytical results generated by the SARIMA model are inputted as the input data of a neural network. Subsequently, the number of neurons in the hidden layer and the number of learning parameters of the neural network architecture are globally optimized using genetic algorithms. This model is subsequently adopted to forecast seasonal time series data of the production value of the mechanical industry in Taiwan. The results presented here provide a valuable reference for decision makers in industry.