Optimization free neural network approach for solving ordinary and partial differential equations

Current work introduces a fast converging neural network-based approach for solution of ordinary and partial differential equations. Proposed technique eliminates the need of time-consuming optimization procedure for training of neural network. Rather, it uses the extreme learning machine algorithm for calculating the neural network parameters so as to make it satisfy the differential equation and associated boundary conditions. Various ordinary and partial differential equations are treated using this technique, and accuracy and convergence aspects of the procedure are discussed.

     

极限学习机集成在骨髓细胞分类中的应用

骨髓细胞的分类有重要的医学诊断意义。先对骨髓细胞图像分割和特征提取,用提取出来的训练集对极限学习机训练,再用该分类器对未知样本识别。针对单个分类器性能的不稳定,提出基于元胞自动机的极限学习机集成算法。通过元胞自动机抽样策略构建差异大的训练子集,多个分类器并行学习,多数投票法联合决策。实验结果表明,与BP、支持向量机比较,该算法基本无参数调整,学习速度快,分类精度高能达到97.33%,且有效克服了神经网络分类器不稳定的缺点。     

Prediction of throw in bench blasting using neural networks: an approach

Tailoring the muckpile shape and its fragmentation to the requirements of the excavating equipment in surface mines can significantly improve the efficiency and savings through increased production, machine life and reduced maintenance. Considering the various blast parameters together to predict the throw is subtle and can lead to wrong conclusions. In this paper, a different approach was followed to combine the representational power of multilayer neural networks and various machine learning techniques to predict the throw of a bench blast using the data from a limestone mine located in central India. Then, using various analysis techniques, the training parameters have been adjusted to reduce the cross-validation error and increase the accuracy. Here, four different architectures of neural networks have been trained by different techniques, and the best model has been selected. The different machine learning techniques have been implemented on the basis of accuracy of the output. The sensitivity analysis has been done to get the relative importance of the variables in prediction of the output.

     

Improving learning and generalization capabilities of the C-Mantec constructive neural network algorithm

C-Mantec neural network constructive algorithm Ortega (C-Mantec neural network algorithm implementation on MATLAB. https://github.com/IvanGGomez/CmantecPaco, 2015) creates very compact architectures with generalization capabilities similar to feed-forward networks trained by the well-known back-propagation algorithm. Nevertheless, constructive algorithms suffer much from the problem of overfitting, and thus, in this work the learning procedure is first analyzed for networks created by this algorithm with the aim of trying to understand the training dynamics that will permit optimization possibilities. Secondly, several optimization strategies are analyzed for the position of class separating hyperplanes, and the results analyzed on a set of public domain benchmark data sets. The results indicate that with these modifications a small increase in prediction accuracy of C-Mantec can be obtained but in general this was not better when compared to a standard support vector machine, except in some cases when a mixed strategy is used.

     

一种卷积神经网络和极限学习机相结合的人脸识别方法

卷积神经网络是一种很好的特征提取器,但却不是最佳的分类器,而极限学习机能够很好地进行分类,却不能学习复杂的特征,根据这两者的优点和缺点,将它们结合起来,提出一种新的人脸识别方法。卷积神经网络提取人脸特征,极限学习机根据这些特征进行识别。本文还提出固定卷积神经网络的部分卷积核以减少训练参 数,从而提高识别精度的方法。在人脸库ORL和XM2VTS上进行测试的结果表明,本文的结合方法能有效提高人脸识别的识别率,而且固定部分卷积核的方式在训练样本少时具有优势。     

A Hopfield neural network with multi-compartmental activation

The Hopfield network is a form of recurrent artificial neural network. To satisfy demands of artificial neural networks and brain activity, the networks are needed to be modified in different ways. Accordingly, it is the first time that, in our paper, a Hopfield neural network with piecewise constant argument of generalized type and constant delay is considered. To insert both types of the arguments, a multi-compartmental activation function is utilized. For the analysis of the problem, we have applied the results for newly developed differential equations with piecewise constant argument of generalized type beside methods for differential equations and functional differential equations. In the paper, we obtained sufficient conditions for the existence of an equilibrium as well as its global exponential stability. The main instruments of investigation are Lyapunov functionals and linear matrix inequality method. Two examples with simulations are given to illustrate our solutions as well as global exponential stability.

     

一种新的混合智能极限学习机

提出一种基于差分进化(DE)和粒子群优化(PSO)的混合智能方法—–DEPSO算法,并通过对10个典型函数进行测试,表明DEPSO算法具有良好的寻优性能。针对单隐层前向神经网络(SLFNs)提出一种改进的学习算法—–DEPSO-ELM算法,即应用DEPSO算法优化SLFNs的隐层节点参数,采用极限学习算法(ELM)求取SLFNs的输出权值。将DEPSO-ELM算法应用于6个典型真实数据集的回归计算,并与DE-ELM、SaE-ELM算法相比,获得了更精确的计算结果。最后,将DEPSO-ELM算法应用于数控机床热误差的建模预测,获得了良好的预测效果。     

Adaptive Control of Discrete-time Nonlinear Systems Using ITF-ORVFL

Random vector functional ink(RVFL)networks belong to a class of single hidden layer neural networks in which some parameters are randomly selected.Their network structure in which contains the direct links between inputs and outputs is unique,and stability analysis and real-time performance are two difficulties of the control systems based on neural networks.In this paper,combining the advantages of RVFL and the ideas of online sequential extreme learning machine(OS-ELM)and initial-training-free online extreme learning machine(ITFOELM),a novel online learning algorithm which is named as initial-training-free online random vector functional link algo rithm(ITF-ORVFL)is investigated for training RVFL.The link vector of RVFL network can be analytically determined based on sequentially arriving data by ITF-ORVFL with a high learning speed,and the stability for nonlinear systems based on this learning algorithm is analyzed.The experiment results indicate that the proposed ITF-ORVFL is effective in coping with nonparametric uncertainty.     

Image recognition with an analog neural net chip

We developed a machine vision system around an analog neural net chip and used it in several applications. Some of them were: locating the address blocks on mail pieces, finding the identification numbers on rail cars, and discriminating between handwritten and machine-printed characters. The chip, operating as a coprocessor of a workstation, provides a speed-up of a factor of 1000, compared with the workstation. The computation speed achieved lies between one and ten billion multiply-accumulates/s. The neural net chip is based on building blocks,neurons, that can be arranged in various network architectures. The dataflow is optimized for implementing large, structured neural nets, and is also suited for any task in which signals are to be convolved with many kernels. Some of the networks are trained on the neural net chip with a weight-perturbation learning algorithm that was adapted to work with the coarse quantization of the weights and the states in the chip.     

Composite Function Wavelet Neural Networks with Differential Evolution and Extreme Learning Machine

In this paper, we introduce a new learning method for composite function wavelet neural networks (CFWNN) by combining the differential evolution (DE) algorithm with extreme learning machine (ELM), in short, as CWN-E-ELM. The recently proposed CFWNN trained with ELM (CFWNN-ELM) has several promising features. But the CFWNN-ELM may have some redundant nodes due to the number of hidden nodes assigned a priori and the input weight matrix and the hidden node parameter vector randomly generated once and never changed during the learning phase. The introduction of DE into CFWNN-ELM is to search for the optimal network parameters and to reduce the number of hidden nodes used in the network. Simulations on several artificial function approximations, real-world data regressions and a chaotic signal prediction problem show some advantages of the proposed CWN-E-ELM. Compared with CFWNN-ELM, CWN-E-ELM has a much more compact network size and Compared with several relevant methods, CWN-E-ELM is able to achieve a better generalization performance.     

Ensemble weighted extreme learning machine for imbalanced data classification based on differential evolution

Extreme learning machine for single-hidden-layer feedforward neural networks has been extensively applied in imbalanced data learning due to its fast learning capability. Ensemble approach can effectively improve the classification performance by combining several weak learners according to a certain rule. In this paper, a novel ensemble approach on weighted extreme learning machine for imbalanced data classification problem is proposed. The weight of each base learner in the ensemble is optimized by differential evolution algorithm. Experimental results on 12 datasets show that the proposed method could achieve more classification performance compared with the simple vote-based ensemble method and non-ensemble method.

     

不同池化模型的卷积神经网络学习性能研究

目的 基于卷积神经网络的深度学习算法在图像处理领域正引起广泛关注。为了进一步提高卷积神经网络特征提取的准确度,加快参数收敛速度,优化网络学习性能,通过对比不同的池化模型对学习性能的影响提出一种动态自适应的改进池化算法。方法 构建卷积神经网络模型,使用不同的池化模型对网络进行训练,并检验在不同迭代次数下的学习结果。在现有算法准确率不高和收敛速度较慢的情况下,通过使用不同的池化模型对网络进行训练,从而构建一种新的动态自适应池化模型,并研究在不同迭代次数下其对识别准确率和收敛速度的影响。结果 通过对比实验发现,使用动态自适应池化算法的卷积神经网络学习性能最优,在手写数字集上的收敛速度最高可以提升18.55%,而模型对图像的误识率最多可以降低20%。结论 动态自适应池化算法不但使卷积神经网络对特征的提取更加精确,而且很大程度地提高了收敛速度和模型准确率,从而达到优化网络学习性能的目的。这种模型可以进一步拓展到其他与卷积神经网络相关的深度学习算法。     

Wind power ramp event detection with a hybrid neuro-evolutionary approach

In this paper, a hybrid system for wind power ramp events (WPREs) detection is proposed. The system is based on modeling the detection problem as a binary classification problem from atmospheric reanalysis data inputs. Specifically, a hybrid neuro-evolutionary algorithm is proposed, which combines artificial neural networks such as extreme learning machine (ELM), with evolutionary algorithms to optimize the trained models and carry out a feature selection on the input variables. The phenomenon under study occurs with a low probability, and for this reason the classification problem is quite unbalanced. Therefore, is necessary to resort to techniques focused on providing a balance in the classes, such as the synthetic minority over-sampling technique approach, the model applied in this work. The final model obtained is evaluated by a test set using both ELM and support vector machine algorithms, and its accuracy performance is analyzed. The proposed approach has been tested in a real problem of WPREs detection in three wind farms located in different areas of Spain, in order to see the spatial generalization of the method.

     

Self-Adaptive Evolutionary Extreme Learning Machine

In this paper, we propose an improved learning algorithm named self-adaptive evolutionary extreme learning machine (SaE-ELM) for single hidden layer feedforward networks (SLFNs). In SaE-ELM, the network hidden node parameters are optimized by the self-adaptive differential evolution algorithm, whose trial vector generation strategies and their associated control parameters are self-adapted in a strategy pool by learning from their previous experiences in generating promising solutions, and the network output weights are calculated using the Moore?CPenrose generalized inverse. SaE-ELM outperforms the evolutionary extreme learning machine (E-ELM) and the different evolutionary Levenberg?CMarquardt method in general as it could self-adaptively determine the suitable control parameters and generation strategies involved in DE. Simulations have shown that SaE-ELM not only performs better than E-ELM with several manually choosing generation strategies and control parameters but also obtains better generalization performances than several related methods.     

The extreme learning machine learning algorithm with tunable activation function

In this paper, we propose an extreme learning machine (ELM) with tunable activation function (TAF-ELM) learning algorithm, which determines its activation functions dynamically by means of the differential evolution algorithm based on the input data. The main objective is to overcome the problem dependence of fixed slop of the activation function in ELM. We mainly considered the issue of processing of benchmark problems on function approximation and pattern classification. Compared with ELM and E-ELM learning algorithms with the same network size or compact network configuration, the proposed algorithm has improved generalization performance with good accuracy. In addition, the proposed algorithm also has very good performance in the TAF neural networks learning algorithms.     

基于Boltzmann 机神经网络认知机制的机器人趋光控制

针对移动机器人未知环境下的趋光控制问题,模拟人或动物“感知-行动”认知机制,对具有趋光特性的移动机器人进行设计,提出一种基于Boltzmann机神经网络的趋光控制方法。该方法首先应用知识集对机器人趋光控制器的Boltzmann机神经网络进行趋光训练;然后应用Boltzmann机神经网络的运行机制实现趋光控制。仿真实验表明,该方法能够提高机器人学习的控制精度。     

深度神经网络结构搜索综述

深度神经网络在图像识别、语言识别和机器翻译等人工智能任务中取得了巨大进展,很大程度上归功于优秀的神经网络结构设计。神经网络大都由手工设计,需要专业的机器学习知识以及大量的试错。为此,自动化的神经网络结构搜索成为研究热点。神经网络结构搜索（neural architecture search,NAS）主要由搜索空间、搜索策略与性能评估方法3部分组成。在搜索空间设计上,出于计算量的考虑,通常不会搜索整个网络结构,而是先将网络分成几块,然后搜索块中的结构。根据实际情况的不同,可以共享不同块中的结构,也可以对每个块单独搜索不同的结构。在搜索策略上,主流的优化方法包含强化学习、进化算法、贝叶斯优化和基于梯度的优化等。在性能评估上,为了节省计算时间,通常不会将每一个网络都充分训练到收敛,而是通过权值共享、早停等方法尽可能减小单个网络的训练时间。与手工设计的网络相比,神经网络结构搜索得到的深度神经网络具有更好的性能。在ImageNet分类任务上,与手工设计的MobileNetV2相比,通过神经网络结构搜索得到的MobileNetV3减少了近30%的计算量,并且top-1分类精度提升了3.2%;在Cityscapes语义分割任务上,与手工设计的DeepLabv3+相比,通过神经网络结构搜索得到的Auto-DeepLab-L可以在没有ImageNet预训练的情况下,达到比DeepLabv3+更高的平均交并比（mean intersection over union,mIOU）,同时减小一半以上的计算量。神经网络结构搜索得到的深度神经网络通常比手工设计的神经网络有着更好的表现,是未来神经网络设计的发展趋势。     

Solving Partial Differential Equation Based on Bernstein Neural Network and Extreme Learning Machine Algorithm

In this paper, we introduce a new method based on Bernstein Neural Network model (BeNN) and extreme learning machine algorithm to solve the differential equation. In the proposed method, we develop a single-layer functional link BeNN, the hidden layer is eliminated by expanding the input pattern by Bernstein polynomials. The network parameters are obtained by solving a system of linear equations using the extreme learning machine algorithm. Finally, the numerical experiment is carried out by MATLAB, results obtained are compared with the existing method, which proves the feasibility and superiority of the proposed method.     

基于差分进化的ELM加权集成分类

集成分类通过将若干个弱分类器依据某种规则进行组合,能有效改善分类性能。在组合过程中,各个弱分类器对分类结果的重要程度往往不一样。极限学习机是最近提出的一个新的训练单隐层前馈神经网络的学习算法。以极限学习机为基分类器,提出了一个基于差分进化的极限学习机加权集成方法。提出的方法通过差分进化算法来优化集成方法中各个基分类器的权值。实验结果表明,该方法与基于简单投票集成方法和基于Adaboost集成方法相比,具有较高的分类准确性和较好的泛化能力。     

Artificial neural networks for job shop simulation

This paper explores the use of artificial neural networks (ANNs) as a valid alternative to the traditional job-shop simulation approach. Feed forward, multi-layered neural network metamodels were trained through the back-error-propagation (BEP) learning algorithm to provide a versatile job-shop scheduling analysis framework. The constructed neural network architectures were capable of satisfactorily estimating the manufacturing lead times (MLT) for orders simultaneously processed in a four-machine job shop. The MLTs produced by the developed ANN models turned out to be as valid as the data generated from three well-known simulation packages, i.e. Arena, SIMAN, and ProModel. The ANN outputs proved not to be substantially different from the results provided by other valid models such as SIMAN and ProModel when compared against the adopted baseline, Arena. The ANN-based simulations were able to fairly capture the underlying relationship between jobs' machine sequences and their resulting average flowtimes, which proves that ANNs are a viable tool for stochastic simulation metamodeling.