基于半监督学习的动态神经网络结构设计

针对神经网络初始结构的设定依赖于工作者的经验、自适应能力较差等问题,提出一种基于半监督学习(SSL)算法的动态神经网络结构设计方法。该方法采用半监督学习方法利用已标记样例和无标记样例对神经网络进行训练,得到一个性能较为完善的初始网络结构,之后采用全局敏感度分析法(GSA)对网络隐层神经元输出权值进行分析,判断隐层神经元对网络输出的影响程度,即其敏感度值大小,适时地删减敏感度值很小的神经元或增加敏感度值较大的神经元,实现动态神经网络结构的优化设计,并给出了网络结构变化过程中收敛性的证明。理论分析和Matlab仿真实验表明,基于SSL算法的神经网络隐层神经元会随训练时间而改变,实现了网络结构动态设计。在液压厚度自动控制(AGC)系统应用中,大约在160 s时系统输出达到稳定,输出误差大约为0.03 mm,与监督学习(SL)方法和无监督学习(USL)方法相比,输出误差分别减小了0.03 mm和0.02 mm,这表明基于SSL算法的动态网络在实际应用中能有效提高系统输出的准确性。     

Artificial neural network based approach for dynamic parameter design

The Taguchi parameter design method has been recognized as an important tool for improving the quality of a product or a process. However, the statistical methods and optimization procedures proposed by Taguchi have much room for improvement. For instance, the two-step procedure proposed by Taguchi may fail to identify an optimum design condition if an adjustment parameter does not exist, the optimal setting of a design parameter is determined only among the levels included in the parameter design experiment, and, for the dynamic parameter design, the signal parameter is assumed to follow a uniform rather than a general distribution. This paper develops an artificial neural network based dynamic parameter design approach to overcome the shortcomings of the Taguchi and existing alternative approaches. First, an artificial neural network is trained to map the relationship between the characteristic, design, noise and signal parameters. Second, Latin hypercube samples of the signal and noise parameters are obtained and used to estimate the slope between the signal parameter and characteristic as well as the variance of the characteristic at each set of design parameter settings. Then, the dynamic parameter design problem is formulated as a nonlinear optimization problem and solved to find the optimal settings of the design parameters using sequential quadratic programming. The effectiveness of the proposed approach is illustrated with an example.     

A reinforcement learning approach based on the fuzzy min-max neural network

The fuzzy min-max neural network constitutes a neural architecture that is based on hyperbox fuzzy sets and can be incrementally trained by appropriately adjusting the number of hyperboxes and their corresponding volumes. Two versions have been proposed: for supervised and unsupervised learning. In this paper a modified approach is presented that is appropriate for reinforcement learning problems with discrete action space and is applied to the difficult task of autonomous vehicle navigation when no a priori knowledge of the enivronment is available. Experimental results indicate that the proposed reinforcement learning network exhibits superior learning behavior compared to conventional reinforcement schemes.     

A nonlinear sliding mode control design approach based on neural network modelling

A complete nonlinear framework for the modelling and robust control of nonlinear systems is proposed. The use of neural networks for continuous time modelling to obtain a certain nonlinear canonical form is investigated. The model obtained is used with recently proposed dynamic sliding mode controller design methods. The robustness bounds needed for controller design are determined from modelling errors. A modified version of the backpropagation theorem is also introduced. Copyright © 1999 John Wiley & Sons, Ltd.     

一种基于代数算法的RBF神经网络优化方法

提出了一种新的RBF神经网络的训练方法,采用动态K-均值方法对RBF 神经网络的隐层中心值和宽度进行了优化,用代数算法训练隐层和输出层之间的权值。在对非线性函数进行逼近的仿真中,验证了该算法的有效性。     

基于样本重构的神经网络集成学习方法

在回顾以往神经网络集成的研究成果基础上,提出一种新的负相关学习方法,该方法易于执行,计算量小,有效的消除了学习中的复合线性问题,减小了集成误差,最后用测试用例对该方法进行了考察,证明该方法可以有效的降低集成预测误差,得到较为理想的集成效果。     

基于EKF的模糊神经网络快速自组织学习算法*

为了快速地构造一个有效的模糊神经网络,提出一种基于扩展卡尔曼滤波(EKF)的模糊神经网络自组织学习算法。在本算法中,按照提出的无须经过修剪过程的生长准则增加规则,加速了网络在线学习过程;使用EKF算法更新网络的自由参数,增强了网络的鲁棒性。仿真结果表明,该算法具有快速的学习速度、良好的逼近精度和泛化能力。     

Deterministic learning and nonlinear observer design

A “deterministic learning” (DL) theory was recently proposed for identification of nonlinear system dynamics under full‐state measurements. In this paper, for a class of nonlinear systems undergoing periodic or recurrent motions with only output measurements, firstly, it is shown that locally‐accurate identification of nonlinear system dynamics can still be achieved. Specifically, by using a high gain observer and a dynamical radial basis function network (RBFN), when state estimation is achieved by the high gain observer, along the estimated state trajectory, a partial persistence of excitation (PE) condition is satisfied, and locally‐accurate identification of system dynamics is achieved in a local region along the estimated state trajectory. Secondly, by embedding the learned knowledge of system dynamics into a RBFN‐based nonlinear observer, it is shown that correct state estimation can be achieved according to the internal matching of the underlying system dynamics, rather than by using high gain domination. The significance of this paper is that it reveals that the difficult problems in nonlinear observer design can be successfully resolved by incorporating the deterministic learning mechanisms. Simulation studies are included to demonstrate the effectiveness of the approach. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Design of an auto-associative neural network by using design of experiments approach

Advanced monitoring systems enable integration of data-driven algorithms for various tasks, for e.g., control, decision support, fault detection and isolation (FDI), etc. Due to improvement of monitoring systems, statistical or other computational methods can be implemented to real industrial systems. Algorithms which rely on process history data sets are promising for real-time operation especially for online process monitoring tasks, e.g., FDI. However, a reliable FDI system should be robust to uncertainties and small process deviations, thus, false alarms can be avoided. To achieve this, a good model for comparison between process and model is needed and for easier FDI implementation, the model has to be derived directly from process history data. In such cases, model-based FDI approaches are not very practical. In this paper a nonlinear statistical multivariate method (nonlinear principal component analysis) was used for modeling, and realized with auto-associative artificial neural network (AANN). A Taguchi design of experiments (DoE) technique was used and compared with a classic approach, where according to the analysis best AANN model structure was chosen for nonlinear model. Parameters that are important for neural network’s performance have been included into a joint orthogonal array to consider interactions between noise and control process variables. Results are compared to AANN design recommendations by other authors, where obtained nonlinear model was designed for reliable fault detection of very small faults under closed-loop conditions. By using Taguchi DoE robust design on AANN, an improved and reliable FDI scheme was achieved even in case of small faults introduced to the system. The accuracy and performance of AANN and FDI scheme were tested by experiments carried out on a real laboratory hydraulic system, to validate the proposed design for industrial cases.     

Artificial neural network approach for LNA design of GPS receiver

Paper presents an ANN modeling of microwave LNA for the global positioning front end receiver, operating at 1.57542 GHz. To design LNA, multilayer perceptron architecture is used. The scattering parameters of LNA are calculated using Levenberg Marquardt Backpropagation Algorithm for the frequency range 100 MHz to 8 GHz. The inputs given to this architecture are drain to source current, drain to source voltage, temperature and frequency and the outputs are maximum available gain, noise figure and scattering parameters (magnitude as well as angle). ANN model is trained using Agilent MGA 72543 GaAs pHEMT Low Noise Amplifier datasheet and this model shows high regression. The smith and polar charts are plotted for frequency range 100 MHz to 8 GHz.     

Adaptive sliding mode approach for learning in a feedforward neural network

An adaptive learning algorithm is proposed for a feedforward neural network. The design principle is based on the sliding mode concept. Unlike the existing algorithms, the adaptive learning algorithm developed does not require a prioriknowledge of upper bounds of bounded signals. The convergence of the algorithm is established and conditions given. Simulations are presented to show the effectiveness of the algorithm.     

基于自编码神经网络的半监督联邦学习模型

联邦学习是一种新型的分布式机器学习方法,可以使得各客户端在不分享隐私数据的前提下共同建立共享模型。然而现有的联邦学习框架仅适用于监督学习,即默认所有客户端数据均带有标签。由于现实中标记数据难以获取,联邦学习模型训练的前提假设通常很难成立。为解决此问题,对原有联邦学习进行扩展,提出了一种基于自编码神经网络的半监督联邦学习模型ANN-SSFL,该模型允许无标记的客户端参与联邦学习。无标记数据利用自编码神经网络学习得到可被分类的潜在特征,从而在联邦学习中提供无标记数据的特征信息来作出自身贡献。在MNIST数据集上进行实验,实验结果表明,提出的ANN-SSFL模型实际可行,在监督客户端数量不变的情况下,增加无监督客户端可以提高原有联邦学习精度。     

基于深度神经网络的少样本学习综述

如何从少数训练样本中学习并识别新的类别对于深度神经网络来说是一个具有挑战性的问题。针对如何解决少样本学习的问题,全面总结了现有的基于深度神经网络的少样本学习方法,涵盖了方法所用模型、数据集及评估结果等各个方面。具体地,针对基于深度神经网络的少样本学习方法,提出将其分为数据增强方法、迁移学习方法、度量学习方法和元学习方法四种类别;对于每个类别,进一步将其分为几个子类别,并且在每个类别与方法之间进行一系列比较,以显示各种方法的优劣和各自的特点。最后强调了现有方法的局限性,并指出了少样本学习研究领域未来的研究方向。     

Multiple graph kernel learning based on GMDH-type neural network

Multiple kernel learning (MKL), as a principled classification method, selects and combines base kernels to increase the categorization accuracy of Support Vector Machines (SVMs). The group method of data handling neural network (GMDH-NN) has been applied in many fields of optimization, data mining, and pattern recognition. It can automatically seek interrelatedness in data, select an optimal structure for the model or network, and enhance the accuracy of existing algorithms. We can utilize the advantages of the GMDH-NN to build a multiple graph kernel learning (MGKL) method and enhance the categorization performance of graph kernel SVMs. In this paper, we first define a unitized symmetric regularity criterion (USRC) to improve the symmetric regularity criterion of GMDH-NN. Second, a novel structure for the initial model of the GMDH-NN is defined, which uses the posterior probability output of graph kernel SVMs. We then use a hybrid graph kernel in the $H^1$-space for MGKL in combination with the GMDH-NN. This way, we can obtain a pool of optimal graph kernels with different kernel parameters. Our experiments on standard graph datasets show that this new MGKL method is highly effective.     

基于字典学习的卷积神经网络压缩方法

为解决深度卷积神经网络模型占用存储空间较大的问题,提出一种基于K-SVD字典学习的卷积神经网络压缩方法。用字典中少数原子的线性组合来近似表示单个卷积核的参数,对原子的系数进行量化,存储卷积核参数时,只须存储原子的索引及其量化后的系数,达到模型压缩的目的。在MNIST数据集上对LeNet-C5和CIFAR-10数据集上对DenseNet的压缩实验结果表明,在准确率波动不足0.1%的情况下,将网络模型占用的存储空间降低至12%左右。     

基于迭代学习和神经网络的船用起重机控制

作为一种重要的海上作业装备,船用起重机被广泛应用于海洋工程的各类场景中.然而,船用起重机是一类复杂的非线性欠驱动系统,存在摩擦、未建模动态等干扰,为控制器设计带来了巨大挑战.更糟糕的是,船用起重机还面临海浪、大风等未知干扰的影响,使得实际控制更加困难.如何稳定高效地控制该类系统,目前仍处于初步探索阶段.为了解决上述问题,本文提出了一种基于迭代学习和神经网络的控制方法.具体来说,首先将未知干扰分为周期与非周期两部分.对于周期干扰,利用周期估计器解决了对未知周期的估计问题,在此基础上通过迭代学习对干扰进行补偿;对于非周期干扰,使用双层神经网络进行逼近和补偿,并设计了权重的更新律;在补偿未知干扰后,基于反馈线性化设计了控制输入.通过Lyapunov分析方法,可以证明期望平衡点是全局有界的.最后,在所搭建的船吊实验平台上进行了大量实验,充分验证了所设计控制方法的有效性与鲁棒性.     

Attention-based convolutional neural network deep learning approach for robust malware classification

Recently, transforming windows files into images and its analysis using machine learning and deep learning have been considered as a state-of-the art works for malware detection and classification. This is mainly due to the fact that image-based malware detection and classification is platform independent, and the recent surge of success of deep learning model performance in image classification. Literature survey shows that convolutional neural network (CNN) deep learning methods are successfully employed for image-based windows malware classification. However, the malwares were embedded in a tiny portion in the overall image representation. Identifying and locating these affected tiny portions is important to achieve a good malware classification accuracy. In this work, a multi-headed attention based approach is integrated to a CNN to locate and identify the tiny infected regions in the overall image. A detailed investigation and analysis of the proposed method was done on a malware image dataset. The performance of the proposed multi-headed attention-based CNN approach was compared with various non-attention-CNN-based approaches on various data splits of training and testing malware image benchmark dataset. In all the data-splits, the attention-based CNN method outperformed non-attention-based CNN methods while ensuring computational efficiency. Most importantly, most of the methods show consistent performance on all the data splits of training and testing and that illuminates multi-headed attention with CNN model's generalizability to perform on the diverse datasets. With less number of trainable parameters, the proposed method has achieved an accuracy of 99% to classify the 25 malware families and performed better than the existing non-attention based methods. The proposed method can be applied on any operating system and it has the capability to detect packed malware, metamorphic malware, obfuscated malware, malware family variants, and polymorphic malware. In addition, the proposed method is malware file agnostic and avoids usual methods such as disassembly, de-compiling, de-obfuscation, or execution of the malware binary in a virtual environment in detecting malware and classifying malware into their malware family.     

A hybrid multi-objective approach based on the genetic algorithm and neural network to design an incremental cellular manufacturing system

One important issue related to the implementation of cellular manufacturing systems (CMSs) is to decide whether to convert an existing job shop into a CMS comprehensively in a single run, or in stages incrementally by forming cells one after the other, taking the advantage of the experiences of implementation. This paper presents a new multi-objective nonlinear programming model in a dynamic environment. Furthermore, a novel hybrid multi-objective approach based on the genetic algorithm and artificial neural network is proposed to solve the presented model. From the computational analyses, the proposed algorithm is found much more efficient than the fast non-dominated sorting genetic algorithm (NSGA-II) in generating Pareto optimal fronts.     

Content based audio classification: a neural network approach

Content based music genre classification is a key component for next generation multimedia search agents. This paper introduces an audio classification technique based on audio content analysis. Artificial Neural Networks (ANNs), specifically multi-layered perceptrons (MLPs) are implemented to perform the classification task. Windowed audio files of finite length are analyzed to generate multiple feature sets which are used as input vectors to a parallel neural architecture that performs the classification. This paper examines a combination of linear predictive coding (LPC), mel frequency cepstrum coefficients (MFCCs), Haar Wavelet, Daubechies Wavelet and Symlet coefficients as feature sets for the proposed audio classifier. Parallel to MLP, a Gaussian radial basis function (GRBF) based ANN is also implemented and analyzed. The obtained prediction accuracy of 87.3% in determining the audio genres claims the efficiency of the proposed architecture. The ANN prediction values are processed by a rule based inference engine (IE) that presents the final decision.     

A neural network based approach for measurement dynamics compensation

In this article a task-oriented neural network (NN) solution is proposed for the problem of article recovering real process outputs from available distorted measurements. It is shown that a neural network can be used as approximator of inverted first-order measurement dynamics with and without time delay. The trained NN is connected in series with the sensor, resulting in an identity mapping between the inputs and the outputs of the composed system. In this way the network acts as a software mechanism to compensate for the existing dynamics of the whole measurement system and recover the actual process output. For those cases where changes in the measurement system occur, a multiple concurrent-NN recovering scheme is proposed. This requires a periodical path-finding calibration to be performed. A procedure for such a calibration purpose has also been developed, implemented, and tested. It is shown that it brings adequate robustness to the overall compensation scheme. Results showing the performance of both the NN compensator and the calibration procedure are presented for closed loop system operation.