基于优化卷积神经网络结构的交通标志识别

现有算法对交通标志进行识别时,存在训练时间短但识别率低,或识别率高但训练时间长的问题。为此,综合批量归一化（BN）方法、逐层贪婪预训练（GLP）方法,以及把分类器换成支持向量机（SVM）这三种方法对卷积神经网络（CNN）结构进行优化,提出基于优化CNN结构的交通标志识别算法。其中：BN方法可以用来改变中间层的数据分布情况,把卷积层输出数据归一化为均值为0、方差为1,从而提高训练收敛速度,减少训练时间;GLP方法则是先训练第一层卷积网络,训练完把参数保留,继续训练第二层,保留参数,直到把所有卷积层训练完毕,这样可以有效提高卷积网络识别率;SVM分类器只专注于那些分类错误的样本,对已经分类正确的样本不再处理,从而提高了训练速度。使用德国交通标志识别数据库进行训练和识别,新算法的训练时间相对于传统CNN训练时间减少了20.67%,其识别率达到了98.24%。所提算法通过对传统CNN结构进行优化,极大地缩短了训练时间,并具有较高的识别率。     

基于脉冲神经网络的迁移学习算法与软件框架

使用脉冲序列进行数据处理的脉冲神经网络具有优异的低功耗特性,但由于学习算法不成熟,多层网络训练存在收敛困难的问题。利用反向传播网络具有学习算法成熟和训练速度快的特点,设计一种迁移学习算法。基于反向传播网络完成训练过程,并通过脉冲编码规则和自适应的权值映射关系,将训练结果迁移至脉冲神经网络。实验结果表明,在多层脉冲神经网络中,迁移学习算法能够有效解决训练过程中收敛困难的问题,在MNIST数据集和CIFAR-10数据集上的识别准确率分别达到98.56%和56.00%,且具有微瓦级别的低功耗特性。     

一种基于深度CNN的入侵检测算法

入侵检测是检测和预防可能对基于网络的计算机系统进行攻击和入侵作出反应的技术。提出一种基于深度卷积神经网络的入侵检测的算法,在卷积神经网络基础上引入Inception模型和残差网络,采用深度学习技术,如Relu、Dropout、Softmax。提高模型的收敛速度,使得训练的模型的泛化能力更强,增加网络的宽度和深度,提升网络对尺度的适应性。使用KDD Cup 99数据对该算法进行验证,实验表明,该网络模型与GoogleNet和Lenet-5相比具有更高的准确率和检测率,准确率能够达到94.37%,误报率仅2.14%,提高了入侵检测识别的分类准确性。     

面向行人重识别的多域批归一化问题研究

近年来基于深度神经网络的行人重识别算法取得了长足的进步,被广泛应用于网络中的批归一化(batch normalization)模块发挥着重要作用.批归一化模块在多数情况下可有效提高网络收敛速度和训练稳定性,然而当多个独立标注的数据库混合在一块进行跨域或者多域训练时,数据之间的分布差异使得目前的批归一化算法工作逻辑存疑....     

基于深层自适应平衡自编码机的手写数字分类

针对深层自动编码机(DAE)内部协变量迁移问题,提出一种深层自适应平衡自动编码机(DSBAE),用于手写数字的分类(HDC)。重点研究了自适应平衡层对内部协变量迁移的纠正方法,构建了DSBAE网络的分类模型,根据平衡网络参数体系原理,制定了自适应参数更新策略。实验在MNIST,USPS以及PENDIGITS三个公开手写数据集上对DSBAE以及深度学习中其他分类算法进行比较,证明DSBAE能有效解决深层网络的内部协变量迁移问题,并在手写数字分类准确率上占有明显优势。     

模拟电路故障诊断的约束聚类集成BP网络算法

针对通用BP网络对于高纬度、大数据量训练收敛困难的问题,在使用动量因子、自适应调整学习速率等方法的基础,引入约束聚类,构造集成神经网络,以提高网络的训练速度及诊断效果;首先,采用约束聚类算法将训练样本集划分为若干个规模相当的子样本集,分别训练生成相应子网络;此外,在诊断过程中除各子网络的输出变量外,还加入了诊断数据相对各子训练样本集的隶属度因子;最后通过一个实际电路板25维采样数据、38类故障的BP网络诊断实例验证了算法的可行性。     

改进型的batch normalization:BNalpha

针对提高卷积神经网络(convolutional neural network,CNN)在图像识别方向的训练速度和识别准确率进行了研究.从BN(batch normalization)着手,通过新增参数对BN的仿射变换进行具体调节,并提出一种改进型的BN——BNalpha.除去带有某些特定结构的神经网络,相对于原始的BN,BNalpha可以在不增加运算复杂度的前提下,提升神经网络的训练速度和识别准确度.通过对BN仿射变换的参数进行分析和对比,尝试解释BN在网络中的运行机理,并以此说明BNalpha相对于BN的改进为何生效.最后通过CIFAR-10和CIFAR-100数据集以及不同类型的卷积神经网络结构对BNalpha和BN进行对比实验分析,实验结果表明BNalpha能够进一步提升训练速度和识别准确度.     

基于面向对象自适应粒子群算法的神经网络训练*

针对传统的神经网络训练算法收敛速度慢和泛化性能低的缺陷,提出一种新的基于面向对象的自适应粒子群优化算法(OAPSO)用于神经网络的训练。该算法通过改进PSO的编码方式和自适应搜索策略以提高网络的训练速度与泛化性能,并结合Iris和Ionosphere分类数据集进行测试。实验结果表明：基于OAPSO算法训练的神经网络在分类准确率上明显优于BP算法及标准PSO算法,极大地提高了网络泛化能力和优化效果,具有快速全局收敛的性能。     

Spark下BP神经网络并行化算法研究

BP算法(反向传播算法)以其良好的非线性逼近能力、泛化能力以及实用性成为了人工神经网络训练算法中应用最为广泛的算法.但同时使用BP算法又存在收敛速度较慢、易陷入局部极小值等问题.为了将BP算法用于大规模数据分类问题,采用MapReduce思想,将大数据集切分成若干小的数据集来并行加速处理,同时引入Bagging算法的思想来综合并行结果,提高分类的准确率.通过在各个节点上根据子数据集独立地训练各个BP神经网络,直至各网络收敛,再将各节点上的网络收集起来进行集成,形成最终的分类器.基于Spark平台的实验表明,本文提出的算法具有良好的并行加速性能,且具有较高的分类准确率.     

双Q网络学习的迁移强化学习算法

深度强化学习在训练过程中会探索大量环境样本,造成算法收敛时间过长,而重用或传输来自先前任务(源任务)学习的知识,对算法在新任务(目标任务)的学习具有提高算法收敛速度的潜力.为了提高算法学习效率,提出一种双Q网络学习的迁移强化学习算法,其基于actor-critic框架迁移源任务最优值函数的知识,使目标任务中值函数网络对策略作出更准确的评价,引导策略快速向最优策略方向更新.将该算法用于Open AI Gym以及在三维空间机械臂到达目标物位置的实验中,相比于常规深度强化学习算法取得了更好的效果,实验证明提出的双Q网络学习的迁移强化学习算法具有较快的收敛速度,并且在训练过程中算法探索更加稳定.     

A comprehensive survey on optimizing deep learning models by metaheuristics

Deep neural networks (DNNs), which are extensions of artificial neural networks, can learn higher levels of feature hierarchy established by lower level features by transforming the raw feature space to another complex feature space. Although deep networks are successful in a wide range of problems in different fields, there are some issues affecting their overall performance such as selecting appropriate values for model parameters, deciding the optimal architecture and feature representation and determining optimal weight and bias values. Recently, metaheuristic algorithms have been proposed to automate these tasks. This survey gives brief information about common basic DNN architectures including convolutional neural networks, unsupervised pre-trained models, recurrent neural networks and recursive neural networks. We formulate the optimization problems in DNN design such as architecture optimization, hyper-parameter optimization, training and feature representation level optimization. The encoding schemes used in metaheuristics to represent the network architectures are categorized. The evolutionary and selection operators, and also speed-up methods are summarized, and the main approaches to validate the results of networks designed by metaheuristics are provided. Moreover, we group the studies on the metaheuristics for deep neural networks based on the problem type considered and present the datasets mostly used in the studies for the readers. We discuss about the pros and cons of utilizing metaheuristics in deep learning field and give some future directions for connecting the metaheuristics and deep learning. To the best of our knowledge, this is the most comprehensive survey about metaheuristics used in deep learning field.

     

基于ArcReLU函数的神经网络激活函数优化研究

近年来深度学习发展迅猛。由于深度学习的概念源于神经网络,而激活函数更是神经网络模型在学习理解非线性函数时不可或缺的部分,因此本文对常用的激活函数进行了研究比较。针对常用的激活函数在反向传播神经网络中具有收敛速度较慢、存在局部极小或梯度消失的问题,将Sigmoid系和ReLU系激活函数进行了对比,分别讨论了其性能,详细分析了几类常用激活函数的优点及不足,并通过研究Arctan函数在神经网络中应用的可能性,结合ReLU函数,提出了一种新型的激活函数ArcReLU。实验证明,该函数既能显著加快反向传播神经网络的训练速度,又能有效降低训练误差并避免梯度消失的问题。     

A new automatic target recognition system based on wavelet extreme learning machine

In this paper, an automatic system is presented for target recognition using target echo signals of High Resolution Range (HRR) radars. This paper especially deals with combination of the feature extraction and classification from measured real target echo signal waveforms by using X-band pulse radar. The past studies in the field of radar target recognition have shown that the learning speed of feedforward neural networks is in general much slower than required and it has been a major disadvantage. There are two key reasons forth is status of feedforward neural networks: (1) the slow gradient-based learning algorithms are extensively used to train neural networks, and (2) all the parameters of the networks are tuned iteratively by using such learning algorithms (Feng et al., 2009, Huang and Siew, 2004, Huang and Chen, 2007, Huang and Chen, 2008, Huang et al., 2006, Huang et al., 2010, Huang et al., 2004, Huang et al., 2005, Huang et al., 2012, Huang et al., 2008, Huang and Siew, 2005, Huang et al., 2011, Huang et al., 2006, Huang et al., 2006a, Huang et al., 2006b, Lan et al., 2009, Li et al., 2005, Liang et al., 2006, Liang et al., 2006, Rong et al., 2009, Wang and Huang, 2005, Wang et al., 2011, Yeu et al., 2006, Zhang et al., 2007, Zhu et al., 2005). To resolve these disadvantages of feedforward neural networks for automatic target recognition area in this paper suggested a new learning algorithm called extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFNs) (Feng et al., 2009, Huang and Siew, 2004, Huang and Chen, 2007, Huang and Chen, 2008, Huang et al., 2006, Huang et al., 2010, Huang et al., 2004, Huang et al., 2005, Huang et al., 2012, Huang et al., 2008, Huang and Siew, 2005, Huang et al., 2011, Huang et al., 2006, Huang et al., 2006a, Huang et al., 2006b, Lan et al., 2009, Li et al., 2005, Liang et al., 2006, Liang et al., 2006, Rong et al., 2009, Wang and Huang, 2005, Wang et al., 2011, Yeu et al., 2006, Zhang et al., 2007, Zhu et al., 2005) which randomly choose hidden nodes and analytically determines the output weights of SLFNs. In theory, this algorithm tends to provide good generalization performance at extremely fast learning speed. Moreover, the Discrete Wavelet Transform (DWT) and wavelet entropy is used for adaptive feature extraction in the time-frequency domain in feature extraction stage to strengthen the premium features of the ELM in this study. The correct recognition performance of this new system is compared with feedforward neural networks. The experimental results show that the new algorithm can produce good generalization performance in most cases and can learn thousands of times faster than conventional popular learning algorithms for feedforward neural networks.     

脉冲神经网络:模型、学习算法与应用

脉冲神经网络是目前最具有生物解释性的人工神经网络,是类脑智能领域的核心组成部分.首先介绍各类常用的脉冲神经元模型以及前馈和循环型脉冲神经网络结构;然后介绍脉冲神经网络的时间编码方式,在此基础上,系统地介绍脉冲神经网络的学习算法,包括无监督学习和监督学习算法,其中监督学习算法按照梯度下降算法、结合STDP规则的算法和基于脉冲序列卷积核的算法3大类别分别展开详细介绍和总结;接着列举脉冲神经网络在控制领域、模式识别领域和类脑智能研究领域的应用,并在此基础上介绍各国脑计划中,脉冲神经网络与神经形态处理器相结合的案例;最后分析脉冲神经网络目前所存在的困难和挑战.     

A hybrid deep learning neural approach for emotion recognition from facial expressions for socially assistive robots

We have recently seen significant advancements in the development of robotic machines that are designed to assist people with their daily lives. Socially assistive robots are now able to perform a number of tasks autonomously and without human supervision. However, if these robots are to be accepted by human users, there is a need to focus on the form of human–robot interaction that is seen as acceptable by such users. In this paper, we extend our previous work, originally presented in Ruiz-Garcia et al. (in: Engineering applications of neural networks: 17th international conference, EANN 2016, Aberdeen, UK, September 2–5, 2016, proceedings, pp 79–93, 2016. https://doi.org/10.1007/978-3-319-44188-7_6), to provide emotion recognition from human facial expressions for application on a real-time robot. We expand on previous work by presenting a new hybrid deep learning emotion recognition model and preliminary results using this model on real-time emotion recognition performed by our humanoid robot. The hybrid emotion recognition model combines a Deep Convolutional Neural Network (CNN) for self-learnt feature extraction and a Support Vector Machine (SVM) for emotion classification. Compared to more complex approaches that use more layers in the convolutional model, this hybrid deep learning model produces state-of-the-art classification rate of \(96.26\%\), when tested on the Karolinska Directed Emotional Faces dataset (Lundqvist et al. in The Karolinska Directed Emotional Faces—KDEF, 1998), and offers similar performance on unseen data when tested on the Extended Cohn–Kanade dataset (Lucey et al. in: Proceedings of the third international workshop on CVPR for human communicative behaviour analysis (CVPR4HB 2010), San Francisco, USA, pp 94–101, 2010). This architecture also takes advantage of batch normalisation (Ioffe and Szegedy in Batch normalization: accelerating deep network training by reducing internal covariate shift. http://arxiv.org/abs/1502.03167, 2015) for fast learning from a smaller number of training samples. A comparison between Gabor filters and CNN for feature extraction, and between SVM and multilayer perceptron for classification is also provided.

     

基于模型的深度学习通信信号鲁棒识别算法

深度学习现在是计算机视觉和自然语言处理的热门话题。在许多应用中,深度神经网络（DNN）的性能都优于 传统的方法,并且已经成功应用于调制分类和无线电信号表示等任务的学习。近几年研究发现深度神经网络极易受到对抗性 攻击,对“对抗性示例”缺乏鲁棒性。笔者就神经网络的通信信号识别算法的鲁棒性问题,将经过PGD攻击的数据看作基于模 型的数据,将该数据输入神经网络,使得信号识别分类结果错误;然后借助基于模型的防御算法,即鲁棒训练算法和对抗训练 算法,进行训练后实验结果表明,两种方法都具有较好的防御效果。     

基于残差网络的血管内超声图像识别

为提高血管内超声(Intravenous Ultrasound,IVUS)图像在动脉粥样硬化识别准确率,实现更高效的计算机辅助诊断,提出综合使用图像增强、特征提取和基于批量归一化(Batch Normalization,BN)优化残差网络的血管内超声图像识别方法。使用Sobel算子在原图像水平和垂直方向进行边缘检测,在此基础上获得锐化增强图像,结合使用灰度共生矩阵提取纹理特征信息;为丰富网络的特征信息,防止梯度消失,使用残差学习对卷积神经网络进行改进。批量归一化通过拟合数据特征分布减少内部协变量转移加速网络收敛。实验结果表明上述方法相比较传统机器学习与改进前的卷积神经网络识别错误率平均降低了58.23%。     

深度学习的轻量化神经网络结构研究综述

随着深度神经网络和智能移动设备的快速发展,网络结构轻量化设计逐渐成为前沿且热门的研究方向,而轻量化的本质是在保持深度神经网络精度的前提下优化存储空间和提升运行速度。阐述深度学习的轻量化网络结构设计方法,对比与分析人工设计的轻量化方法、基于神经网络结构搜索的轻量化方法和基于自动模型压缩的轻量化方法的创新点与优劣势,总结与归纳上述3种主流轻量化方法中性能优异的网络结构并分析各自的优势和局限性。在此基础上,指出轻量化网络结构设计所面临的挑战,同时对其应用方向及未来发展趋势进行展望。     

基于深度学习的网络流量预测研究综述

精准地预判网络流量变化趋势可以帮助运营商准确预估网络的使用情况,合理分配并高效利用网络资源,以满足日益增长且多样化的用户需求。以深度学习算法在网络流量预测领域的进展为线索,阐述了网络流量预测的评价指标和目前公开的网络流量数据集及应用,具体分析了网络流量预测中常用的深度信念网络、卷积神经网络、循环神经网络和长短时记忆网络共四种深度学习方法,并重点介绍了近年来针对不同问题所提出的改进神经网络模型,总结了各模型特点及应用场景。最后对网络流量预测未来发展进行了展望。     

深度卷积神经网络在计算机视觉中的应用研究综述

随着大数据时代的到来,含更多 隐含层的深度卷积神经网络(Convolutional neural networks, CNNs)具有更复杂的网络结构,与传统机器学习方法相比具有更强大的特征学习和特征表达能力。使用深度学习算法训练的卷积神经网络模型自提出以来在计算机视觉领域的多个大规模识别任务上取得了令人瞩目的 成绩。本文首先简要介绍深度学习和卷积神经网络的兴起与展,概述卷积神经网络的基本模型结构、卷积特征提取和池化操作。然后综述了基于深度学习的卷积神经网络模型在图像分类、物体检测、姿态估计、图像分割和人脸识别等多个计算机视觉应用领域中的研究现状 和发展趋势,主要从典型的网络结构的构建、训练方法和性能表现3个方面进行介绍。最后对目前研究中存在的一些问题进行简要的总结和讨论,并展望未来发展的新方向。