使用改进自编码器的细粒度图像分类研究

针对深度连续聚类算法(Deep Continuous Clustering, DCC)特征提取能力有限,对复杂图像不能提取足够有效细节特征的不足,本文提出一个新的循环卷积自编码器(Recurrent Convolutional Auto-Encoder, R-CAE).自编码器结合门控循环网络GRU和卷积网络CNN构造编码层;同时在门控循环网络GRU部分添加空间域注意力通道,增强网络的特征学习能力.图像信息经过R-CAE自编码器编码后获取细节信息,传入经典卷积神经网络学习特征;当优化结果接近或者达到聚类阈值的时候,获得最终的聚类结果实现分类.训练过程中,模型首先预训练,确定自编码器参数;然后结合编码部分和经典网络学习训练,微调网络参数.本文通过实验证明了改进方法结合DCC在聚类实验中优于大部分经典聚类算法,在针对真实图像的细粒度分类实验中也有显著的进步.     

混合高斯变分自编码器的聚类网络

目的 经典的聚类算法在处理高维数据时存在维数灾难等问题,使得计算成本大幅增加并且效果不佳。以自编码或变分自编码网络构建的聚类网络改善了聚类效果,但是自编码器提取的特征往往比较差,变分自编码器存在后验崩塌等问题,影响了聚类的结果。为此,本文提出了一种基于混合高斯变分自编码器的聚类网络。方法 使用混合高斯分布作为隐变量的先验分布构建变分自编码器,并以重建误差和隐变量先验与后验分布之间的KL散度(Kullback-Leibler divergence)构造自编码器的目标函数训练自编码网络;以训练获得的编码器对输入数据进行特征提取,结合聚类层构建聚类网络,以编码器隐层特征的软分配分布与软分配概率辅助目标分布之间的KL散度构建目标函数并训练聚类网络;变分自编码器采用卷积神经网络实现。结果 为了验证本文算法的有效性,在基准数据集MNIST(Modified National Institute of Standards and Technology Database)和Fashion-MNIST上评估了该网络的性能,聚类精度(accuracy,ACC)和标准互信息(normalized mutua...     

嵌入式深度神经网络高光谱图像聚类

目的 高光谱图像的高维特性和非线性结构给聚类任务带来了"维数灾难"和线性不可分问题，以往的工作将特征提取过程与聚类过程互相剥离，难以同时优化。为了解决上述问题，提出了一种新的嵌入式深度神经网络模糊C均值聚类方法（EDFCC）。方法 EDFCC算法为了提取更加有效的深层特征，联合优化高光谱图像的特征提取和聚类过程，将模糊C均值聚类算法嵌入至深度自编码器网络中，可以保持两任务联合优化的优势，同时利用深度自编码器网络降维以及逼近任意非线性函数的能力，逐步将原始数据映射到潜在特征空间，提取数据的深层特征。所提方法采用模糊C均值聚类算法约束特征提取过程，学习适用于聚类的高光谱数据深层特征，动态调整聚类指示矩阵。结果 实验结果表明，EDFCC算法在Indian Pines和Pavia University两个高光谱数据集上的聚类精度分别达到了42.95%和60.59%，与当前流行的低秩子空间聚类算法（LRSC）相比分别提高了3%和4%，相比于基于自编码器的数据聚类算法（AEKM）分别提高了2%和3%。结论 EDFCC算法能够从高光谱图像的高维光谱信息中提取更加有效的深层特征，提升聚类精度，并且由于EDFCC算法不需要额外的训练过程，大大提升了聚类效率。     

基于栈式自编码器模型的汇率时间序列预测

针对目前具有非线性特征的金融时间序列浅层模型预测精度有限的问题,提出一种由底层的栈式自编码器和顶层的回归神经元组成的栈式自编码神经网络预测模型。首先利用自编码器的无监督学习机制对时间序列进行特征识别与学习,逐层贪婪学习神经网络各层,之后将栈式自编码器扩展为有监督机制的SAEP模型,将SAE学习到的参数用于初始化神经网络,最后利用有监督学习对权值进行微调。实验设计利用汇率时间序列作为训练及测试样本,与目前较成熟的方法进行对比实验,验证了所提出的模型在汇率时序预测应用中的有效性。     

非对称卷积编码器的聚类算法

基于深度神经网络的非监督学习方法通过联合优化特征表示和聚类指派,大大提升了聚类任务的性能。但大量的参数降低了运行速度,另外,深度模型提取的特征的区分能力也影响聚类性能。为此,提出一种新的聚类算法(asymmetric fully-connected layers convolutional auto-encoder, AFCAE),其中卷积编码器结合非对称全连接进行无监督的特征提取,然后K-means算法对所得特征执行聚类。网络采用3×3和2×2的小卷积核,大大减少了参数个数,降低了算法复杂性。在MNIST上AFCAE获得0.960的聚类精度,比联合训练的DEC(deep embedding clustering)方法(0.840)提高了12个百分点。在6个图像数据集上实验结果表明AFCAE网络有优异的特征表示能力,能出色完成下游的聚类任务。     

基于融合变分图注意自编码器的深度聚类模型

聚类作为数据挖掘和机器学习中最基本的任务之一,在各种现实世界任务中已得到广泛应用.随着深度学习的发展,深度聚类成为一个研究热点.现有的深度聚类算法主要从节点表征学习或者结构表征学习两个方面入手,较少考虑同时将这两种信息进行融合以完成表征学习.提出一种融合变分图注意自编码器的深度聚类模型FVGTAEDC(Deep Clustering Model Based on Fusion Varitional Graph Attention Self-encoder),此模型通过联合自编码器和变分图注意自编码器进行聚类,模型中自编码器将变分图注意自编码器从网络中学习(低阶和高阶)结构表示进行集成,随后从原始数据中学习特征表示.在两个模块训练的同时,为了适应聚类任务,将自编码器模块融合节点和结构信息的表示特征进行自监督聚类训练.通过综合聚类损失、自编码器重构数据损失、变分图注意自编码器重构邻接矩阵损失、后验概率分布与先验概率分布相对熵损失,该模型可以有效聚合节点的属性和网络的结构,同时优化聚类标签分配和学习适合于聚类的表示特征.综合实验证明,该方法在5个现实数据集上的聚类效果均优于当前先进的深度聚类方法.     

基于量子双缝干涉实验的神经网络模型

本文利用量子理论中的双缝干涉实验(Double-slit Interference Experiment)构造了一种全新的量子神经网络(Quantum Neural Network,QNN)模型.通过理论分析,推导出该模型的动力学表达式,并给出相应的训练算法.仿真实验表明,该模型具有学习布尔逻辑函数的功能,特别是两层网络结构能够实现类似异或(XOR)逻辑的学习,体现出了量子计算对传统神经网络的优越性.本文的研究为探索神经网络与量子计算的结合提供了一个新的途径.     

改进深层小波自编码器的轴承故障诊断方法

针对传统滚动轴承故障诊断方法过度依赖专家经验和故障特征提取困难的问题,结合深度学习处理高维、非线性数据的优势,提出一种基于改进深层小波自编码器的轴承智能故障诊断方法。该方法改进小波自编码器的损失函数并引入收缩项限制,再将多个小波自编码器进行堆叠构成深层小波自编码器,并引入“跨层”连接缓解梯度消失现象,最后利用大量无标签数据对网络进行无监督预训练并利用少量带标签数据对模型参数有监督微调。轴承诊断实验结果表明,该方法能有效地对轴承进行多种故障类型和多种故障程度的识别,特征提取能力和识别能力优于人工神经网络、深度信念网络、深度自编码器等方法。     

一种用于旋转机械故障特征提取的簇紧凑自编码器

针对传统自编码器以无监督方式学习特征、缺乏监督信息的指导造成特征判别性弱的问题,提出一种簇紧凑自编码器(cluster compact auto-encoder,CCAE).首先,利用模糊C均值算法对样本进行聚类得到伪标签,并通过PBMF指标确定最佳聚类数;然后,利用伪标签构建簇紧凑正则项,嵌入样本所属类别的判别性信息;最后,将簇紧凑正则项与标准自编码器的损失函数相结合作为CCAE的损失函数,所提出的CCAE通过伪标签的方式嵌入区分类别的判别性信息,可增强特征的判别性,从而显著提升诊断性能;最后,在旋转机械齿轮和轴承数据集上验证所提出方法的有效性,结果表明,CCAE可广泛用于旋转机械故障诊断的特征提取阶段,为工程人员实现判别性特征的自动提取提供一种解决方案.     

一种新的无监督前景目标检测方法

针对基于无监督特征提取的目标检测方法效率不高的问题,提出一种在无标记数据集中准确检测前景目标的方法.其基本出发点是:正确的特征聚类结果可以指导目标特征提取,同时准确提取的目标特征可以提高特征聚类的精度.该方法首先对无标记样本图像进行局部特征提取,然后根据最小化特征距离进行无监督特征聚类.将同一个聚类内的图像两两匹配,将特征匹配的重现程度作为特征权重,最后根据更新后的特征权重指导下一次迭代的特征聚类.多次迭代后同时得到聚类结果和前景目标.实验结果表明,该方法有效地提高Caltech-256数据集和Google车辆图像的检测精度.此外,针对目前绝大部分无监督目标检测方法不具备增量学习能力这一缺点,提出了增量学习方法实现,实验结果表明,增量学习方法有效地提高了计算速度.     

量子神经网络在PID参数调整中的应用

提出一种基于量子神经网络(QNNs)的比例积分微分(PID)参数在线调整方法.通过构造受控量子旋转门,给出一个量子神经元模型,其中包括输入量子比特相位的旋转角度和控制量2种设计参数.在此基础上提出一个量子神经网络模型,利用梯度下降法设计该模型的学习算法,并将其用于PID参数的在线调整,实验结果表明,QNNs的调整能力及...     

Training Reformulated Radial Basis Function Neural Networks Capable of Identifying Uncertainty in Data Classification

This paper introduces a learning algorithm that can be used for training reformulated radial basis function neural networks (RBFNNs) capable of identifying uncertainty in data classification. This learning algorithm trains a special class of reformulated RBFNNs, known as cosine RBFNNs, by updating selected adjustable parameters to minimize the class-conditional variances at the outputs of their radial basis functions (RBFs). The experiments verify that quantum neural networks (QNNs) and cosine RBFNNs trained by the proposed learning algorithm are capable of identifying uncertainty in data classification, a property that is not shared by cosine RBFNNs trained by the original learning algorithm and conventional feed-forward neural networks (FFNNs). Finally, this study leads to a simple classification strategy that can be used to improve the classification accuracy of QNNs and cosine RBFNNs by rejecting ambiguous feature vectors based on their responses.     

Stacked autoencoder based deep random vector functional link neural network for classification

Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input–output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks.     

基于栈式去噪自动编码器的边际Fisher分析算法

特征学习是模式识别领域的关键问题。基于自动编码器的深度神经网络通过无监督预训练与有监督微调能够有效地提取数据中关键信息,形成特征。提出一种基于栈式去噪自编码器的边际Fisher分析算法,该算法将边际Fisher分析运用于有监督微调阶段,进一步提升算法的特征学习能力。实验结果表明,该算法与标准的栈式去噪自编码器和基于受限玻尔兹曼机的深度信念网相比,具有更好的识别效果。     

Quantum neural networks (QNNs): inherently fuzzy feedforward neuralnetworks

This paper introduces quantum neural networks (QNNs), a class of feedforward neural networks (FFNNs) inherently capable of estimating the structure of a feature space in the form of fuzzy sets. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Unlike other approaches attempting to merge fuzzy logic and neural networks, QNNs can be used in pattern classification problems without any restricting assumptions such as the availability of a priori knowledge or desired membership profile, convexity of classes, a limited number of classes, etc. Experimental results presented here show that QNNs are capable of recognizing structures in data, a property that conventional FFNNs with sigmoidal hidden units lack.     

基于SOFM网络的聚类分析

基于自组织特征映射网络的聚类分析,是在神经网络基础上发展起来的一种新的非监督聚类方法,分析了基于自组织特征映射网络聚类的学习过程,分析了权系数自组织过程中邻域函数和学习步长的一般取值问题,给出了基于自组织特征映射网络聚类实现的具体算法,并通过实际示例测试,证实了算法的正确性。     

基于受控旋转门的量子神经网络模型算法及应用

提出一种量子神经网络模型及算法．首先借鉴受控非门的含义提出一种受控量子旋转门,基于该门的物理意义,提出一种量子神经元模型,该模型包含对输入量子比特相位的旋转角度和对旋转角度的控制量两种设计参数;然后基于上述量子神经元提出一种量子神经网络模型,基于梯度下降法详细设计了该模型的学习算法：最后通过模式识别和时间序列预测两个仿...     

An evaluation of quantum neural networks in the detection of epileptic seizures in the neonatal electroencephalogram

This paper presents the results of an experimental study that evaluated the ability of quantum neural networks (QNNs) to capture and quantify uncertainty in data and compared their performance with that of conventional feedforward neural networks (FFNNs). In this work, QNNs and FFNNs were trained to classify short segments of epileptic seizures in neonatal EEG. The experiments revealed significant differences between the internal representations created by trained QNNs and FFNNs from sample information provided by the training data. The results of this experimental study also confirmed that the responses of trained QNNs are more reliable indicators of uncertainty in the input data compared with the responses of trained FFNNs.     

Unsupervised feature selection using clustering ensembles and population based incremental learning algorithm

This paper describes a novel feature selection algorithm for unsupervised clustering, that combines the clustering ensembles method and the population based incremental learning algorithm. The main idea of the proposed unsupervised feature selection algorithm is to search for a subset of all features such that the clustering algorithm trained on this feature subset can achieve the most similar clustering solution to the one obtained by an ensemble learning algorithm. In particular, a clustering solution is firstly achieved by a clustering ensembles method, then the population based incremental learning algorithm is adopted to find the feature subset that best fits the obtained clustering solution. One advantage of the proposed unsupervised feature selection algorithm is that it is dimensionality-unbiased. In addition, the proposed unsupervised feature selection algorithm leverages the consensus across multiple clustering solutions. Experimental results on several real data sets demonstrate that the proposed unsupervised feature selection algorithm is often able to obtain a better feature subset when compared with other existing unsupervised feature selection algorithms.     

Growing radial basis neural networks: merging supervised andunsupervised learning with network growth techniques

This paper proposes a framework for constructing and training radial basis function (RBF) neural networks. The proposed growing radial basis function (GRBF) network begins with a small number of prototypes, which determine the locations of radial basis functions. In the process of training, the GRBF network gross by splitting one of the prototypes at each growing cycle. Two splitting criteria are proposed to determine which prototype to split in each growing cycle. The proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of GRBF networks. These include unsupervised algorithms for clustering and learning vector quantization, as well as learning algorithms for training single-layer linear neural networks. A supervised learning scheme based on the minimization of the localized class-conditional variance is also proposed and tested. GRBF neural networks are evaluated and tested on a variety of data sets with very satisfactory results.