基于SSAE深度学习特征表示的高光谱遥感图像分类方法

针对遥感图像中高光谱数据的分类问题,提出一种基于堆叠稀疏自动编码器(SSAE)深度学习特征表示的高光谱遥感图像分类方法。首先,将光谱数据样本进行预处理和归一化。然后,将其输入到SSAE中进行特征表示学习,并通过网格搜索来获得最优网络参数,以此获得有效的特征表示。最后通过支持向量机(SVM)分类器对输入图像特征进行分类,最终实现遥感图像中像素的分类。在两个标准数据集上的实验结果表明,该方法能够实现准确的高光谱地物分类。     

基于自编码器的特征迁移算法

近年来,栈式自编码网络（stacked auto-encoder,SAE）在大规模数据集上表现出优异的图像分类性能。相对于其他图像分类方法中手工设计的低级特征,SAE的成功归因于深度网络能够学习到丰富的中级图像特征。然而,估计上百万个网络参数需要非常庞大的带标签的图像样本数据集。这样的性质阻止了SAE在小规模训练数据上的许多应用。在这篇文章中,提出的算法展示如何将SAE在大规模数据集上学习到的图像表示有效地迁移到只有有限训练数据的视觉识别任务中。实验部分设计了一个方法来复用在MNIST数据集上训练得到的隐藏层,以此计算在MNIST-variations数据集上的中级图像表示。实验结果展示了尽管两个数据集之间存在差异,但是被迁移的图像特征能够使得模型的分类性能得到极大的提升。     

稀疏和标签约束半监督自动编码机的分类算法

自动编码机通过深度无监督学习能够表达数据的语义特征,但由于其隐含层节点个数难以有效确定,所处理的数据进一步用于分类时常会导致分类准确度低、稳定性弱等问题。针对这些问题,提出了一种稀疏和标签约束的半监督自动编码机（SLRAE）,以实现无监督学习与监督学习的有机结合,更准确地抽取样本的本质特征。稀疏约束项针对每个隐含节点的响应添加约束条件,从而在隐含神经元数量较多的情况下仍可发现数据中潜在的结构;同时引入标签约束项,以监督学习的方式比对实际标签与期望标签,针对性地调整网络参数,进一步提高分类准确率。为验证所提方法的有效性,实验中对多个数据集进行广泛地测试,其结果表明,相对传统自动编码机（AE）、稀疏自动机（SAE）以及极限学习机（ELM）,SLRAE所处理的数据应用于同一分类器,能明显提高分类准确率和稳定性。     

奇异值分解和稀疏自编码器的轴承故障诊断

针对滚动轴承故障特征提取和分类需要进行有监督训练才能实现等问题,提出了一种基于奇异值分解（SVD）和时域统计特征分析并结合堆栈稀疏自编码器（SAE）以及Softmax分类器实现滚动轴承故障诊断方法。该方法利用Hankle矩阵对原始数据进行矩阵重构,利用奇异值分解和时域分析对重构后的故障信号进行特征预提取,融合两种特征并输入到堆栈稀疏自编码器中进行特征优化,将优化后的特征输入到Softmax分类器中进行分类识别。实验结果表明,3种工况下10类故障数据的识别准确率均在96%左右,且高于文中其他方法,因此该方法能有效地进行滚动轴承复杂信号的特征预处理以及分类。     

Scene classification based on a hierarchical convolutional sparse auto-encoder for high spatial resolution imagery

Efficiently representing and recognizing the semantic classes of the subregions of large-scale high spatial resolution (HSR) remote-sensing images are challenging and critical problems. Most of the existing scene classification methods concentrate on the feature coding approach with handcrafted low-level features or the low-level unsupervised feature learning approaches, which essentially prevent them from better recognizing the semantic categories of the scene due to their limited mid-level feature representation ability. In this article, to overcome the inadequate mid-level representation, a patch-based spatial-spectral hierarchical convolutional sparse auto-encoder (HCSAE) algorithm, based on deep learning, is proposed for HSR remote-sensing imagery scene classification. The HCSAE framework uses an unsupervised hierarchical network based on a sparse auto-encoder (SAE) model. In contrast to the single-level SAE, the HCSAE framework utilizes the significant features from the single-level algorithm in a feedforward and full connection approach to the maximum extent, which adequately represents the scene semantics in the high level of the HCSAE. To ensure robust feature learning and extraction during the SAE feature extraction procedure, a ‘dropout’ strategy is also introduced. The experimental results using the UC Merced data set with 21 classes and a Google Earth data set with 12 classes demonstrate that the proposed HCSAE framework can provide better accuracy than the traditional scene classification methods and the single-level convolutional sparse auto-encoder (CSAE) algorithm.     

Ensemble deep learning based semi-supervised soft sensor modeling method and its application on quality prediction for coal preparation process

Coal preparation is the most effective and economical technique to reduce impurities and improve the product quality for run-of-mine coal. The timely and accurate prediction for key quality characteristics of separated coal plays a significant role in condition monitoring and production control. However, these quality characteristics are usually difficult to directly measure online in industrial practices Although some computation intelligence based soft sensor modeling methods have been developed and reported in existing research for these quality variables estimation, some problems still exist, i.e., manual feature extraction, considerable unlabeled data, temporal dynamic behavior in data, which will influence the accuracy and efficiency for established soft sensor model. To address above-mentioned problem and develop an more excellent quality prediction model for coal preparation process, a novel deep learning based semi-supervised soft sensor modeling approach is proposed which combining the advantage of unsupervised deep learning technique (i.e., Stacked Auto-Encoder (SAE)) with the advantage of supervised deep bidirectional recurrent learner (i.e., Bidirectional Long Short-Term Memory (BLSTM)). More specifically, the unsupervised SAE networks are implemented to learn the representative features hidden in all available input data (labeled and unlabeled samples) and store them as context vector. Then, partial context vector with corresponding labels and the quality variable measure value at previous time are concatenated to form a new merged input feature vector. After that, the temporal and dynamic features are further extracted from the new merged input feature vector via BLSTM networks. Subsequently, the fully connected layers (FCs) are exploited to learn the higher-level features from the last hidden layer of the BLSTM. Lastly, the learned output features by FCs are fed into a supervised liner regression layer to predict the coal quality metrics. Meanwhile, to avoid over-fitting, some regularization techniques are utilized and discussed in proposed network. The application in ash content estimation for a real dense medium coal preparation process and some comparison experiment result demonstrate that the effectiveness and priority of proposed soft sensor modeling approach.     

On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery

In recent years, satellite imagery has greatly improved in both spatial and spectral resolution. One of the major unsolved problems in highly developed remote sensing imagery is the manual selection and combination of appropriate features according to spectral and spatial properties. Deep learning framework can learn global and robust features from the training data set automatically, and it has achieved state-of-the-art classification accuracies over different image classification tasks. In this study, a technique is proposed which attempts to classify hyperspectral imagery by incorporating deep learning features. Firstly, deep learning features are extracted by multiscale convolutional auto-encoder. Then, based on the learned deep learning features, a logistic regression classifier is trained for classification. Finally, parameters of deep learning framework are analysed and the potential development is introduced. Experiments are conducted on the well-known Pavia data set which is acquired by the reflective optics system imaging spectrometer sensor. It is found that the deep learning-based method provides a more accurate classification result than the traditional ones.     

基于深度迁移自编码器的变工况下滚动轴承故障诊断方法

在实际工业场景下的轴承故障诊断,存在轴承故障样本不足,训练样本与实际信号样本存在分布差异的问题;文章提出一种新的基于深度迁移自编码器的故障诊断方法FS-DTAE,应用于不同工况下的轴承故障诊断;该方法首先采用小波包变换进行信号处理与特征提取;其次,采用提出的基于朴素贝叶斯与域间差异的特征选取(FSBD)方法对统计特征进行评估,选取更有利于跨域故障诊断和迁移学习的特征;然后,利用源域特征数据训练深度自编码器,将训练得到的模型参数迁移至目标域,再利用目标域正常状态样本对深度迁移自编码器模型进行微调,微调后的模型用于目标域无标签特征数据的故障分类;最后,基于CWRU轴承故障数据开展不同工况下故障诊断实验,结果表明,所提出的FS-DTAE方法能够有效提高不同工况下的故障诊断准确率。     

基于SAE和GNDO-SVM的脑电信号情绪识别

情感计算是现代人机交互中的关键问题, 随着人工智能的发展, 基于脑电信号(electroencephalogram, EEG)的情绪识别已经成为重要的研究方向. 为了提高情绪识别的分类精度, 本研究引入堆叠自动编码器(stacked auto-encoder, SAE)对EEG多通道信号进行深度特征提取, 并提出一种基于广义正态分布优化的支持向量机(generalized normal distribution optimization based support vector machine, GNDO-SVM)情绪识别模型. 实验结果表明, 与基于遗传算法、粒子群算法和麻雀搜索算法优化的支持向量机模型相比, 所提出的GNDO-SVM模型具有更优的分类性能, 基于SAE深度特征的情感识别准确率达到了90.94%, 表明SAE能够有效地挖掘EEG信号不同通道间的深度相关性信息. 因此, 利用SAE深度特征结合GNDO-SVM模型可以有效地实现EEG信号的情绪识别.     

Design teacher and supervised dual stacked auto-encoders for quality-relevant fault detection in industrial process

Current fault detection methods based on deep neural networks only consider process information and ignore quality indicators. In order to obtain features representing both process variables and quality indicators efficiently, this paper designs teacher and supervise dual stacked auto-encoder (TSSAE) for quality-relevant fault detection in industrial process which separates the feature extraction and model construction. To separate the feature extraction and model construction, a mixing stacked auto-encoder which consists of a nonlinear encoder and a linear decoder is designed to extract features of process variables and quality indicators. Another encoder is supervised by the extracted features and further predict the process variables and quality indicators only from process variables. Then quality-relevant, quality-irrelevant and residual subspaces are constructed in a linear way and fault detection is implemented in these subspaces based on Euclidean distance and kernel density estimation. Finally, the effectiveness of TSSAE is evaluated by a numerical example and the Tennessee-Eastman process.     

改进集成深层自编码器在轴承故障诊断中的应用

针对滚动轴承振动信号故障特征难以自动提取和故障类别难以自动准确识别的问题,提出一种改进集成深层自编码器(IEDAE)方法.首先,改进自编码器的损失函数并设计3种小波卷积自编码器;其次,利用区分自编码器、小波卷积自编码器等5种自编码器构造相应的深层自编码器,并设计“跨层”连接以缓解深层网络的梯度消失现象,实现对轴承振动信号的无监督预训练和有监督微调;最后,通过加权平均法输出识别结果,以保证诊断结果的准确性和稳定性.实验结果表明,改进集成深层自编码器方法能有效地对滚动轴承进行多种工况和多种故障程度的识别,较好地摆脱了对人工特征提取的依赖,特征提取能力和识别能力优于现有其他方法.     

基于深层长短期记忆网络与批规范化的间歇过程故障检测方法

传统的基于数据驱动的间歇过程故障诊断方法往往需要对过程数据的分布进行假设，而且对非线性等复杂数据的监控往往会出现误报和漏报，为此提出一种基于长短期记忆网络（LSTM）与批规范化（BN）结合的监督学习方法，不需要对原始数据的分布进行假设。首先，对间歇过程原始数据运用一种按变量展开并连续采样的预处理方式，使处理后的数据可以向LSTM单元输入；然后，利用改进的深层LSTM网络进行特征学习，该网络通过添加BN层，结合交叉熵损失的表示方法，可以有效提取间歇过程数据的特征并进行快速学习；最后，在一类半导体蚀刻过程上进行仿真实验。实验结果表明，所提方法比多元线性主成分分析（MPCA）方法故障识别的种类更多，可以有效地识别各类故障，对故障的整体检测率达到95%以上；比传统单层LSTM模型建模速度更快，且对故障的整体检测率提高了8个百分点以上，比较适合处理间歇过程中具有非线性、多工况等特征的故障检测问题。     

Stacked auto-encoder for classification of polarimetric SAR images based on scattering energy

This paper proposes a new algorithm, for polarimetric synthetic aperture radar (PolSAR) classification, based on a stacked auto-encoder and scattering energy. Previous approaches to PolSAR classification predominantly consider only the single pixel of distribution of the polarimetric data and scattering characteristics, and ignore other kinds of image features like the relationship of the local pixels. Besides, because of the complexities of PolSAR data, it is difficult to compute the derivatives that are needed for back-propagation in deep-learning classifiers. To overcome these difficulties, we propose a new approach that combines the scattering power and stacks sparse auto-encoder (Scattering SSAE) for PolSAR classification. Firstly, orientation compensation is used to compensate the polarization orientation angle, reducing the impact of polarimetric angle noise. Secondly, Freeman-Durden decomposition is adopted to extract three basic scattering powers: surface, double bounce and volume. Each PolSAR image pixel is transformed into these scattering powers, yielding a new kind of feature from the PolSAR data. Finally, using the three kinds of scattering power as inputs, we combine local spatial information using a patch-based approach, and use a deep learning architecture to achieve classification. We compare our method against several other state-of-the-art methods using ground-truthed test-data, and show that the Scattering SSAE method achieves higher accuracy than other methods on most categories.     

Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach

Fault diagnosis methods for rotating machinery have always been a hot research topic, and artificial intelligence-based approaches have attracted increasing attention from both researchers and engineers. Among those related studies and methods, artificial neural networks, especially deep learning-based methods, are widely used to extract fault features or classify fault features obtained by other signal processing techniques. Although such methods could solve the fault diagnosis problems of rotating machinery, there are still two deficiencies. (1) Unable to establish direct linear or non-linear mapping between raw data and the corresponding fault modes, the performance of such fault diagnosis methods highly depends on the quality of the extracted features. (2) The optimization of neural network architecture and parameters, especially for deep neural networks, requires considerable manual modification and expert experience, which limits the applicability and generalization of such methods. As a remarkable breakthrough in artificial intelligence, AlphaGo, a representative achievement of deep reinforcement learning, provides inspiration and direction for the aforementioned shortcomings. Combining the advantages of deep learning and reinforcement learning, deep reinforcement learning is able to build an end-to-end fault diagnosis architecture that can directly map raw fault data to the corresponding fault modes. Thus, based on deep reinforcement learning, a novel intelligent diagnosis method is proposed that is able to overcome the shortcomings of the aforementioned diagnosis methods. Validation tests of the proposed method are carried out using datasets of two types of rotating machinery, rolling bearings and hydraulic pumps, which contain a large number of measured raw vibration signals under different health states and working conditions. The diagnosis results show that the proposed method is able to obtain intelligent fault diagnosis agents that can mine the relationships between the raw vibration signals and fault modes autonomously and effectively. Considering that the learning process of the proposed method depends only on the replayed memories of the agent and the overall rewards, which represent much weaker feedback than that obtained by the supervised learning-based method, the proposed method is promising in establishing a general fault diagnosis architecture for rotating machinery.     

基于自动编码器组合的深度学习优化方法

为了提高自动编码器算法的学习精度,更进一步降低分类任务的分类错误率,提出一种组合稀疏自动编码器(SAE)和边缘降噪自动编码器(mDAE)从而形成稀疏边缘降噪自动编码器(SmDAE)的方法,将稀疏自动编码器和边缘降噪自动编码器的限制条件加载到一个自动编码器(AE)之上,使得这个自动编码器同时具有稀疏自动编码器的稀疏性约束条件和边缘降噪自动编码器的边缘降噪约束条件,提高自动编码器算法的学习能力。实验表明,稀疏边缘降噪自动编码器在多个分类任务上的学习精度都高于稀疏自动编码器和边缘降噪自动编码器的分类效果;与卷积神经网络(CNN)的对比实验也表明融入了边缘降噪限制条件,而且更加鲁棒的SmDAE模型的分类精度比CNN还要好。     

融合PSO优化的相关变模态分解与深度学习的旋转机械早期故障智能分类方法

针对旋转机械早期故障信号呈现微弱、相互干扰,易导致故障智能分类精度低的现状。提出一种融合优化的PSO-RVMD (Particle swarm optimization-Relevant Variational Mode Decomposition)与SAE (Stacked AutoEncoder)的旋转机械早期故障分类方法。智能分类方法主要有信号增强与智能分类两阶段组成。首先该方法利用所改进的PSO-RVMD分解电机-轴承系统的早期故障振动信号,通过定义的相关能量比概念计算各分量信号(IMFs)与原始信号之间的相关程度,筛选并重构相关程度高的分量,去除冗余与不相干的干扰与噪声成分,实现信号增强。最后,将增强的早期微弱信号输入到SAE模型中进行训练。利用SAE模型提取高层、抽象且利于分类的深度特征且在最后一层添加BP层,直接对提取的深度特征进行故障分类。通过仿真与实际电机-轴承系统振动信号验证了该方法的有效性,结果表明该方法能快速的实现旋转机械早期微弱故障的精确识别与诊断,提高故障特征学习与自动分类程度。     

基于堆叠降噪稀疏自动编码器的软件缺陷预测

特征提取是软件缺陷预测中的关键步骤,特征提取的质量决定了缺陷预测模型的性能,但传统的特征提取方法难以提取出软件缺陷数据的深层本质特征。深度学习理论中的自动编码器能够从原始数据中自动学习特征,并获得其特征表示,同时为了增强自动编码器的鲁棒性,本文提出一种基于堆叠降噪稀疏自动编码器的特征提取方法,通过设置不同的隐藏层数、稀疏性约束和加噪方式,可以直接高效地从软件缺陷数据中提取出分类预测所需的各层次特征表示。利用Eclipse缺陷数据集的实验结果表明,该方法较传统特征提取方法具有更好的性能。     

稀疏降噪自编码器在IR-BCI的应用研究

针对脑-机接口的特征提取问题,提出了一种基于非监督学习的稀疏降噪自编码器,对刺激诱发的脑电信号进行自主学习,构建原始数据的深层特征表达。该编码器引用稀疏自编码神经网络,通过加入噪声,增强其学习的泛化能力,增加了神经网络的鲁棒性。首先对多导联信号进行重新拼接,输入稀疏降噪自编码器,得到原始数据的稀疏特征表达;然后,采用支持向量机将学习到的特征进行分类;最后,同直接使用最优单通道相对比。实验结果为：稀疏降噪自编码器的分类准确率要优于单通道,表明该方法能够更好地学习到特征,并提高了“模拟阅读”脑-机接口的识别正确率,为脑-机接口系统的特征提取和分类提供了新思路。     

Extracting multi-features and optimizing feature space with sparse auto-encoder over WorldView-2 images

Feature selection of very high-resolution (VHR) images is a key prerequisite for supervised classification. However, it is always difficult to acquire the features which have the highest correlation to the type of land cover for improving classification accuracy. To address this problem, this paper proposed a methodology of feature selection using the results of multiple segmentation via genetic algorithm (GA) and correlation feature selection (CFS) integrating sparse auto-encoder (SAE). Firstly, 61 features, including spectral features and spatial features, are extracted from the results of multi-scale segmentation over a WorldView-2 image in Xicheng District, Beijing. Then, 40-dimensional features and 30-dimensional features are derived from the selection with GA+CFS and the optimization with SAE, respectively. Thirdly, the final classification is achieved by logistic regression (LR) based on different subsets of features extracted from the WorldView-2 image. It is found that the result of feature selection could contribute to increase in the intra-species separation and reduction in the inner-species variability. Adding extra lower-ranked features appeared to reduce the accuracy of classification. The results indicate that the overall classification accuracy with 30-dimensional features reached 87.56%, and increased 5.61% compared to the results with 61-dimensional features. For the two kinds of optimized features, the Z-test values are all greater than 1.96, which implied that feature dimensionality reduction and feature space optimization could significantly improve the accuracy of image land cover classification. The texture features in the wavelet domain are the most important features for the study area in the WorldView-2 image classification. Adding wavelet and the grey-level co-occurrence matrix (GLCM) information, especially for GLCM features in wavelet, appeared not to improve classification accuracy. The SAE-based method can produce feature subsets for improving mapping accuracy more efficiently.     

基于孪生变分自编码器的小样本图像分类方法

当前深度学习大都基于大量数据通过构建深层次的网络实现自动识别,但在很多场景中难以获得大量的样本数据.针对这一问题,提出一种基于孪生变分自编码器(siamese variational auto-encoder,S-VAE)的小样本图像分类方法.通过变分自编码器提取原始训练数据的高层语义特征,然后由两个训练好的变分自编码...