VAE过采样与迁移学习在网络入侵检测中的应用

在网络入侵检测中,异常样本通常要比正常样本少得多,数据的不平衡问题会导致检测模型的分类结果倾向于多数类,影响模型准确率.文章提出应用变分自编码器(VAE)模型对网络入侵检测中的不平衡数据进行过采样,通过学习原数据的特征后生成新样本重新平衡数据分布,以提高检测模型的性能.在训练检测模型时采用迁移学习方法,先在过采样后混合的数据集上预训练,再迁移到原数据集上进行训练,得到最终的检测模型.在NSL-KDD数据集上进行实验,网络入侵检测模型使用前馈神经网络.结果表明,基于深度学习的VAE过采样方法比传统的SMOTE过采样方法要更加有效,提高了网络入侵检测模型准确率3.23％.     

基于VAE-CGAN的牦牛等级评定算法

在牦牛高效养殖过程中, 牦牛等级评定是牦牛育种工作中的重要环节. 为了在牦牛等级评定研究中, 降低数据集分布不平衡对牦牛等级预测结果的影响, 提出一种基于改进条件生成对抗网络模型的牦牛等级评定模型VAE-CGAN. 首先, 为获取高质量生成样本, 模型通过引入变分自编码器取代条件生成对抗网络输入中的随机噪声, 降低了随机变量带来的不确定性. 此外, 模型将牦牛标签作为条件信息输入到生成对抗模型中来获取指定类别的生成样本, 生成样本及训练样本则会被用于训练深度神经网络分类器. 实验结果显示, 模型整体预测准确率达到了97.9%. 而且与生成对抗网络相比较, 在数量较少的特级牦牛等级预测上的精准率、召回率和F1值分别提升了16.7%、16.6%和19.4%. 实验结果表明该模型可以实现高精准度和低误分类率的牦牛等级分类.     

混合高斯变分自编码器的聚类网络

目的 经典的聚类算法在处理高维数据时存在维数灾难等问题,使得计算成本大幅增加并且效果不佳。以自编码或变分自编码网络构建的聚类网络改善了聚类效果,但是自编码器提取的特征往往比较差,变分自编码器存在后验崩塌等问题,影响了聚类的结果。为此,本文提出了一种基于混合高斯变分自编码器的聚类网络。方法 使用混合高斯分布作为隐变量的先验分布构建变分自编码器,并以重建误差和隐变量先验与后验分布之间的KL散度（Kullback-Leibler divergence）构造自编码器的目标函数训练自编码网络;以训练获得的编码器对输入数据进行特征提取,结合聚类层构建聚类网络,以编码器隐层特征的软分配分布与软分配概率辅助目标分布之间的KL散度构建目标函数并训练聚类网络;变分自编码器采用卷积神经网络实现。结果 为了验证本文算法的有效性,在基准数据集MNIST （Modified National Institute of Standards and Technology Database）和Fashion-MNIST上评估了该网络的性能,聚类精度（accuracy,ACC）和标准互信息（normalized mutual information,NMI）指标在MNIST数据集上分别为95.86%和91%,在Fashion-MNIST数据集上分别为61.34%和62.5%,与现有方法相比性能有了不同程度的提升。结论 实验结果表明,本文网络取得了较好的聚类效果,且优于当前流行的多种聚类方法。     

基于分类特征约束变分伪样本生成器的类增量学习

针对神经网络模型进行类增量训练时产生的灾难性遗忘问题,提出一种基于分类特征约束变分伪样本生成器的类增量学习方法.首先,通过构造伪样本生成器记忆旧类样本来训练新的分类器及新的伪样本生成器.伪样本生成器以变分自编码器为基础,用分类特征进行约束,使生成的样本更好地保留旧类在分类器上的性能.然后,用旧分类器的输出作为伪样本的精馏标签,进一步保留从旧类获得的知识.最后,为了平衡旧类样本的生成数量,采用基于分类器分数的伪样本选择,在保持每个旧类伪样本数量平衡的前提下选择一些更具代表性的旧类伪样本.在MNIST、FASHION、E-MNIST和SVHN数据集上的实验结果表明,所提出的方法能有效减少灾难性遗忘的影响,提高图像的分类精度.     

基于重构对比的广义零样本图像分类

广义零样本图像分类中常使用生成模型重构视觉信息或语义信息用于再进一步学习.然而,基于变分自编码器的方法对重构样本利用不够充分,表示性能欠缺.因此,文中提出基于重构对比的广义零样本图像分类模型.首先,使用两个变分自编码器将视觉信息和语义信息编码为同维度的低维隐向量,再将隐向量分别解码到两种模态.然后,使用投影模块投影视觉信息与语义模态的隐向量重构的视觉模态信息.最后,对投影后的特征进行重构对比学习.在保持变分自编码器重构性能的基础上增强编码器重构的判别性能,提高预训练特征在广义零样本图像分类任务上的应用能力.在4个标准数据集上的实验证实文中模型的有效性.     

面向高维数据的安全半监督分类算法

半监督学习过程中,由于无标记样本的随机选择造成分类器性能降低及不稳定性的情况经常发生;同时,面对仅包含少量有标记样本的高维数据的分类问题,传统的半监督学习算法效果不是很理想.为了解决这些问题,本文从探索数据样本空间和特征空间两个角度出发,提出一种结合随机子空间技术和集成技术的安全半监督学习算法（A safe semi-supervised learning algorithm combining stochastic subspace technology and ensemble technology,S3LSE）,处理仅包含极少量有标记样本的高维数据分类问题.首先,S3LSE采用随机子空间技术将高维数据集分解为B个特征子集,并根据样本间的隐含信息对每个特征子集优化,形成B个最优特征子集;接着,将每个最优特征子集抽样形成G个样本子集,在每个样本子集中使用安全的样本标记方法扩充有标记样本,生成G个分类器,并对G个分类器进行集成;然后,对B个最优特征子集生成的B个集成分类器再次进行集成,实现高维数据的分类.最后,使用高维数据集模拟半监督学习过程进行实验,实验结果表明S3LSE具有较好的性能.     

一种混合CGAN与SMOTEENN的不平衡数据处理方法

CGAN能够从数据中学习其分布特性,被引入不平衡数据处理中对少数类样本进行过采样,可以生成符合原始数据分布的新样本,因此比传统的重采样方法具有更好的处理效果.然而,CGAN对数据分布特性的学习易受限于样本规模,在少数类样本规模较小时不能充分学习其分布特性,难以保证生成样本的质量.针对这一问题,提出一种将CGAN与SMOTEENN相结合的不平衡数据平衡化处理方法.首先,从既有的少数类样本出发,采用SMOTEENN方法生成一定规模的少数类样本;然后,在此基础上训练CGAN模型,保证其能够生成符合原始少数类样本分布特征的新样本;最后,再利用CGAN重新生成符合原始少数类样本分布的新样本构建平衡数据集.为验证所提出方法的有效性,基于公开的不平衡数据集开展对比实验研究.实验结果表明,相对几种经典的不平衡数据处理方法与近期文献报道的方法,所提出方法在几项不平衡数据分类评价指标上表现出明显的优势.     

基于Spark的油藏数据挖掘与分析

为了方便油藏数据特征的分析和石油的勘探开发过程,本文利用Spark并行计算框架分析油藏数据,并通过数据挖掘算法分析油藏属性之间的潜在关系,对油藏的不同层段进行了分类和预测.本文的主要工作包括：搭建Spark分布式集群和数据处理、分析平台,Spark是流行的大数据并行计算框架,相对传统的一些分析方法和工具,可以实现快速、准确的数据挖掘任务;根据油藏数据的特点建立多维异常检测函数,并新增渗孔比判别属性Pr;在处理不平衡数据时,针对逻辑回归分类提出交叉召回训练模型,并优化代价函数,针对决策树,提出KR-SMOTE对小类别样本进行过采样扩充,这两种方法都可以有效处理数据不平衡问题,提高分类精度.     

基于混合采样的不平衡数据集算法研究

针对不平衡数据集分类效果不理想的问题，提出了一种新的基于混合采样的不平衡数据集算法（BSI）。通过引进“变异系数”找出样本的稀疏域和密集域，针对稀疏域中的少数类样本，提出了一种改进SMOTE算法的过采样方法（BSMOTE）；对密集域中的多数类样本，提出了一种改进的欠采样方法（IS）。通过在六种不平衡数据集上的实验表明，该算法与传统算法相比，取得了更高的G-mean值、F-value值、AUC值，有效改善了不平衡数据集的综合分类性能。     

多层次过采样集成的不平衡数据缺陷预测模型

针对软件缺陷预测中不平衡数据的分类问题，提出了一种基于过采样和集成学习的类不平衡软件缺陷预测模型XG-AJCC(AJCC-Ram+XGBoost).在预处理阶段，提出了AJCC-Ram(Adaptive Judgment Cure Clustering Random Sampling)多层次过采样方法.该方法基于改进的ADASYN自适应过采样和CURE-SMOTE过采样分别在类边缘和类中心层面生成新样本，通过CLNI方法对样本生成后的数据集进行噪声过滤及清理.在模型构建阶段，与集成算法XGBoost(eXtreme Gradient Boosting)相结合形成最终的不平衡数据缺陷预测模型.本文在AEEEM数据集和NASA数据集中进行了验证，实验结果表明：较于经典的采样方法和采样集成预测模型，在F1指标上AJCC-Ram过采样方法及XG-AJCC采样集成算法模型均能够取得有效的预测结果.     

Visual Motion Segmentation in Crowd Videos Based on Spatial-Angular Stacked Sparse Autoencoders

Visual motion segmentation (VMS) is an important and key part of many intelligent crowd systems. It can be used to figure out the flow behavior through a crowd and to spot unusual life-threatening incidents like crowd stampedes and crashes, which pose a serious risk to public safety and have resulted in numerous fatalities over the past few decades. Trajectory clustering has become one of the most popular methods in VMS. However, complex data, such as a large number of samples and parameters, makes it difficult for trajectory clustering to work well with accurate motion segmentation results. This study introduces a spatial-angular stacked sparse autoencoder model (SA-SSAE) with l2-regularization and softmax, a powerful deep learning method for visual motion segmentation to cluster similar motion patterns that belong to the same cluster. The proposed model can extract meaningful high-level features using only spatial-angular features obtained from refined tracklets (a.k.a ‘trajectories’). We adopt l2-regularization and sparsity regularization, which can learn sparse representations of features, to guarantee the sparsity of the autoencoders. We employ the softmax layer to map the data points into accurate cluster representations. One of the best advantages of the SA-SSAE framework is it can manage VMS even when individuals move around randomly. This framework helps cluster the motion patterns effectively with higher accuracy. We put forward a new dataset with its manual ground truth, including 21 crowd videos. Experiments conducted on two crowd benchmarks demonstrate that the proposed model can more accurately group trajectories than the traditional clustering approaches used in previous studies. The proposed SA-SSAE framework achieved a 0.11 improvement in accuracy and a 0.13 improvement in the F-measure compared with the best current method using the CUHK dataset.     

Generating Adversarial Samples on Multivariate Time Series using Variational Autoencoders

Classification models for multivariate time series have drawn the interest of many researchers to the field with the objective of developing accurate and efficient models. However, limited research has been conducted on generating adversarial samples for multivariate time series classification models. Adversarial samples could become a security concern in systems with complex sets of sensors. This study proposes extending the existing gradient adversarial transformation network (GATN) in combination with adversarial autoencoders to attack multivariate time series classification models. The proposed model attacks classification models by utilizing a distilled model to imitate the output of the multivariate time series classification model. In addition, the adversarial generator function is replaced with a variational autoencoder to enhance the adversarial samples. The developed methodology is tested on two multivariate time series classification models: 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN). This study utilizes 30 multivariate time series benchmarks provided by the University of East Anglia (UEA) and University of California Riverside (UCR). The use of adversarial autoencoders shows an increase in the fraction of successful adversaries generated on multivariate time series. To the best of our knowledge, this is the first study to explore adversarial attacks on multivariate time series. Additionally, we recommend future research utilizing the generated latent space from the variational autoencoders.      

Research of stacked denoising sparse autoencoder

Learning results depend on the representation of data, so how to efficiently represent data has been a research hot spot in machine learning and artificial intelligence. With the deepening of the deep learning research, studying how to train the deep networks to express high dimensional data efficiently also has been a research frontier. In order to present data more efficiently and study how to express data through deep networks, we propose a novel stacked denoising sparse autoencoder in this paper. Firstly, we construct denoising sparse autoencoder through introducing both corrupting operation and sparsity constraint into traditional autoencoder. Then, we build stacked denoising sparse autoencoders which has multi-hidden layers by layer-wisely stacking denoising sparse autoencoders. Experiments are designed to explore the influences of corrupting operation and sparsity constraint on different datasets, using the networks with various depth and hidden units. The comparative experiments reveal that test accuracy of stacked denoising sparse autoencoder is much higher than other stacked models, no matter what dataset is used and how many layers the model has. We also find that the deeper the network is, the less activated neurons in every layer will have. More importantly, we find that the strengthening of sparsity constraint is to some extent equal to the increase in corrupted level.

     

基于变分自编码器的异构缺陷预测特征表示方法

跨项目软件缺陷预测技术可以利用现有的已标注缺陷数据集对新的无标记项目进行预测,但需要两者之间具有相同的度量集合,难以用于实际开发.异构缺陷预测技术可以在具有异构度量集合的项目间进行缺陷预测,该技术引起了大量研究人员的关注.现有的异构缺陷预测技术利用朴素的或者传统机器学习方法为源项目和目标项目学习特征表示,所学习到的特征表示能力很弱且缺陷预测性能很差.鉴于深度神经网络强大的特征抽取和表示能力,本文基于变分自编码器技术提出了一种面向异构缺陷预测的特征表示方法.该模型结合了变分自编码器和最大均值差异距离,能有效地学习源项目和目标项目的共性特征表示,基于该特征表示可以训练出有效的缺陷预测模型.在多组缺陷数据集上通过与传统跨项目缺陷预测方法及异构缺陷预测方法实验对比验证了所提方法的有效性.     

Feature Fusion Models for Deep Autoencoders: Application to Traffic Flow Prediction

ABSTRACT

Due to reduction in dimensionality and extraction of the definitive features of input data, deep architectures have achieved significant success in various machine learning applications. Considering their successful applications in speech recognition and image classification, the main goal of this research is to investigate the performance of the sparse autoencoders utilized in regression analysis. To this end, deep sparse autoencoders with the standard method of training, cascaded, and partially cascaded architectures, fed with the fusion of low- and high-level features, are proposed and implemented. The regression task is to forecast the vehicular flow rate of a location on an arterial highway using different traffic variables of several locations ahead in the Twin Cities Metro area of Minneapolis. The results demonstrate that the partially cascaded model exhibits advancements in yielding more accurate results than the other two architectures fed with the features that correlate the most to the traffic flow rate.     

基于图自编码器的无监督多变量时间序列异常检测

针对多变量时间序列复杂的时间相关性和高维度使得异常检测性能较差的问题,以对抗训练框架为基础提出基于图自编码的无监督多变量时间序列异常检测模型.首先,将特征转换为嵌入向量来表示;其次,将划分好的时间序列结合嵌入向量转换为图结构数据;然后,用两个图自编码器模拟对抗训练重构数据样本;最后,根据测试数据在模型训练下的重构误差进行异常判定.将提出的方法与5种基线异常检测方法进行比较.实验结果表明,提出的模型在测试数据集获得了最高的F1分数,总体性能分F1分数比最新的异常检测模型USAD提高了28.4%.可见提出的模型有效提高异常检测性能.     

面向推荐系统中有偏和无偏一元反馈建模的三任务变分自编码器

一元反馈建模在推荐系统中的应用非常广泛,例如点击预测和购买预测等。然而,推荐系统作为一个闭环的反馈系统,在用户与系统的交互过程中可能存在着多种偏置问题,例如位置偏置、流行偏置等,进而导致用户的反馈数据存在有偏性。现有的大部分推荐模型都只基于这样的有偏数据进行构建,忽略了偏置的影响,进而导致推荐结果是次优的。目前已有的偏置消除方法大致可以分为基于反事实学习的方法、基于启发式的方法和基于无偏数据增强的方法。其中,基于无偏数据增强的方法通常被认为在稳定性和准确性方面表现较好。
本文重点研究了推荐系统中一元反馈的偏置问题,通过引入由一种特定策略收集的无偏数据,结合有偏数据进行联合建模,从而得到更准确和无偏的推荐模型。具体而言,本文从多任务学习的角度对问题进行建模,将有偏数据、无偏数据以及它们的并集当作三种相互关联的信号,并设计了三个不同但相关的学习任务。变分自编码器是目前最先进的一元反馈建模方法之一,有着独特的建模方式,从而使得它在很多问题中取得了优越的推荐效果。本文基于变分自编码器提出了一种新的推荐模型,即三任务变分自编码器（Tri-VAE）。该模型包含三个变分自编码器,分别对三种信号进行重构。三个变分自编码器之间共享同一个编码器和同一个解码器。此外,本文还设计了特征校正模块和标签增强模块以加强任务之间的关联。其中,特征校正模块用来校正用户的潜在特征,得到更无偏的潜在特征,进而从潜在特征的角度缓解偏置的影响。标签增强模块用于生成可靠性较高的伪标签并加以利用,进而更有效地利用无偏数据中的信息。
在Yahoo!R3和Coat Shopping两个公开数据集上的实验结果表明,所提出的模型相比于目前最新的基线模型在绝大多数情况下取得了显著的效果提升。为了进一步研究所提出的模型,本文进行了消融实验、超参数敏感性分析和收敛性分析,还对特征校正模块的有效性进行了探讨。     

Network Intrusion Detection in Internet of Blended Environment Using Ensemble of Heterogeneous Autoencoders (E-HAE)

Contemporary attackers, mainly motivated by financial gain, consistently devise sophisticated penetration techniques to access important information or data. The growing use of Internet of Things (IoT) technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation, as it facilitates multiple new attack vectors to emerge effortlessly. As such, existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems. To address this problem, we designed a blended threat detection approach, considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence. We collectively refer to the convergence of different technology sectors as the internet of blended environment. The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder. An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02% detection accuracy. Furthermore, performance of the proposed approach was compared with various single model (autoencoder)-based network intrusion detection approaches: autoencoder, variational autoencoder, convolutional variational autoencoder, and long short-term memory variational autoencoder. The proposed model outperformed all compared models, demonstrating F1-score improvements of 4.99%, 2.25%, 1.92%, and 3.69%, respectively.     

Information-theoretic regularization for learning global features by sequential VAE

Machine Learning - Sequential variational autoencoders (VAEs) with a global latent variable z have been studied for disentangling the global features of data, which is useful for several downstream...     

Classification of Parkinson's disease using feature weighting method on the basis of fuzzy C-means clustering

This study presents the application of fuzzy c-means (FCM) clustering-based feature weighting (FCMFW) for the detection of Parkinson's disease (PD). In the classification of PD dataset taken from University of California – Irvine machine learning database, practical values of the existing traditional and non-standard measures for distinguishing healthy people from people with PD by detecting dysphonia were applied to the input of FCMFW. The main aims of FCM clustering algorithm are both to transform from a linearly non-separable dataset to a linearly separable one and to increase the distinguishing performance between classes. The weighted PD dataset is presented to k-nearest neighbour (k-NN) classifier system. In the classification of PD, the various k-values in k-NN classifier were used and compared with each other. Also, the effects of k-values in k-NN classifier on the classification of Parkinson disease datasets have been investigated and the best k-value found. The experimental results have demonstrated that the combination of the proposed weighting method called FCMFW and k-NN classifier has obtained very promising results on the classification of PD.