无监督混阶栈式稀疏自编码器的图像分类学习

目前多数图像分类的方法是采用监督学习或者半监督学习对图像进行降维,然而监督学习与半监督学习需要图像携带标签信息。针对无标签图像的降维及分类问题,提出采用混阶栈式稀疏自编码器对图像进行无监督降维来实现图像的分类学习。首先,构建一个具有三个隐藏层的串行栈式自编码器网络,对栈式自编码器的每一个隐藏层单独训练,将前一个隐藏层的输出作为后一个隐藏层的输入,对图像数据进行特征提取并实现对数据的降维。其次,将训练好的栈式自编码器的第一个隐藏层和第二个隐藏层的特征进行拼接融合,形成一个包含混阶特征的矩阵。最后,使用支持向量机对降维后的图像特征进行分类,并进行精度评价。在公开的四个图像数据集上将所提方法与七个对比算法进行对比实验,实验结果表明,所提方法能够对无标签图像进行特征提取,实现图像分类学习,减少分类时间,提高图像的分类精度。     

基于伪标签的无监督领域自适应分类方法

现有的领域自适应方法在匹配分布时并未完全考虑伪标签置信度或伪标签损失计算问题,针对此类问题,提出循环选择伪标签分类模型（CSPL）。利用深度网络提取图像特征,为目标域打上高置信度伪标签使得训练数据增强,采用MMD距离度量方法对齐源域和目标域的概率分布,同时设计伪标签损失同步迭代学习,更新模型作为下一次循环的训练模型直至模型收敛。在常用的领域自适应数据集Office31、Office-Home、ImageCLEF-DA以及Amazon-Review上实验表明,该模型相比之前域适应模型在准确度方面平均提升4%~8%且模型的鲁棒性也明显增加。     

Self-corrected unsupervised domain adaptation

Unsupervised domain adaptation (UDA), which aims to use knowledge from a label-rich source domain to help learn unlabeled target domain, has recently attracted much attention. UDA methods mainly concentrate on source classification and distribution alignment between domains to expect the correct target prediction. While in this paper, we attempt to learn the target prediction end to end directly, and develop a Self-corrected unsupervised domain adaptation (SCUDA) method with probabilistic label correction. SCUDA adopts a probabilistic label corrector to learn and correct the target labels directly. Specifically, besides model parameters, those target pseudo-labels are also updated in learning and corrected by the anchor-variable, which preserves the class candidates for samples. Experiments on real datasets show the competitiveness of SCUDA.     

注意力迁移的联合平衡领域自适应

目的 现有的图像识别方法应用于从同一分布中提取的训练数据和测试数据时具有良好性能,但这些方法在实际场景中并不适用,从而导致识别精度降低。使用领域自适应方法是解决此类问题的有效途径,领域自适应方法旨在解决来自两个领域相关但分布不同的数据问题。方法 通过对数据分布的分析,提出一种基于注意力迁移的联合平衡自适应方法,将源域有标签数据中提取的图像特征迁移至无标签的目标域。首先,使用注意力迁移机制将有标签源域数据的空间类别信息迁移至无标签的目标域。通过定义卷积神经网络的注意力,使用关注信息来提高图像识别精度。其次,基于目标数据集引入网络参数的先验分布,并且赋予网络自动调整每个领域对齐层特征对齐的能力。最后,通过跨域偏差来描述特定领域的特征对齐层的输入分布,定量地表示每层学习到的领域适应性程度。结果 该方法在数据集Office-31上平均识别准确率为77.6%,在数据集Office-Caltech上平均识别准确率为90.7%,不仅大幅领先于传统手工特征方法,而且取得了与目前最优的方法相当的识别性能。结论 注意力迁移的联合平衡领域自适应方法不仅可以获得较高的识别精度,而且能够自动学习领域间特征的对齐程度,同时也验证了进行域间特征迁移可以提高网络优化效果这一结论。     

基于双矫正机制的源域无关域适应学习

标准域无监督域适应学习是从相关的源域学习知识迁移到目标域,通常假设源域数据在训练阶段是可直接使用的。但是由于隐私和安全问题,在一些现实的应用中,源域数据往往是不可直接获取的,如何有效利用目标域数据从而减少噪声类的输出或特征的产生是源域无关域适应学习的巨大挑战。为解决这个问题,提出了一个基于双矫正机制的源域无关域适应学习模型（source-free domain adaptation with dual-correction mechanism,DCM）。首先,探索目标域样本信息结构,对噪声类输出进行矫正;其次,采用教师—学生模型指导特征的学习,最大化高置信度特征间的一致性以及低置信度特征间的差异性。最后,在数字集、Office-31和Office-Home数据集上的实验结果证实了DCM的有效性。     

多尺度非监督特征学习的人脸识别

为了充分利用人脸图像的潜在信息,提出一种通过设置不同尺寸的卷积核来得到图像多尺度特征的方法,多尺度卷积自动编码器（Multi-Scale Convolutional Auto-Encoder,MSCAE）。该结构所提取的不同尺度特征反映人脸的本质信息,可以更好地还原人脸图像。这种特征提取框架是一个卷积和采样交替的层级结构,使得特征对旋转、平移、比例缩放等具有高度不变性。MSCAE以encoder-decoder模式训练得到特征提取器,用它提取特征,并融合形成用于分类的特征向量。BP神经网络在ORL和Yale人脸库上的分类结果表明,多尺度特征在识别率和性能上均优于单尺度特征。此外,MSCAE特征与HOG（Histograms of Oriented Gradients）的融合特征取得了比单一特征更高的识别率。     

面向无监督分割的双分支上采样域自适应网络

工业应用中,表面压印字符图像全监督语义分割将会给企业带来高昂的数据集标注成本,针对该问题,提出了双支路特征融合的域适应分割方法（dual-branch feature fusion domain adaptation,DbFFDA）。借鉴U-Net的跨层连接设计思路,提出了双分支上采样结构的残差域适应分割网络（residual adaptation network,Res-Adp）;同时提出了融合特征输入用于提升网络分割性能,克服了字符缺失的问题;此外,提出了分割连续性损失函数LCon,抑制了分割图像中噪点的产生。在石墨电极表面压印字符无监督分割实验中,所提方法MIoU值可达69.60%,实际分割效果已基本满足字符识别需求,有望在特定工业场景中投入实际应用,为企业节省数据集标注成本。     

Multi-source unsupervised domain adaptation for object detection

Domain adaptation for object detection has been extensively studied in recent years. Most existing approaches focus on single-source unsupervised domain adaptive object detection. However, a more practical scenario is that the labeled source data is collected from multiple domains with different feature distributions. The conventional approaches do not work very well since multiple domain gaps exist. We propose a Multi-source domain Knowledge Transfer (MKT) method to handle this situation. First, the low-level features from multiple domains are aligned by learning a shallow feature extraction network. Then, the high-level features from each pair of source and target domains are aligned by the followed multi-branch network. After that, we perform two parts of information fusion: (1) We train a detection network shared by all branches based on the transferability of each source sample feature. The transferability of a source sample feature means the indistinguishable degree to the target domain sample features. (2) For using our model, the target sample features output by the multi-branch network are fused based on the average transferability of each domain. Moreover, we leverage both image-level and instance-level attention to promote positive cross-domain transfer and suppress negative transfer. Our main contributions are the two-stage feature alignments and information fusion. Extensive experimental results on various transfer scenarios show that our method achieves the state-of-the-art performance.     

基于对抗学习的医学图像分割领域自适应研究

为了解决跨领域医学图像分析中不匹配的问题,提出了一种基于对抗学习的无监督领域自适应框架（UAL-DAF）。具体而言,该框架通过外观转移模块（ATM）和结合条件生成对抗网络的语义转移模块（STM）分别缩小了跨领域医学图像外观和语义层次的差异。最后,在具有挑战性的医学图像分割实验中,结果显著优于已有方法。因此,该框架能够提取领域自适应知识的外观和语义层次信息,实现领域知识的协同融合。     

基于稀疏自编码深度神经网络的林火图像分类

针对林火与相似目标很难区分的问题,提出一种基于稀疏自编码深度神经网络的林火图像分类新方法。采用无监督的特征学习算法稀疏自编码从无标签图像小块中学习特征参数,完成深度神经网络的训练;利用学习到的特征从原始大小分类图像中提取特征并卷积和均值池化特征;对卷积和池化后的特征采用softmax回归来训练最终softmax分类器。实验结果表明,跟传统的BP神经网络相比,新方法能够更有效区分林火与红旗、红叶等类似物体。     

A kernel learning framework for domain adaptation learning

Domain adaptation learning(DAL) methods have shown promising results by utilizing labeled samples from the source(or auxiliary) domain(s) to learn a robust classifier for the target domain which has a few or even no labeled samples.However,there exist several key issues which need to be addressed in the state-of-theart DAL methods such as sufficient and effective distribution discrepancy metric learning,effective kernel space learning,and multiple source domains transfer learning,etc.Aiming at the mentioned-above issues,in this paper,we propose a unified kernel learning framework for domain adaptation learning and its effective extension based on multiple kernel learning(MKL) schema,regularized by the proposed new minimum distribution distance metric criterion which minimizes both the distribution mean discrepancy and the distribution scatter discrepancy between source and target domains,into which many existing kernel methods(like support vector machine(SVM),v-SVM,and least-square SVM) can be readily incorporated.Our framework,referred to as kernel learning for domain adaptation learning(KLDAL),simultaneously learns an optimal kernel space and a robust classifier by minimizing both the structural risk functional and the distribution discrepancy between different domains.Moreover,we extend the framework KLDAL to multiple kernel learning framework referred to as MKLDAL.Under the KLDAL or MKLDAL framework,we also propose three effective formulations called KLDAL-SVM or MKLDAL-SVM with respect to SVM and its variant μ-KLDALSVM or μ-MKLDALSVM with respect to v-SVM,and KLDAL-LSSVM or MKLDAL-LSSVM with respect to the least-square SVM,respectively.Comprehensive experiments on real-world data sets verify the outperformed or comparable effectiveness of the proposed frameworks.     

多标记学习自编码网络无监督维数约简

多标记学习是针对一个实例同时与一组标签相关联而提出的一种机器学习框架,是该领域研究热点之一,降维是多标记学习一个重要且具有挑战性的工作。针对有监督的多标记维数约简方法,提出一种无监督自编码网络的多标记降维方法。首先,通过构建自编码神经网络,对输入数据进行编码和解码输出;然后,引入稀疏约束计算总体成本,使用梯度下降法进行迭代求解;最后,通过深度学习训练获得自编码网络学习模型,提取数据特征实现维数约简。实验中使用多标记算法ML-kNN做分类器,在6个公开数据集上与其他4种方法对比。实验结果表明,该方法能够在不使用标记的情况下有效提取特征,降低多标记数据维度,稳定提高多标记学习性能。     

Representation learning: serial-autoencoder for personalized recommendation

Nowadays, the personalized recommendation has become a research hotspot for addressing information overload. Despite this, generating effective recommendations from sparse data remains a challenge. Recently, auxiliary information has been widely used to address data sparsity, but most models using auxiliary information are linear and have limited expressiveness. Due to the advantages of feature extraction and no-label requirements, autoencoder-based methods have become quite popular. However, most existing autoencoder-based methods discard the reconstruction of auxiliary information, which poses huge challenges for better representation learning and model scalability. To address these problems, we propose Serial-Autoencoder for Personalized Recommendation (SAPR), which aims to reduce the loss of critical information and enhance the learning of feature representations. Specifically, we first combine the original rating matrix and item attribute features and feed them into the first autoencoder for generating a higher-level representation of the input. Second, we use a second autoencoder to enhance the reconstruction of the data representation of the prediciton rating matrix. The output rating information is used for recommendation prediction. Extensive experiments on the MovieTweetings and MovieLens datasets have verified the effectiveness of SAPR compared to state-of-the-art models.     

Constructing a health indicator for bearing degradation assessment via an unsupervised and enhanced stacked autoencoder

To eliminate the noise from the extracted health indicator (HI) and reduce dependence on manual experience including knowledge for selecting the appropriate time–frequency domain indicators and conversions for data pre-processing, multiple-step processing, and multiple model combinations. An enhanced stacked autoencoder (ESAE) based on an exponent weight moving average (EWMA) in a deep learning model is proposed. This ESAE uses the vibration amplitude of an unlabeled original time-domain signal to construct the HI directly. To demonstrate our proposed method is better than other models, including the stacked autoencoder (SAE), stacked denoising autoencoder (SDAE), root mean square (RMS), kurtosis, K-medoids clustering, and self-organizing map (SOM) neural network models. The proposed model is simulated for multiple bearings in two case studies. The experiment result shows that the extracted HI curve is smoother than mentioned-above other models reduce the wrong judgment of bearing health caused by noise effectively. Moreover, high monotonicity is lay a good foundation for following reaming useful life prediction. An index, named Mon, is used to assess the monotonicity performance of all models, the experiment result also shows that the extracted HI by our proposed is superior to other models.     

面向眼底图像小目标检测的无监督学习方法

小目标检测是图像处理领域的一个难点,尤其是医学图像中的小目标检测。微动脉瘤MA作为眼底图像中的一类小目标,尺寸小、局部对比度较低,并且存在较多的噪声干扰,检测难度较大。传统的检测方法需要手工提取特征,难以准确检测MA。而基于深度学习的检测需要进行复杂的前期准备工作,工作量大,并且难以解决正负样本数量不平衡的问题,容易产生过拟合。稀疏编码器SAE是一种无监督机器学习算法,可以在样本数量不平衡的环境中有效地提取样本的特征。因此,提出了一种基于SAE的无监督学习方法检测MA,采用反向传播更新SAE的权重和偏置以提取样本的特征,并利用提取的特征训练Softmax,最终实现MA的准确检测。为验证方法性能,选取了Retinopathy Online Challenge、DIARETDB1和E-ophtha-MA 3个数据库分别进行实验。实验结果表明,本文方法能够准确地检测出眼底图像中的MA,并且获得了较高的准确率和灵敏度。准确率分别为98.5%,87.2%和92.6%,灵敏度分别为99.9%,99.8%和98.7%。     

Neurocontroller design via supervised and unsupervised learning

In this paper we study the role of supervised and unsupervised neural learning schemes in the adaptive control of nonlinear dynamic systems. We suggest and demonstrate that the teacher's knowledge in the supervised learning mode includes a-priori plant sturctural knowledge which may be employed in the design of exploratory schedules during learning that results in an unsupervised learning scheme. We further demonstrate that neurocontrollers may realize both linear and nonlinear control laws that are given explicitly in an automated teacher or implicitly through a human operator and that their robustness may be superior to that of a model based controller. Examples of both learning schemes are provided in the adaptive control of robot manipulators and a cart-pole system.     

渐进式分离的开放集模糊域自适应算法

域自适应的目的是利用有标记（源）域中的信息来提高未标记（目标）域模型的分类性能,且这种方法已经取得了不错的成果。然而在具有开放性的现实场景下,目标域通常包含源域中未观察到的未知类样本,这被称为开放集域自适应问题。传统的域自适应算法对这样具有挑战性的场景设定无能为力,因此提出了渐进式分离的开放集模糊域自适应算法。首先,基于引进隶属度的开放集模糊域自适应算法,探索了逐步分离目标域中已知类和未知类样本的方法;然后,仅将从目标域中分离出的已知类与源域对齐,从而减小两个域之间的分布差异,进行模糊域自适应。所提算法很好地解决了由于未知类和已知类之间的不匹配而导致的负迁移所带来的影响。在Office数据集上的6组域自适应转化实验结果表明,与传统的域自适应算法比较,所提算法在图像分类中的精度有显著的提升,验证了该算法可以逐步增强域自适应分类模型的准确性和鲁棒性。     

基于熵增强的无监督域适应遥感图像语义分割

为了实现利用有标注源数据获得在无标注目标数据上可用的遥感图像语义分割模型,提出了一种基于熵增强的域适应端到端语义分割方法.首先,为了充分利用遥感图像多尺度信息并且减少域之间传感器分辨率带来的域偏移,采用空洞空间金字塔池化模块作为分类器;其次,为了使无标注的目标域类别正确对应,使用了两个分类器进行协同训练;将像素点预测值的信息熵当做分类置信度的度量,将其作为对抗损失的权重,从而使训练能专注于难分类的像素,降低域偏移.在ISPRS(WGII/4)2D数据集上进行实验,所提方法相对于直接使用分割模型和使用传统对抗方法,mIoU分别提高了18％和12％.实验结果表明,所提方法在遥感图像域适应语义分割表现上优于直接使用分割模型或使用传统对抗域适应分割方法.     

稀疏自编码和Softmax回归的快速高效特征学习

针对特征学习效果与时间平衡问题,提出了一种快速高效的特征学习方法.将稀疏自编码和Softmax回归组合成一个新的特征提取模型,在提取原始图像潜在信息的基础上,利用多分类器返回值可以反映输入信息的相似程度的特点,快速高效的学习利于分类的特征向量.鉴于标签信息已知,该算法在图像分类效果上明显优于几种典型的特征学习方法.为了使所提算法具有更好的泛化能力,回归模型的损失函数中加入了L2范数防止过拟合,同时,采用随机梯度下降的方法得到模型的最优参数.在4个标准数据集上的测试结果表明该算法是有效可行的.