Joint Feature Representation and Classifier Learning Based Unsupervised Domain Adaption ELM

In the problem of unsupervised domain adaption Extreme learning machine (ELM), the output layer parameters need to have both classification and domain adaptation functions, which often cannot be simultaneously fully utilized. In addition, traditional matching method based on data probability distribution cannot find the common subspace of source and target domains under large difference between domains. In order to alleviate the pressure of double functions of classifier parameters, the entire ELM learning process is mainly divided into two stages: feature representation and adaptive classifier learning, thus a joint feature representation and classifier learning based unsupervised domain adaption ELM model is proposed. In the feature representation stage, the source and target domain data are projected to their respective subspace while minimizing the difference in probability distribution between the two domains. In the adaptive classifier learning stage, the smooth manifold regularization term of target domain is used to improve the parameter adaptive ability. Experiments on six different types of datasets show that the proposed model has higher cross-domain classification accuracy.     

深度复极限学习机在雷达HRRP目标识别中的应用

传统雷达高分辨一维距离像(High-resolution Range Profile,HRRP)目标识别方法只利用目标幅度信息而丢失其相位信息,这势必会造成信息不完备。为解决此问题,提出将深度极限学习机从实数域扩展到复数域,以有效提取复HRRP序列的深层潜在结构信息。同时为更好地保持数据间的邻域信息,将流形正则化引入到网络模型训练过程中,提出流形正则深度复极限学习机。在雷达暗室测量数据上的实验结果表明,所提算法相比常用的深度学习模型具有更好的识别效果和更快的训练速度,验证了算法的有效性。     

LLE-based classification algorithm for MMW radar target recognition

In this paper, a new multiclass classification algorithm is proposed based on the idea of Locally Linear Embedding (LLE), to avoid the defect of traditional manifold learning algorithms, which can not deal with new sample points. The algorithm defines an error as a criterion by computing a sample’s reconstruc-tion weight using LLE. Furthermore, the existence and characteristics of low dimensional manifold in range-profile time-frequency information are explored using manifold learning algorithm, aiming at the problem of target recognition about high range resolution MilliMeter-Wave (MMW) radar. The new algo-rithm is applied to radar target recognition. The experiment results show the algorithm is efficient. Com-pared with other classification algorithms, our method improves the recognition precision and the result is not sensitive to input parameters.     

Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization

Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. Several techniques have been developed and successfully applied for certain application domains. However, this work demands professional knowledge and expert experience. And sometimes it has to resort to the brute-force search. Therefore, if an efficient hyperparameter optimization algorithm can be developed to optimize any given machine learning method, it will greatly improve the efficiency of machine learning. In this paper, we consider building the relationship between the performance of the machine learning models and their hyperparameters by Gaussian processes. In this way, the hyperparameter tuning problem can be abstracted as an optimization problem and Bayesian optimization is used to solve the problem. Bayesian optimization is based on the Bayesian theorem. It sets a prior over the optimization function and gathers the information from the previous sample to update the posterior of the optimization function. A utility function selects the next sample point to maximize the optimization function. Several experiments were conducted on standard test datasets. Experiment results show that the proposed method can find the best hyperparameters for the widely used machine learning models, such as the random forest algorithm and the neural networks, even multi-grained cascade forest under the consideration of time cost.     

张量局部判别投影的人脸识别

 经典的向量子空间学习算法是以数据流形的向量表示进行计算的,但是在现实世界中数据流形从本质上而言是以张量的形式存在,因此基于张量子空间的学习算法能够更好地揭示流形内在的几何结构.本文提出了一种新的张量子空间的学习算法:张量局部判别投影.首先构建类内和类间图,然后保持流形的局部结构并且利用数据的判别信息,推导出算法的计算公式,最后通过迭代计算广义特征向量,解得最优张量子空间.在标准人脸数据库上的实验表明该算法有效.     

混合数据的邻域区分度增量式属性约简算法

增量式属性约简是一种针对动态环境下的数据挖掘方法.目前已经提出的增量式属性约简算法仅适用于符号型的信息系统,而很少有对混合信息系统进行相关的研究,这促使在混合信息系统下构建相关的增量式属性约简算法.区分度是用于设计属性约简的一种重要方法,本文将传统的区分度在混合信息系统下进行推广,提出邻域区分度的概念,然后分别研究了邻域区分度在混合信息系统下对象增加和对象减少时的增量式学习,最后根据这种增量式学习分别提出了对应的增量式属性约简算法.UCI数据集上的相关实验结果表明,所提出的增量式属性约简比非增量式属性约简能够更快速的更新约简结果.     

Evolutionary Extreme Learning Machine and Its Application to Image Analysis

Extreme learning machine (ELM) and evolutionary ELM (E-ELM) were proposed as a new class of learning algorithm for single-hidden layer feedforward neural network (SLFN). In order to achieve good generalization performance, E-ELM calculates the error on a subset of testing data for parameter optimization. Since E-ELMemploys extra data for validation to avoid the overfitting problem, more samples are needed for model training. In this paper, the cross-validation strategy is proposed to be embedded into the training phase so as to solve the overtraining problem. Based on this new learning structure, two extensions of E-ELM are introduced. Experimental results demonstrate that the proposed algorithms are efficient for image analysis.     

基于极限学习机的目标智能威胁感知技术

空中目标威胁感知是智能空战的关键技术之一。针对空中目标威胁评估问题,提出一种基于极限学习机的目标威胁智能感知策略。该方法首先提取目标威胁评估的高权重态势因子,并采用隶属度函数进行归一化数值解译;然后,借助专家知识采用极限学习机理论对威胁感知的输入输出数据进行建模,构建智能感知推理模型;最后,建立基于极限学习机的目标威胁智能感知流程。仿真结果表明,该算法具有较高的威胁感知精度以及较好的算法实时性。     

基于量子遗传的超参数自动调优算法的设计与实现

机器学习伴随着海量数据的支持以及强大的计算能力为其提供了强有力的保证下不断地向前发展,训练过程变得更加高效便捷。在此基础上,机器学习算法的超参数对其性能的影响是非常巨大的,因此对众多的超参数进行优化选择就自然有了强烈的需求。由此本文提出了一种基于量子遗传的超参数自动调优算法,实验表明,在针对多种机器学习模型的超参数调优问题上,既解决了一般随机算法的不稳定性的问题,也解决了一般进化算法迭代缓慢、收敛速度较低的问题,并且通过实验结果表明取得了不错的效果。     

一种基于Comid的非光滑损失随机坐标下降方法

坐标下降方法以简洁的操作流程、低廉的计算代价和快速的实际收敛效果,成为处理大规模优化最有效的方法之一.但目前几乎所有的坐标下降方法都由于子问题解析求解的需要而假设损失函数的光滑性.本文在结构学习的框架下,在采用Comid方法求解随机挑选单变量子问题的基础上,提出了一种新的关于非光滑损失的随机坐标下降方法.理论分析表明本文所提出的算法在一般凸条件下可以得到Ο(√t/t)的收敛速度,在强凸条件下可以得到Ο(lnt/t)的收敛速度.实验结果表明本文所提出的算法对正则化Hinge损失问题实现了坐标优化预期的效果.     

非平衡化标签补全核极限学习机多标签学习

目前众多的研究者通常直接将标签置信度矩阵作为先验知识直接加入到分类模型中,并没有考虑未标注先验知识对标签集质量的影响.基于此,引入非平衡参数的方法,将先验知识获得的基础置信度矩阵进行非平衡化,从而提出一种非平衡化的标签补全的核极限学习机多标签学习算法（KELM-NeLC）：首先使用信息熵计算标签之间的相关关系得到标签置信度矩阵,然后利用非平衡参数方法对基础的标签置信度矩阵进行改进,构建出一个非平衡的标签补全矩阵,最后为了学习获得更加准确的标签置信度矩阵,将非平衡化的标签补全矩阵与核极限学习机进行联合学习,依此解决多标签分类问题.提出的算法在公开的多个基准多标签数据集中的实验结果表明,KELM-NeLC算法较其他对比的多标签学习算法有一定优势,使用统计假设检验进一步说明所提出算法的有效性.     

基于优先级扫描Dyna结构的贝叶斯Q学习方法

贝叶斯Q学习方法使用概率分布来描述Q值的不确定性,并结合Q值分布来选择动作,以达到探索与利用的平衡。然而贝叶斯Q学习存在着收敛速度慢且收敛精度低的问题。针对上述问题,提出一种基于优先级扫描Dyna结构的贝叶斯Q学习方法—Dyna-PS-BayesQL。该方法主要分为2部分：在学习部分,对环境的状态迁移函数及奖赏函数建模,并使用贝叶斯Q学习更新动作值函数的参数;在规划部分,基于建立的模型,使用优先级扫描方法和动态规划方法对动作值函数进行规划更新,以提高对历史经验信息的利用,从而提升方法收敛速度及收敛精度。将Dyna-PS-BayesQL应用于链问题和迷宫导航问题,实验结果表明,该方法能较好地平衡探索与利用,且具有较优的收敛速度及收敛精度。     

基于光谱梯度角的高光谱影像流形学习降维法

为了挖掘高光谱数据的光谱局部特征,从高光谱遥感数据内在的非线性结构出发,提出了一种基于光谱梯度角的高光谱影像流形学习降维方法。采用局部化流形学习算法局部保持投影（LPP）对高光谱遥感数据进行非线性降维,对距离度量进行改进,将能够更好刻画高光谱影像光谱局部特征的光谱梯度角相似性度量应用于LPP方法,并用真实高光谱图像进行降维实验,取得了优于LPP方法和采用光谱角的LPP方法的结果。结果表明,在光谱规范化特征值方面,所提方法优于LPP方法和采用光谱角的LPP方法;在信息量的保持方面,具有更好的局部细节信息保持量。采用光谱梯度角的流形学习方法用于高光谱影像降维能取得较好的降维效果。     

流形学习和相关反馈相融合的医学图像检索

图像检索是医学图像辅助诊断的基础,为了提高医学图像检索的正确率,提出一种流形学习和相关反馈相融合的医学图像检索算法(LLE-MF)。首先根据方块编码的思想提取颜色分量的信息熵,并利用邻域灰度共生矩阵提取纹理特征;然后采用非线性流形学习对颜色和纹理特征进行组合、降维处理,并采用欧式距离相似度量模型对图像初步进行检索,最后最小二乘支持向量机对初步检索结果进行相关反馈,并进行仿真测试。结果表明,相对于其它医学检索算法,LLE-MF不仅提高了医学图像的检索准确率,同时提高了医学图像的检索效率,可以准确地找到用户所需的图像.     

基于多任务学习的立体匹配算法

引入辅助任务信息有助于立体匹配模型理解相关知识,但也会增加模型训练的复杂度。为解决模型训练对额外标签数据的依赖问题,提出了一种利用双目图像的自相关性进行多任务学习的立体匹配算法。该算法在多层级渐进细化过程中引入了边缘和特征一致性信息,并采用循环迭代的方式更新视差图。根据双目图像中视差的局部平滑性和左右特征一致性构建了损失函数,在不依赖额外标签数据的情况下就可以引导模型学习边缘和特征一致性信息。提出了一种尺度注意的空间金字塔池化,使模型能够根据局部图像特征来确定不同区域中不同尺度特征的重要性。实验结果表明:辅助任务的引入提高了视差图精度,为视差图的可信区域提供了重要依据,在无监督学习中可用于确定单视角可见区域;在KITTI2015测试集上,所提算法的精度和运行效率均具有一定的竞争力。     

基于稀疏贝叶斯的流形学习

针对当前监督学习算法在流形数据集上分类性能的缺陷,如分类精度低且稀疏性有限,本文在稀疏贝叶斯方法和流行正则化框架的基础上,提出一种稀疏流形学习算法（Manifold Learning Based on Sparse Bayesian Approach,MLSBA）.该算法是对稀疏贝叶斯模型的扩展,通过在模型的权值上定义稀疏流形先验,有效利用了样本数据的流形信息,提高了算法的分类准确率.在多种数据集上进行实验,结果表明：MLSBA不仅在流形数据集上取得良好的分类性能,而且在非流形数据集上效果也比较好;同时算法在两类数据集上均具有良好的稀疏性能.     

面向机器学习的隐私保护关键技术研究综述

随着信息通信技术的发展,机器学习已经成为多个研究领域与垂直行业必不可少的技术工具。然而,机器学习所需数据中往往包含了大量的个人信息,使其隐私保护面临风险与挑战,受到了越来越多的关注。对现有机器学习下隐私保护法规政策与标准化现状进行梳理,对适用于机器学习的隐私保护技术进行详细介绍与分析。隐私保护算法通常会对数据质量、通信开支与模型表现等造成影响,因此对于隐私保护算法的评估应当进行多维度的综合评估。总结了适用于机器学习应用的隐私保护性能评估指标,并指出隐私保护需要考虑对数据质量、通信开支以及模型准确率等之间的影响。     

直接线性图嵌入算法及其在人脸识别中的应用

图嵌入算法使用无向有权图来描述数据集的流形结构,目前许多流形学习算法都可统一到这个框架下。线性图嵌入算法(LGE)在高维小样本应用中往往会遇到的奇异值问题,因此需把数据集预先投影到PCA子空间,往往会丢失了一些有用的信息。本文提出了一种直接的线性图嵌入算法(DLGE),可直接从原始数据集中提取特征。此外DLGE算法相对于基于迭代的正交化算法,在最小二乘意义下对截断的征向量进行正交化处理,计算简便有效。在多个人脸数据库库上的仿真结果表明,相对于传统算法,DLGE算法具有更强的人脸表征能力,更好的分类性能,且更加鲁棒。     

An efficient pursuit automata approach for estimating stable all‐pairs shortest paths in stochastic network environments

This paper presents a new solution to the dynamic all‐pairs shortest‐path routing problem using a fast‐converging pursuit automata learning approach. The particular instance of the problem that we have investigated concerns finding the all‐pairs shortest paths in a stochastic graph, where there are continuous probabilistically based updates in edge‐weights. We present the details of the algorithm with an illustrative example. The algorithm can be used to find the all‐pairs shortest paths for the ‘statistical’ average graph, and the solution converges irrespective of whether there are new changes in edge‐weights or not. On the other hand, the existing popular algorithms will fail to exhibit such a behavior and would recalculate the affected all‐pairs shortest paths after each edge‐weight update. There are two important contributions of the proposed algorithm. The first contribution is that not all the edges in a stochastic graph are probed and, even if they are, they are not all probed equally often. Indeed, the algorithm attempts to almost always probe only those edges that will be included in the final list involving all pairs of nodes in the graph, while probing the other edges minimally. This increases the performance of the proposed algorithm. The second contribution is designing a data structure, the elements of which represent the probability that a particular edge in the graph lies in the shortest path between a pair of nodes in the graph. All the algorithms were tested in environments where edge‐weights change stochastically, and where the graph topologies undergo multiple simultaneous edge‐weight updates. Its superiority in terms of the average number of processed nodes, scanned edges and the time per update operation, when compared with the existing algorithms, was experimentally established. Copyright © 2008 John Wiley & Sons, Ltd.