李群机器学习十年研究进展

该文主要从3个方面介绍李群机器学习近年来的研究进展。首先,该文将解释为什么采用李群结构进行数据或特征描述,以此阐明李群机器学习与传统机器学习方法的区别,并且通过李群在人工智能领域的广泛应用来说明李群表示的普遍性。其次,该文概述了李群机器学习自提出以来的主要学习算法,着重强调最近的一些研究进展。最后,针对目前的研究现状,该文给出李群机器学习未来的一些研究方向。     

多连通李群覆盖学习算法在图像分类上的应用

李群机器学习作为一种新的学习范式已被学术界广泛关注。根据李群的连通性质,将具有不同类别特征的研究对象映射到多连通李群空间,并从各个单连通李群空间上连线的同伦等价出发,运用覆盖的思想寻找对应不同类别的最优道路等价表示,从而用多连通李群的多值表示来呈现图像的类别信息,因此提出了多连通李群覆盖学习算法。在MPEG7_CE-Shape01_Part_B图像库的图像和MNIST手写体数字图像上进行了实验验证,结果表明与两种基于李群均值的学习算法相比,多连通李群覆盖学习算法具有较好的分类效果。     

一种新的李群分类器在手写体数字中的应用

李群是变换空间的一种基本表示理论。目前针对李群数据所设计的分类器较少,对多分类的效果也不是很好。以手写体数字的应用为背景,引入了支持向量机分类算法来处理李群数据。由于李群数据具有矩阵表现的形式,设计了一种矩阵高斯核函数,使得支持向量机能够处理矩阵数据。仿真结果表明,支持向量机方法在李群数据上具有很好的性能。     

Some new simple Lie algebras in characteristic 2

We describe an algorithm for computing automorphism groups and testing isomorphisms of finite dimensional Lie algebras over finite fields. The algorithm is particularly effective for simple or almost simple Lie algebras. We show how it can be used in a computer search for new low dimensional simple Lie algebras over the field with two elements.     

李群平均方法在单轮姿态和运动控制中的应用

针对一个单轮系统,采用欧拉群描述其位形空间,对于Wei-Norman方程中不能精确求解的参数,采用平均方法对其进行近似计算,从而使被控系统在满足李代数可控性秩条件下,在李群上的平均轨迹能以一定的精度逼近真实轨迹.通过对单轮系统的仿真实验与结果分析,证明了所用方法的有效性,并指出该方法更适合驱动一类量子力学系统的状态.     

多李群覆盖学习优化算法

目前,已针对李群多连通空间上的道路交叉问题提出了多李群核覆盖学习算法,降低了道路交叉情况,使得分类正确率有了显著提高。但是,核学习算法的性能依赖于核函数的选择。考虑利用李群同态映射将原始李群样本映射到目标李群空间中,使在目标李群空间中不同单连通空间上的道路的关联度最小化,同一单连通空间上的道路的关联度最大化,从而减少道路交叉问题。     

量子机器学习

人工智能和量子物理是上世纪发展起来的两个截然不同但又影响深远的学科.近年来,它们在数据科学方面的结合引起了学术界的高度关注,形成了量子机器学习这个新兴领域.利用量子态的叠加性,量子机器学习有望通过量子并行解决目前机器学习中数据量大,训练过程慢的困难,并有望从量子物理的角度提出新的学习模型.目前该领域的研究还处于探索阶段,涵盖内容虽然广泛,但还缺乏系统的梳理.本文将从数据和算法角度总结量子机器学习与经典机器学习的不同,以及其中涉及的关键加速技巧,针对数据结构(数字型、模拟型),计算技巧(相位估计、Grover搜索、内积计算),基础算法(求解线性系统、主成分分析、梯度算法),学习模型(支持向量机、近邻法、感知器、玻尔兹曼机)等4个方面对现有研究成果进行综述,并建议一些可能的研究方向,供本领域的研究人员参考.     

基于流形学习的极端学习机及其在人脸识别中的应用

极端学习机以其快速高效和良好的泛化能力在模式识别领域得到了广泛应用,然而现有的ELM及其改进算法并没有充分考虑到数据维数对ELM分类性能和泛化能力的影响,当数据维数过高时包含的冗余属性及噪音点势必降低ELM的泛化能力,针对这一问题本文提出一种基于流形学习的极端学习机,该算法结合维数约减技术有效消除数据冗余属性及噪声对ELM分类性能的影响,为验证所提方法的有效性,实验使用普遍应用的图像数据,实验结果表明本文所提算法能够显著提高ELM的泛化性能。     

A study on effectiveness of extreme learning machine

Extreme learning machine (ELM), proposed by Huang et al., has been shown a promising learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Nevertheless, because of the random choice of input weights and biases, the ELM algorithm sometimes makes the hidden layer output matrix H of SLFN not full column rank, which lowers the effectiveness of ELM. This paper discusses the effectiveness of ELM and proposes an improved algorithm called EELM that makes a proper selection of the input weights and bias before calculating the output weights, which ensures the full column rank of H in theory. This improves to some extend the learning rate (testing accuracy, prediction accuracy, learning time) and the robustness property of the networks. The experimental results based on both the benchmark function approximation and real-world problems including classification and regression applications show the good performances of EELM.     

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.     

Ensemble of online sequential extreme learning machine

Liang et al. [A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Transactions on Neural Networks 17 (6) (2006), 1411–1423] has proposed an online sequential learning algorithm called online sequential extreme learning machine (OS-ELM), which can learn the data one-by-one or chunk-by-chunk with fixed or varying chunk size. It has been shown [Liang et al., A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Transactions on Neural Networks 17 (6) (2006) 1411–1423] that OS-ELM runs much faster and provides better generalization performance than other popular sequential learning algorithms. However, we find that the stability of OS-ELM can be further improved. In this paper, we propose an ensemble of online sequential extreme learning machine (EOS-ELM) based on OS-ELM. The results show that EOS-ELM is more stable and accurate than the original OS-ELM.     

Finite-time optimal formation control of multi-agent systems on the Lie group SE(3)

This paper investigates the finite-time optimal formation problem of multi-agent systems on the Lie group SE(3), for the situation when the formation time and/or the cost function need to be considered. Under the condition that the formation time is given according to the task requirement, a finite-time optimal formation controller is proposed for the two-agent case to guarantee that the desired formation is achieved at the given time and the corresponding cost function is optimal. For the systems with multiple agents, the obtained finite-time optimal formation control law has second-order approximation accuracy. Finally, some numerical simulations are provided to illustrate the effectiveness of the theoretical results.     

Evolutionary extreme learning machine

Extreme learning machine (ELM) [G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN2004), Budapest, Hungary, 25-29 July 2004], a novel learning algorithm much faster than the traditional gradient-based learning algorithms, was proposed recently for single-hidden-layer feedforward neural networks (SLFNs). However, ELM may need higher number of hidden neurons due to the random determination of the input weights and hidden biases. In this paper, a hybrid learning algorithm is proposed which uses the differential evolutionary algorithm to select the input weights and Moore-Penrose (MP) generalized inverse to analytically determine the output weights. Experimental results show that this approach is able to achieve good generalization performance with much more compact networks.     

动态模糊机器学习模型及验证

根据学习系统中存在的动态模糊性,提出了动态模糊机器学习模型,给出了动态模糊机器学习算法和它的几何模型描述,并进行了算法的稳定性分析,最后给出了实例验证。实例结果与BP算法产生结果相比较,优于BP算法的结果。     

一种基于鲁棒估计的极限学习机方法

极限学习机(ELM)是一种单隐层前馈神经网络(single-hidden layer feedforward neural networks,SLFNs),它相较于传统神经网络算法来说结构简单,具有较快的学习速度和良好的泛化性能等优点。ELM的输出权值是由最小二乘法(least square,LE)计算得出,然而经典的LS估计的抗差能力较差,容易夸大离群点和噪声的影响,从而造成训练出的参数模型不准确甚至得到完全错误的结果。为了解决此问题,提出一种基于M估计的采用加权最小二乘方法来取代最小二乘法计算输出权值的鲁棒极限学习机算法(RBELM),通过对多个数据集进行回归和分类分析实验,结果表明,该方法能够有效降低异常值的影响,具有良好的抗差能力。     

在线增量极限学习机及其性能研究

针对在线学习中极限学习机需要事先确定模型结构的问题,提出了兼顾数据增量和结构变化的在线极限学习机算法。算法于在线序列化极限学习机的基础上,通过误差变化判断是否新增节点,并利用分块矩阵的广义逆矩阵对新增节点后的模型进行更新,使模型保持较高正确率。通过在不同类型和大小的数据集上的实验表明,所提算法相较于经典极限学习机及其在线和增量学习版本都具有较好的分类和回归准确率,能够适应不同类型的数据分析任务。     

基于核极限学习机的标记分布学习

标记分布学习作为一种新的学习范式,利用最大熵模型构造的专用化算法能够很好地解决某些标记多样性问题,但是计算量巨大。基于此,引入运行速度快、稳定性更高的核极限学习机模型,提出基于核极限学习机的标记分布学习算法（KELM-LDL）。首先在极限学习机算法中通过RBF核函数将特征映射到高维空间,然后对原标记空间建立KELM回归模型求得输出权值,最后通过模型计算预测未知样本的标记分布。与现有算法在各领域不同规模数据集的实验表明,实验结果均优于多个对比算法,统计假设检验进一步说明KELM-LDL算法的有效性和稳定性。     

浅谈机器学习方法简

本文以什么是机器学习、机器学习的发展历史和机器学习的主要策略这一线索,对机器学习进行系统性的描述。接着,着重介绍了流形学习、李群机器学习和核机器学习三种新型的机器学习方法,为更好的研究机器学习提供了新的思路。     

Differential invariants of a Lie group action: Syzygies on a generating set

Given a group action, known by its infinitesimal generators, we exhibit a complete set of syzygies on a generating set of differential invariants. For that we elaborate on the reinterpretation of Cartan’s moving frame by Fels and Olver [Fels, M., Olver, P.J., 1999. Moving coframes. II. Regularization and theoretical foundations. Acta Appl. Math. 55 (2), 127–208]. This provides constructive tools for exploring algebras of differential invariants.     

Label distribution for multimodal machine learning

Multimodal machine learning(MML)aims to understand the world from multiple related modalities.It has attracted much attention as multimodal data has become increasingly available in real-world application.It is shown that MML can perform better than single-modal machine learning,since multi-modalities containing more information which could complement each other.However,it is a key challenge to fuse the multi-modalities in MML.Different from previous work,we further consider the side-information,which reflects the situation and influences the fusion of multi-modalities.We recover multimodal label distribution(MLD)by leveraging the side-information,representing the degree to which each modality contributes to describing the instance.Accordingly,a novel framework named multimodal label distribution learning(MLDL)is proposed to recover the MLD,and fuse the multimodalities with its guidance to learn an in-depth understanding of the jointly feature representation.Moreover,two versions of MLDL are proposed to deal with the sequential data.Experiments on multimodal sentiment analysis and disease prediction show that the proposed approaches perform favorably against state-of-the-art methods.