一种模糊森林学习方法及其行人检测应用

针对随机森林学习方法训练数据时存在的过拟合问题,通过改进各决策节点的决策函数设计一种模糊森林学习方法。利用高斯隶属度函数构建决策树上各节点的决策函数,将确定决策路径转换为模糊决策路径。根据样本从根节点到叶节点所经过的所有决策节点的模糊决策值乘积生成模糊路径。结合各模糊路径与相应叶节点预测参数得到预测结果。将模糊森林学习方法应用到行人检测领域,分别对Haar特征和方向梯度直方图特征进行学习与分类。实验结果表明,与经典的Adaboost、支持向量机和随机森林分类器相比,模糊森林方法可有效提高行人检测的识别率。     

基于因果影响独立模型的贝叶斯网络参数学习

基于因果影响独立模型及其中形成的特定上下文独立关系,提出一种适于样本学习的贝叶斯网络参数学习算法。该算法在对局部概率模型降维分解的基础上,通过单父节点条件下的子节点概率分布来合成局部结构的条件概率分布,参数定义复杂度较低且能较好地处理稀疏结构样本集。实验结果表明,该算法与标准最大似然估计算法相比,能充分利用样本信息,具有较好的学习精度。     

一种基于簇的极限学习机的在线学习算法

针对传统的批量学习算法学习速度慢、对空间需求量高的缺点,提出了一种基于簇的极限学习机的在线学习算法。该算法将分簇的理念融入到极限学习机中,并结合极限学习机,提出了一种基于样本类别和样本输出的分簇标准;同时提出了一种加权的Moore-Penrose算法求隐层节点与输出节点的连接权重。实验结果表明,该算法具有学习能力好、拟合度高、泛化性能好等优点。     

基于生成对抗网络和元路径的异质网络表示学习

现实世界中的信息网络大多为异质信息网络,旨在表示低维空间中节点数据的网络表示方法已普遍用于分析异质信息网络,从而有效融合异质网络中丰富的语义信息和结构信息。但是现有的异质网络表示方法通常采用负采样从网络中随机选择节点,并且对节点和边的异质性学习能力不足。受生成式对抗网络和元路径的启发,文中提出了一种新型的异质网络表示方法。首先对采样方法使用元路径的策略进行改进,根据元路径不同的权重取样,使样本更好地体现节点之间存在的直接和间接关系,增强样本的语义关联。然后在生成对抗的博弈过程中使模型充分考虑节点和边的异质性并具备关系感知能力,实现对异质信息网络的表示学习。实验结果表明,与目前的表示算法相比,该模型学习到的表示向量在分类和链路预测实验中具有更好的性能表现。     

一种基于云模型的贝叶斯网络EM参数学习算法

针对贝叶斯网络连续节点离散化后,概念知识表达存在模糊性和随机性的问题,提出一种将云模型与EM(Expectations Maximization)算法相结合的贝叶斯网络参数学习算法。首先运用启发式高斯云变换算法(Heuristic Gaussian Cloud Transformation)和云发生器将连续节点定量样本转换成定性概念,并记录下样本对所属概念的确定度,运用确定度概率转换公式将确定度转换成相应概率;随后复制扩充样本并按概率选择所属概念;样本更新后结合EM算法进行参数优化,实现贝叶斯网络的参数学习。仿真实验结果表明,通过云模型表征概念得到的参数学习结果更加符合实际情况,参数学习精度和网络推理准确性得到了提高。     

容量约束的自组织增量联想记忆模型

自组织联想记忆神经网络因其并行、容错及自我学习等优点而得到广泛应用,但现有主流模型在增量学习较大规模样本时,网络节点数可能无限增长,从而给实际应用带来不可承受的内存及计算开销。针对该问题,提出了一种容量约束的自组织增量联想记忆模型。以网络节点数为先决控制参数,结合设计新的节点间自竞争学习策略,新模型可满足大规模样本的增量式学习需求,并能以较低的计算容量取得较高的联想记忆性能。理论分析表明了新模型的正确性与有效性,实验分析同时显示了新模型可有效控制计算容量,提升增量样本学习效率,并获得较高的联想记忆性能,从而能更好地满足现实应用需求。     

基于线性SVM的级联检测算子的构建

本文介绍了一种基于支持向量机(SVM)理论的线性级联式分类器,用于解决较复杂目标的快速检测问题.该分类器由若干个线性SVM分类器组成,结合了级联分类器和SVM理论的优点,给出了级联结构中的每个节点的约束最优化模型,使得每个节点都有较高的正样本检测率和适当的负样本错检率.实验结果表明,与经典非线性svM分类器相比,这种分类器在保持SCM较强泛化性能的优点的同时,在检测效率方面更是具有明显的优势.     

一种反馈过程神经元网络模型及在动态信号分类中的应用*

针对动态信号模式分类问题,提出了一种反馈过程神经元网络模型和基于该模型的分类方法。这种网络的输入可直接为时变函数,网络的信息传输既有与前馈神经元网络一样的前向流,也有后面各层节点到前层节点的反馈,且可对节点自身反馈输出信息,能直接用于动态信号的模式分类。由于反馈过程神经元网络在对输入样本的学习中增加了神经元输出信息的反馈,可提高网络的学习效率和稳定性。给出了具体学习算法,以时变函数样本集的分类问题为例,实验结果验证了模型和算法的有效性。     

无线传感器网络中移动节点定位算法研究

提出一种利用临时锚节点的蒙特卡罗箱定位算法.该算法是基于蒙特卡罗定位方法之上,通过引入节点平均速率来获取临时锚节点,并利用一跳范围内的临时锚节点构建最小锚盒、增强样本过滤条件,从而加速了采样和样本过滤.此外,在样本的获取上采用了非随机采样的均衡采样方法,有效地降低了采样次数.仿真结果表明:该算法同蒙特卡罗定位算法等相比,提高了节点的定位精度,降低了节点的能耗.     

一种基于叶分量分析的带有监督信息的在线学习方法

在一些模式识别应用中,具有类属信息的样本数量较少,此时监督学习算法会遇到小样本问题,导致分类器的识别精度大幅低于预期水平.基于叶分量分析,提出一种带监督信息的在线学习方法.该方法在训练过程进行监督学习,而在模式识别阶段能够在对输入样本进行分类的同时基于这些样本进行非监督在线学习,因此实现了监督学习与非监督学习的结合.在小本量情况下,在线学习可以弥补训练阶段监督学习的不足,仍能保证获得较高的识别精度.实验证明,该方法能够有效克服小样本问题.     

面向流数据分类的在线学习综述

流数据分类旨在从连续不断到达的流式数据中增量学习一个从输入变量到类标变量的映射函数,以便对随时到达的测试数据进行准确分类.在线学习范式作为一种增量式的机器学习技术,是流数据分类的有效工具.主要从在线学习的角度对流数据分类算法的研究现状进行综述.具体地,首先介绍在线学习的基本框架和性能评估方法,然后着重介绍在线学习算法在一般流数据上的工作现状,在高维流数据上解决“维度诅咒”问题的工作现状,以及在演化流数据上处理“概念漂移”问题的工作现状,最后讨论高维和演化流数据分类未来仍然存在的挑战和亟待研究的方向.     

基于网络编码的对等网流媒体传输模型和算法

在对等网(peer-to-peer,简称P2P)流媒体系统中,节点(用户)的输出带宽(上行带宽)容量利用率的提高能够降低服务器的带宽开销.网络编码可以实现组播的最大吞吐率,因而具有提高系统中节点输出带宽容量利用率的潜力.将随机线性网络编码应用到P2P流媒体系统中,建立了基于随机线性网络编码的P2P流媒体传输过程模型,并据此建立传输算法的优化模型,比较研究了贪婪式算法、最少者优先算法和随机算法等.优化结果表明,随机算法可以平等均匀地获取数据包,能够最充分地利用节点的输出带宽容量,降低服务提供商的运营成本.通过对优化模型解的分析对实际系统中的传输算法给出了设计指导原则.     

Incremental Learning of Linear Model Trees

A linear model tree is a decision tree with a linear functional model in each leaf. Previous model tree induction algorithms have been batch techniques that operate on the entire training set. However there are many situations when an incremental learner is advantageous. In this article a new batch model tree learner is described with two alternative splitting rules and a stopping rule. An incremental algorithm is then developed that has many similarities with the batch version but is able to process examples one at a time. An online pruning rule is also developed. The incremental training time for an example is shown to only depend on the height of the tree induced so far, and not on the number of previous examples. The algorithms are evaluated empirically on a number of standard datasets, a simple test function and three dynamic domains ranging from a simple pendulum to a complex 13 dimensional flight simulator. The new batch algorithm is compared with the most recent batch model tree algorithms and is seen to perform favourably overall. The new incremental model tree learner compares well with an alternative online function approximator. In addition it can sometimes perform almost as well as the batch model tree algorithms, highlighting the effectiveness of the incremental implementation. Editor: Johannes Fürnkranz     

Learning Changing Concepts by Exploiting the Structure of Change

This paper examines learning problems in which the target function is allowed to change. The learner sees a sequence of random examples, labelled according to a sequence of functions, and must provide an accurate estimate of the target function sequence. We consider a variety of restrictions on how the target function is allowed to change, including infrequent but arbitrary changes, sequences that correspond to slow walks on a graph whose nodes are functions, and changes that are small on average, as measured by the probability of disagreements between consecutive functions. We first study estimation, in which the learner sees a batch of examples and is then required to give an accurate estimate of the function sequence. Our results provide bounds on the sample complexity and allowable drift rate for these problems. We also study prediction, in which the learner must produce online a hypothesis after each labelled example and the average misclassification probability over this hypothesis sequence should be small. Using a deterministic analysis in a general metric space setting, we provide a technique for constructing a successful prediction algorithm, given a successful estimation algorithm. This leads to sample complexity and drift rate bounds for the prediction of changing concepts.     

Online Ensemble Learning: An Empirical Study

We study resource-limited online learning, motivated by the problem of conditional-branch outcome prediction in computer architecture. In particular, we consider (parallel) time and space-efficient ensemble learners for online settings, empirically demonstrating benefits similar to those shown previously for offline ensembles. Our learning algorithms are inspired by the previously published boosting by filtering framework as well as the offline Arc-x4 boosting-style algorithm. We train ensembles of online decision trees using a novel variant of the ID4 online decision-tree algorithm as the base learner, and show empirical results for both boosting and bagging-style online ensemble methods. Our results evaluate these methods on both our branch prediction domain and online variants of three familiar machine-learning benchmarks. Our data justifies three key claims. First, we show empirically that our extensions to ID4 significantly improve performance for single trees and additionally are critical to achieving performance gains in tree ensembles. Second, our results indicate significant improvements in predictive accuracy with ensemble size for the boosting-style algorithm. The bagging algorithms we tried showed poor performance relative to the boosting-style algorithm (but still improve upon individual base learners). Third, we show that ensembles of small trees are often able to outperform large single trees with the same number of nodes (and similarly outperform smaller ensembles of larger trees that use the same total number of nodes). This makes online boosting particularly useful in domains such as branch prediction with tight space restrictions (i.e., the available real-estate on a microprocessor chip).     

A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance     

分布式在线交替方向乘子法

针对如何对分布式网络采集的数据进行在线学习的问题,提出了一种基于交替方向乘子法(ADMM)的分布式在线学习优化算法--分布式在线交替方向乘子法(DOM)。首先,针对分布式在线学习需要各节点根据新采集的数据来更新本地估计,同时保持网络中所有节点的估计趋于一致这一问题,建立了数学模型并设计DOM算法对其进行求解。其次,针对分布式在线学习问题定义了Regret 界,用以表征在线估计的性能;证明了当本地即时损失函数是凸函数时,DOM算法是收敛的,并给出了其收敛速度。最后,通过数值仿真实验结果表明,相比现有的分布式在线梯度下降法(DOGD)和分布式在线自主学习算法(DAOL),所提出的DOM算法具有更快的收敛性能。     

Toward online node classification on streaming networks

The proliferation of networked data in various disciplines motivates a surge of research interests on network or graph mining. Among them, node classification is a typical learning task that focuses on exploiting the node interactions to infer the missing labels of unlabeled nodes in the network. A vast majority of existing node classification algorithms overwhelmingly focus on static networks and they assume the whole network structure is readily available before performing learning algorithms. However, it is not the case in many real-world scenarios where new nodes and new links are continuously being added in the network. Considering the streaming nature of networks, we study how to perform online node classification on this kind of streaming networks (a.k.a. online learning on streaming networks). As the existence of noisy links may negatively affect the node classification performance, we first present an online network embedding algorithm to alleviate this problem by obtaining the embedding representation of new nodes on the fly. Then we feed the learned embedding representation into a novel online soft margin kernel learning algorithm to predict the node labels in a sequential manner. Theoretical analysis is presented to show the superiority of the proposed framework of online learning on streaming networks (OLSN). Extensive experiments on real-world networks further demonstrate the effectiveness and efficiency of the proposed OLSN framework.     

一种自适应在线核密度估计方法

给定一组观察数据,估计其潜在的概率密度函数是统计学中的一项基本任务,被称为密度估计问题.随着数据收集技术的发展,出现了大量的实时流式数据,其特点是数据量大,数据产生速度快,并且数据的潜在分布也可能随着时间而发生变化,对这类数据分布的估计也成为亟待解决的问题.然而,在传统的密度估计算法中,参数式算法因为有较强的模型假设导致其表达能力有限,非参数式算法虽然具有更好的表达能力,但其计算复杂度通常很高.因此,它们都无法很好地应用于这种流式数据的场景.通过分析基于竞争学习的学习过程,提出了一种在线密度估计算法来完成流式数据上的密度估计任务,并且分析了其与高斯混合模型之间的密切联系.最后,将所提算法与现有的密度估计算法进行对比实验.实验结果表明,与现有的在线密度估计算法相比,所提算法能够取得更好的估计结果,并且能够基本上达到当前最好的离线密度估计算法的估计性能.