基于机器学习的文本分类技术研究进展

文本自动分类是信息检索与数据挖掘领域的研究热点与核心技术,近年来得到了广泛的关注和快速的发展.提出了基于机器学习的文本分类技术所面临的互联网内容信息处理等复杂应用的挑战,从模型、算法和评测等方面对其研究进展进行综述评论.认为非线性、数据集偏斜、标注瓶颈、多层分类、算法的扩展性及Web页分类等问题是目前文本分类研究的关键问题,并讨论了这些问题可能采取的方法.最后对研究的方向进行了展望.  

多核学习方法

多核学习方法是当前核机器学习领域的一个新的热点. 核方法是解决非线性模式分析问题的一种有效方法, 但在一些复杂情形下, 由单个核函数构成的核机器并不能满足诸如数据异构或不规则、样本规模巨大、样本不平坦分布等实际的应用需求, 因此将多个核函数进行组合, 以获得更好的结果是一种必然选择. 本文根据多核的构成, 从合成核、多尺度核、无限核三个角度, 系统综述了多核方法的构造理论, 分析了多核学习典型方法的特点及不足, 总结了各自的应用领域, 并凝炼了其进一步的研究方向.  

回归支持向量机的改进序列最小优化学习算法

支持向量机(support vector machine,简称SVM)是一种基于结构风险最小化原理的学习技术,也是一种新的具有很好泛化性能的回归方法,提出了实现回归支持向量机的一种改进的SMO(sequential minimal optimization)算法,给出了两变量子优化问题的解析解,设计了新的工作集选择方法和停止条件,仿真实例说明,所提出的SMO算法比原始SMO算法具有更快的运算速度.  

Kernel Matching Pursuit

Matching Pursuit algorithms learn a function that is a weighted sum of basis functions, by sequentially appending functions to an initially empty basis, to approximate a target function in the least-squares sense. We show how matching pursuit can be extended to use non-squared error loss functions, and how it can be used to build kernel-based solutions to machine learning problems, while keeping control of the sparsity of the solution. We present a version of the algorithm that makes an optimal choice of both the next basis and the weights of all the previously chosen bases. Finally, links to boosting algorithms and RBF training procedures, as well as an extensive experimental comparison with SVMs for classification are given, showing comparable results with typically much sparser models.  

基于KPLS的网络入侵特征抽取及检测方法

从特征抽取的角度研究提高入侵检测性能问题,提出应用核偏最小二乘(KPLS)进行入侵特征抽取和检测的方法.其优点在于KPLS能非线性地抽取输入特征的多个正交分量,并保持与输出类别的相关性,可同时完成入侵特征抽取和判别.将该方法应用于基于Linux主机的入侵检测实验,取得了比SVM和KPCR等方法更好的效果.  

Kernels and Distances for Structured Data

This paper brings together two strands of machine learning of increasing importance: kernel methods and highly structured data. We propose a general method for constructing a kernel following the syntactic structure of the data, as defined by its type signature in a higher-order logic. Our main theoretical result is the positive definiteness of any kernel thus defined. We report encouraging experimental results on a range of real-world data sets. By converting our kernel to a distance pseudo-metric for 1-nearest neighbour, we were able to improve the best accuracy from the literature on the Diterpene data set by more than 10%.  

基于支持对象的野点检测方法

从模式识别的角度出发，分析了基于核的野点检测方法，指出随着样本数目的增加，该方法会因为过大的优化规模而无法实际操作，为此提出了基于支持对象的野点检测方法，该方法大大降低了运算规模和内存需求，保证了野点检测实时性的要求。  

核函数的性质及其构造方法

支持向量机是一项机器学习技术，发展至今近10年了，已经成功地用于模式识别、回归估计以及聚类等，并由此衍生出了核方法。支持向量机由核函数与训练集完全刻画。进一步提高支持向量机性能的关键，是针对给定的问题设计恰当的核函数，这就要求对核函数本身有深刻了解。本文首先分析了核函数的一些重要性质，接着对3类核函数，即平移不变核函数、旋转不变核函数和卷积核，提出了简单实用的判别准则。在此基础上，验证和构造了很多重要核函数。  

Latent Semantic Kernels

Kernel methods like support vector machines have successfully been used for text categorization. A standard choice of kernel function has been the inner product between the vector-space representation of two documents, in analogy with classical information retrieval (IR) approaches.Latent semantic indexing (LSI) has been successfully used for IR purposes as a technique for capturing semantic relations between terms and inserting them into the similarity measure between two documents. One of its main drawbacks, in IR, is its computational cost.In this paper we describe how the LSI approach can be implemented in a kernel-defined feature space.We provide experimental results demonstrating that the approach can significantly improve performance, and that it does not impair it.