Online adaptive decision trees

Decision trees and neural networks are widely used tools for pattern classification. Decision trees provide highly localized representation, whereas neural networks provide a distributed but compact representation of the decision space. Decision trees cannot be induced in the online mode, and they are not adaptive to changing environment, whereas neural networks are inherently capable of online learning and adpativity. Here we provide a classification scheme called online adaptive decision trees (OADT), which is a tree-structured network like the decision trees and capable of online learning like neural networks. A new objective measure is derived for supervised learning with OADT. Experimental results validate the effectiveness of the proposed classification scheme. Also, with certain real-life data sets, we find that OADT performs better than two widely used models: the hierarchical mixture of experts and multilayer perceptron.     

基于邻域粗糙集的多标记分类特征选择算法

多标记学习是一类复杂的决策任务,同一个对象可能同时属于多个类别.此类任务在文本分类、图像识别、基因功能分析等领域广泛存在.多标记分类任务往往由高维特征描述,存在大量无关和冗余的信息.目前已经提出了大量的单标记特征选择算法以应对维数灾难问题,但对于多标记的属性约简和特征选择却鲜有研究.将粗糙集应用于多标记数据的特征选择中,针对多标记分类任务,重新定义了邻域粗糙集的下近似和依赖度计算方法,探讨了这一模型的性质,进而构造了基于邻域粗糙集的多标记分类任务的特征选择算法,并给出了在公开数据上的实验结果.实验分析证明算法的有效性.     

Classification-oriented structure learning in Bayesian networks for multimodal event detection in videos

We investigate the use of structure learning in Bayesian networks for a complex multimodal task of action detection in soccer videos. We illustrate that classical score-oriented structure learning algorithms, such as the K2 one whose usefulness has been demonstrated on simple tasks, fail in providing a good network structure for classification tasks where many correlated observed variables are necessary to make a decision. We then compare several structure learning objective functions, which aim at finding out the structure that yields the best classification results, extending existing solutions in the literature. Experimental results on a comprehensive data set of 7 videos show that a discriminative objective function based on conditional likelihood yields the best results, while augmented approaches offer a good compromise between learning speed and classification accuracy.     

基于模糊优势互补互信息的有序决策树算法

传统决策树算法应用于有序分类任务时存在两个问题：传统决策树算法没有引入序关系,因此无法学习和抽取数据集中的序结构;现实生活中存在大量模糊而非精确的知识,而传统的决策树算法无法处理存在模糊属性取值的数据。针对上述问题,提出了基于模糊优势互补互信息的有序决策树算法。首先,使用优势集表示数据中的序关系,并引入模糊集来计算优势集以形成模糊优势集。模糊优势集不仅能反映数据中的序信息,而且能自动获取不精确知识。然后,在模糊优势集的基础上将互补互信息进行推广,并提出了模糊优势互补互信息。最后,使用模糊优势互补互信息作为启发式,设计出基于模糊优势互补互信息的有序决策树算法。在5个人工数据集及9个现实数据集上的实验结果表明,所提算法在有序分类任务上较经典决策树算法取得了更低的分类误差。     

潜在属性空间树分类器

提出一种潜在属性空间树分类器(latent attribute space tree classifier,简称LAST)框架,通过将原属性空间变换到更容易分离数据或更符合决策树分类特点的潜在属性空间,突破传统决策树算法的决策面局限,改善树分类器的泛化性能.在LAST 框架下,提出了两种奇异值分解斜决策树(SVD (singular value decomposition) oblique decision tree,简称SODT)算法,通过对全局或局部数据进行奇异值分解,构建正交的潜在属性空间,然后在潜在属性空间内构建传统的单变量决策树或树节点,从而间接获得原空间内近似最优的斜决策树.SODT 算法既能够处理整体数据与局部数据分布相同或不同的数据集,又可以充分利用有标签和无标签数据的结构信息,分类结果不受样本随机重排的影响,而且时间复杂度还与单变量决策树算法相同.在复杂数据集上的实验结果表明,与传统的单变量决策树算法和其他斜决策树算法相比,SODT 算法的分类准确率更高,构建的决策树大小更稳定,整体分类性能更鲁棒,决策树构建时间与C4.5 算法相近,而远小于其他斜决策树算法.     

APACS: a system for the automatic analysis and classification of conceptual patterns

Many existing inductive learning systems have been developed under the assumption that the learning tasks are performed in a noise-free environment. To cope with most real-world problems, it is important that a learning system be equipped with the capability to handle uncertainty. In this paper, we first identify the various sources of uncertainty that may be encountered in a noisy problem domain. Next, we present a method for the efficient acquisition of classification rules from training instances which may contain inconsistent, incorrect, or missing information. This algorithm consists of three phases: ( i ) the detection of inherent patterns in a set of noisy training data; ( ii ) the construction of classification rules based on these patterns; and ( iii ) the use of these rules to predict the class membership of an object. The method has been implemented in a system known as APACS (automatic pattern analysis and classification system). This system has been tested using both real-life and simulated data, and its performance is found to be superior to many existing systems in terms of efficiency and classification accuracy. Being able to handle uncertainty in the learning process, the proposed algorithm can be employed for applications in real-world problem domains involving noisy data.     

Integrating Spectral Kernel Learning and Constraints in Semi-Supervised Classification

Recently, integrating new knowledge sources such as pairwise constraints into various classification tasks with insufficient training data has been actively studied in machine learning. In this paper, we propose a novel semi-supervised classification approach, called semi-supervised classification with enhanced spectral kernel, which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first design a non-parameter spectral kernel learning model based on the squared loss function. Then we develop an efficient semi-supervised classification algorithm which takes advantage of Laplacian spectral regularization: semi-supervised classification with enhanced spectral kernel under the squared loss (ESKS). Finally, we conduct many experiments on a variety of synthetic and real-world data sets to demonstrate the effectiveness of the proposed ESKS algorithm.     

基于信息熵和几何轮廓相似度的多变量决策树

现有的多变量决策树在分类准确性与树结构复杂性两方面优于单变量决策树,但其训练时间却高于单变量决策树,使得现有的多变量决策树不适用于快速响应的分类任务.针对现有多变量决策树训练时间高的问题,提出了基于信息熵和几何轮廓相似度的多变量决策树(IEMDT).该算法利用几何轮廓相似度函数的一对一映射特性,将n维空间样本点投影到一维空间的数轴上,进而形成有序的投影点集合,然后通过类别边界和信息增益计算最优分割点集将有序投影点集合划分为多个子集,接着分别对每个子集继续投影分割,最终生成决策树.在8个数据集上的实验结果表明：IEMDT具有较低的训练时间,并且具有较高的分类准确性.     

A decision support system for substage-zoning filling design of rock-fill dams based on particle swarm optimization

In this paper, we studied a substage-zoning filling design problem, which is considered as a complex problem with numerous tasks such as construction planning, dam access road and borrow placement, workspace filling, and construction project management. In analyzing workflows and the mechanism of substage-zoning filling, not only the above-mentioned tasks are considered, but also the environmental factors such as rainfall and hydrology characteristic temperature are taken into account. In this study, an optimization model for dam filling which aimed at reducing the disequilibrium degree of filling intensity was proposed; in addition, a technique based on particle swarm optimization was introduced as the basis of a decision support system for rock-fill dams. The system has been employed in a water conservancy and hydropower project which shows that the system is able to provide quality decision support and facilitate the rock-fill dam construction effectively.     

Entropy-based shadowed set approximation of intuitionistic fuzzy sets

We propose a method to approximate Intuitionistic Fuzzy Sets (IFSs) with Shadowed Sets that could be used, in decision making or similar tasks, when the full information about membership values is not necessary, is difficult to process or to interpret. Our approach is based on an information-theoretic perspective and aims at preserving the uncertainty, represented through an entropy measure, in the original IFS by minimizing the difference between the entropy in the input IFS and the output Shadowed Set. We propose three different efficient optimization algorithms that retain Fuzziness, Lack of Knowledge, or both, and illustrate their computation through an illustrative example. We also evaluate the application of the proposed approximation methods in the Machine Learning setting by showing that the approximation, through the proposed methods, of IFS k-Nearest Neighbors is able to outperform, in terms of running time, the standard algorithm.     

基于Spark的高效并行自动编码机

机器学习中一个非常关键的问题就是如何获取良好的数据特征表示,许多经典的特征提取方法是基于数据间关系或利用简单线性组合降维后得到数据的特征表示。其中深度学习算法在各种学习任务中都可以取得良好的效果,而且可以学到很好的数据特征表示。但现有深度学习算法或模型大多为单机串行实现,不能处理较大规模的数据且运行时间较长。本文设计实现了一种基于Spark分布式平台的高效并行自动编码机,该编码机可以有效地进行特征表示学习,并且利用分布式计算平台Spark对 算法进行加速,优化了对稀疏数据的操作,大大提升了运行效率。本文通过在文本数据特征学习以及协同过滤两个任务上的实验,表明本文所实现的并行自动编码机的有效性和高效性。     

Robust Bayes classifiers

Naive Bayes classifiers provide an efficient and scalable approach to supervised classification problems. When some entries in the training set are missing, methods exist to learn these classifiers under some assumptions about the pattern of missing data. Unfortunately, reliable information about the pattern of missing data may be not readily available and recent experimental results show that the enforcement of an incorrect assumption about the pattern of missing data produces a dramatic decrease in accuracy of the classifier. This paper introduces a Robust Bayes Classifier (rbc) able to handle incomplete databases with no assumption about the pattern of missing data. In order to avoid assumptions, the rbc bounds all the possible probability estimates within intervals using a specialized estimation method. These intervals are then used to classify new cases by computing intervals on the posterior probability distributions over the classes given a new case and by ranking the intervals according to some criteria. We provide two scoring methods to rank intervals and a decision theoretic approach to trade off the risk of an erroneous classification and the choice of not classifying unequivocally a case. This decision theoretic approach can also be used to assess the opportunity of adopting assumptions about the pattern of missing data. The proposed approach is evaluated on twenty publicly available databases.     

Classification and recognition of dynamical models: the role of phase, independent components, kernels and optimal transport

We address the problem of performing decision tasks, and in particular classification and recognition, in the space of dynamical models in order to compare time series of data. Motivated by the application of recognition of human motion in image sequences, we consider a class of models that include linear dynamics, both stable and marginally stable (periodic), both minimum and non-minimum phase, driven by non-Gaussian processes. This requires extending existing learning and system identification algorithms to handle periodic modes and nonminimum phase behavior, while taking into account higher-order statistics of the data. Once a model is identified, we define a kernel-based cord distance between models that includes their dynamics, their initial conditions as well as input distribution. This is made possible by a novel kernel defined between two arbitrary (non-Gaussian) distributions, which is computed by efficiently solving an optimal transport problem. We validate our choice of models, inference algorithm, and distance on the tasks of human motion synthesis (sample paths of the learned models), and recognition (nearest-neighbor classification in the computed distance). However, our work can be applied more broadly where one needs to compare historical data while taking into account periodic trends, non-minimum phase behavior, and non-Gaussian input distributions.     

一种新的李群分类器在手写体数字中的应用

李群是变换空间的一种基本表示理论。目前针对李群数据所设计的分类器较少,对多分类的效果也不是很好。以手写体数字的应用为背景,引入了支持向量机分类算法来处理李群数据。由于李群数据具有矩阵表现的形式,设计了一种矩阵高斯核函数,使得支持向量机能够处理矩阵数据。仿真结果表明,支持向量机方法在李群数据上具有很好的性能。     

基于关联规则的决策树算法

通过将关联规则与决策树算法相结合,形成一种基于关联规则的决策树算法。该算法对不同时期同一事务的异种数据结构进行处理,得到一种可扩展的多分支分类决策树,使得改进后的决策树算法具有良好的可扩展性。该算法解决了传统分类算法在数据集维度发生变化时分类过程无法持续进行的问题。     

On the versatility of radial basis function neural networks: A case study in the field of intrusion detection

Classifiers based on radial basis function neural networks have a number of useful properties that can be exploited in many practical applications. Using sample data, it is possible to adjust their parameters (weights), to optimize their structure, and to select appropriate input features (attributes). Moreover, interpretable rules can be extracted from a trained classifier and input samples can be identified that cannot be classified with a sufficient degree of “certainty”. These properties support an analysis of radial basis function classifiers and allow for an adaption to “novel” kinds of input samples in a real-world application. In this article, we outline these properties and show how they can be exploited in the field of intrusion detection (detection of network-based misuse). Intrusion detection plays an increasingly important role in securing computer networks. In this case study, we first compare the classification abilities of radial basis function classifiers, multilayer perceptrons, the neuro-fuzzy system NEFCLASS, decision trees, classifying fuzzy-k-means, support vector machines, Bayesian networks, and nearest neighbor classifiers. Then, we investigate the interpretability and understandability of the best paradigms found in the previous step. We show how structure optimization and feature selection for radial basis function classifiers can be done by means of evolutionary algorithms and compare this approach to decision trees optimized using certain pruning techniques. Finally, we demonstrate that radial basis function classifiers are basically able to detect novel attack types. The many advantageous properties of radial basis function classifiers could certainly be exploited in other application fields in a similar way.     

Performance Capture of High‐Speed Motion Using Staggered Multi‐View Recording

We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware. To this end we introduce two novel concepts: First, a staggered surround multi‐view recording setup that enables us to perform model‐based motion capture on motion‐blurred images, and second, a model‐based deblurring algorithm which is able to handle disocclusion, self‐occlusion and complex object motions. We show that the model‐based approach is not only a powerful strategy for tracking but also for deblurring highly complex blur patterns.     

Classifier architectures for single chamber arrhythmia recognition

In this paper we study heart arrhythmia classification for single chamber implantable cardio-verter defibrillators. Our research shows that performance of conventional classification methods using only simple heart rate timing based features can be improved with the inclusion of morphology analysis on samples of the right ventricular apex lead. While morphology classification is typically patient dependent, and computationally expensive, we show that the performance of a patient independent classifier which uses a multi-layer perceptron for morphology recognition and heart beat timing decision tree is superior to that of a timing only classifier, while remaining economical (silicon area and power dissipation) from an implementation perspective. We also show that performance can be significantly improved in the patient dependent case.     

Unsupervised feature selection with graph learning via low-rank constraint

Feature selection is one of the most important machine learning procedure, and it has been successfully applied to make a preprocessing before using classification and clustering methods. High-dimensional features often appear in big data, and it’s characters block data processing. So spectral feature selection algorithms have been increasing attention by researchers. However, most feature selection methods, they consider these tasks as two steps, learn similarity matrix from original feature space (may be include redundancy for all features), and then conduct data clustering. Due to these limitations, they do not get good performance on classification and clustering tasks in big data processing applications. To address this problem, we propose an Unsupervised Feature Selection method with graph learning framework, which can reduce the redundancy features influence and utilize a low-rank constraint on the weight matrix simultaneously. More importantly, we design a new objective function to handle this problem. We evaluate our approach by six benchmark datasets. And all empirical classification results show that our new approach outperforms state-of-the-art feature selection approaches.