An adaptive ensemble classifier for mining concept drifting data streams

It is challenging to use traditional data mining techniques to deal with real-time data stream classifications. Existing mining classifiers need to be updated frequently to adapt to the changes in data streams. To address this issue, in this paper we propose an adaptive ensemble approach for classification and novel class detection in concept drifting data streams. The proposed approach uses traditional mining classifiers and updates the ensemble model automatically so that it represents the most recent concepts in data streams. For novel class detection we consider the idea that data points belonging to the same class should be closer to each other and should be far apart from the data points belonging to other classes. If a data point is well separated from the existing data clusters, it is identified as a novel class instance. We tested the performance of this proposed stream classification model against that of existing mining algorithms using real benchmark datasets from UCI (University of California, Irvine) machine learning repository. The experimental results prove that our approach shows great flexibility and robustness in novel class detection in concept drifting and outperforms traditional classification models in challenging real-life data stream applications.     

Kernel matching pursuit classifier ensemble

Kernel Matching Pursuit Classifier (KMPC), a novel classification machine in pattern recognition, has an excellent advantage in solving classification problems for the sparsity of the solution. Unfortunately, the performance of the KMPC is far from the theoretically expected level of it. Ensemble Methods are learning algorithms that construct a collection of individual classifiers which are independent and yet accurate, and then classify a new data point by taking vote of their predictions. In such a way, the performance of classifiers can be improved greatly. In this paper, on a thorough investigation into the principle of KMPC and Ensemble Method, we expatiate on the theory of KMPC ensemble and pointed out the ways to construct it. The experiments performed on the artificial data and UCI data show KMPC ensemble combines the advantages of KMPC with ensemble method, and improves classification performance remarkably.     

Efficient classification of multi-labeled text streams by clashing

We present a method for the classification of multi-labeled text documents explicitly designed for data stream applications that require to process a virtually infinite sequence of data using constant memory and constant processing time.Our method is composed of an online procedure used to efficiently map text into a low-dimensional feature space and a partition of this space into a set of regions for which the system extracts and keeps statistics used to predict multi-label text annotations. Documents are fed into the system as a sequence of words, mapped to a region of the partition, and annotated using the statistics computed from the labeled instances colliding in the same region. This approach is referred to as clashing.We illustrate the method in real-world text data, comparing the results with those obtained using other text classifiers. In addition, we provide an analysis about the effect of the representation space dimensionality on the predictive performance of the system. Our results show that the online embedding indeed approximates the geometry of the full corpus-wise TF and TF-IDF space. The model obtains competitive F measures with respect to the most accurate methods, using significantly fewer computational resources. In addition, the method achieves a higher macro-averaged F measure than methods with similar running time. Furthermore, the system is able to learn faster than the other methods from partially labeled streams.     

贝叶斯数据流分类算法研究

为了有效解决传统的数据分类算法不能很好的适应数据流的数据无限性和概念漂移性带来的问题,提出了一种实时的数据流的挖掘算法.贝叶斯数据流分类算法充分考虑了离散属性和连续属性的不同处理,对时间窗口内的数据进行压缩,然后根据各个时间窗口的权重,重组了压缩后的数据并在重组后的压缩数据上学习和生成了单个贝叶斯分类器.实验结果表明,该算法在分类性能、分类准确率、分类速度上优于同类算法.     

基于主动-被动增量集成的概念漂移适应方法

数据流是一组随时间连续到来的数据序列,在数据流不断产生的过程中,由于各种因素的影响,数据分布随时间推移可能以不可预测的方式发生变化,这种现象称为概念漂移.在漂移发生后,当前模型需要及时响应数据流中的实时分布变化,并有效处理不同类型的概念漂移,从而避免模型泛化性能下降.针对这一问题,提出一种基于主动–被动增量集成的概念漂移适应方法 (CDAM-APIE).该方法首先使用在线增量集成策略构建被动集成模型,对新样本进行实时预测以动态更新基模型权重,有利于快速响应数据分布的瞬时变化,并增强模型适应概念漂移的能力.在此基础上,利用增量学习和概念漂移检测技术构建主动基模型,提升模型在平稳数据流状态下的鲁棒性和漂移后的泛化性能.实验结果表明, CDAM-APIE能够对概念漂移做出及时响应,同时有效提高模型的泛化性能.     

TCBR-HMM: An HMM-based text classifier with a CBR system

This paper presents an innovative solution to model distributed adaptive systems in biomedical environments. We present an original TCBR-HMM (Text Case Based Reasoning-Hidden Markov Model) for biomedical text classification based on document content. The main goal is to propose a more effective classifier than current methods in this environment where the model needs to be adapted to new documents in an iterative learning frame. To demonstrate its achievement, we include a set of experiments, which have been performed on OSHUMED corpus. Our classifier is compared with Naive Bayes and SVM techniques, commonly used in text classification tasks. The results suggest that the TCBR-HMM Model is indeed more suitable for document classification. The model is empirically and statistically comparable to the SVM classifier and outperforms it in terms of time efficiency.     

基于集成分类器的肿瘤亚型识别研究

针对肿瘤基因表达谱样本少,维数高的特点,提出一种用于肿瘤信息基因提取和亚型识别的集成分类器算法.该算法根据基因的Fisher比率值建立候选子集,再采用相关系数和互信息两种度量方法,分别构造反映基因共表达行为和调控关系的特征子集.粒子群优化算法分别与SVM和KNN构成两个基分类器,从候选子集中提取信息基因并对肿瘤亚型进行分类,最后利用绝对多数投票方法对基分类器的结果进行整合.G.Gordon肺癌亚型识别的实验结果表明了该算法的可行性和有效性.     

Effective classification of noisy data streams with attribute-oriented dynamic classifier selection

Recently, mining from data streams has become an important and challenging task for many real-world applications such as credit card fraud protection and sensor networking. One popular solution is to separate stream data into chunks, learn a base classifier from each chunk, and then integrate all base classifiers for effective classification. In this paper, we propose a new dynamic classifier selection (DCS) mechanism to integrate base classifiers for effective mining from data streams. The proposed algorithm dynamically selects a single “best” classifier to classify each test instance at run time. Our scheme uses statistical information from attribute values, and uses each attribute to partition the evaluation set into disjoint subsets, followed by a procedure that evaluates the classification accuracy of each base classifier on these subsets. Given a test instance, its attribute values determine the subsets that the similar instances in the evaluation set have constructed, and the classifier with the highest classification accuracy on those subsets is selected to classify the test instance. Experimental results and comparative studies demonstrate the efficiency and efficacy of our method. Such a DCS scheme appears to be promising in mining data streams with dramatic concept drifting or with a significant amount of noise, where the base classifiers are likely conflictive or have low confidence. A preliminary version of this paper was published in the Proceedings of the 4th IEEE International Conference on Data Mining, pp 305–312, Brighton, UK Xingquan Zhu received his Ph.D. degree in Computer Science from Fudan University, Shanghai, China, in 2001. He spent four months with Microsoft Research Asia, Beijing, China, where he was working on content-based image retrieval with relevance feedback. From 2001 to 2002, he was a Postdoctoral Associate in the Department of Computer Science, Purdue University, West Lafayette, IN. He is currently a Research Assistant Professor in the Department of Computer Science, University of Vermont, Burlington, VT. His research interests include Data mining, machine learning, data quality, multimedia computing, and information retrieval. Since 2000, Dr. Zhu has published extensively, including over 40 refereed papers in various journals and conference proceedings. Xindong Wu is a Professor and the Chair of the Department of Computer Science at the University of Vermont. He holds a Ph.D. in Artificial Intelligence from the University of Edinburgh, Britain. His research interests include data mining, knowledge-based systems, and Web information exploration. He has published extensively in these areas in various journals and conferences, including IEEE TKDE, TPAMI, ACM TOIS, IJCAI, ICML, KDD, ICDM, and WWW, as well as 11 books and conference proceedings. Dr. Wu is the Editor-in-Chief of the IEEE Transactions on Knowledge and Data Engineering (by the IEEE Computer Society), the founder and current Steering Committee Chair of the IEEE International Conference on Data Mining (ICDM), an Honorary Editor-in-Chief of Knowledge and Information Systems (by Springer), and a Series Editor of the Springer Book Series on Advanced Information and Knowledge Processing (AI&KP). He is the 2004 ACM SIGKDD Service Award winner. Ying Yang received her Ph.D. in Computer Science from Monash University, Australia in 2003. Following academic appointments at the University of Vermont, USA, she currently holds a Research Fellow at Monash University, Australia. Dr. Yang is recognized for contributions in the fields of machine learning and data mining. She has published many scientific papers and book chapters on adaptive learning, proactive mining, noise cleansing and discretization. Contact her at yyang@mail.csse.monash.edu.au.     

一种带潜在类别主题词的简单贝叶斯文本分类器

简单贝叶斯器是一种有效的文本分类方法。文中提出一种改进的简单贝叶斯文本分类器。即利用有限次的迭代来提升分类的精度，在迭代过程中，为每个分类实例引入了一个权值系数和一个呆滞系数，经过一次迭代，两个系数就会相应的改变，体现了提升的思想。最终的分类结果则是整个迭代结果的综合。     

数据流集成分类算法综述

详细介绍了国内外集成分类算法,对集成分类算法的两个部分(基分类器组合和动态更新集成模型)进行了详细综述,明确区分不同集成算法的优缺点,对比算法和实验数据集。并且提出进一步的研究方向和考虑的解决办法。     

Non-stationary data sequence classification using online class priors estimation

Online classification is important for real time data sequence classification. Its most challenging problem is that the class priors may vary for non-stationary data sequences. Most of the current online-data-sequence-classification algorithms assume that the class labels of some new-arrived data samples are known and retrain the classifier accordingly. Unfortunately, such assumption is often violated in real applications. But if we were able to estimate the class priors on the test data sequence accurately, we could adjust the classifier without retraining it while preserving a reasonable accuracy. There has been some work on the class priors estimation to classify static data sets using the offline iterative EM algorithm, which has been proved to be quite effective to adjust the classifier. Inspired by the offline iterative EM algorithm for static data sets, in this paper, we propose an online incremental EM algorithm to estimate the class priors along the data sequence. The classifier is adjusted accordingly to keep pace with the varying distribution. The proposed online algorithm is more computationally efficient because it scans the sequence only once. Experimental results show that the proposed algorithm indeed performs better than the conventional offline iterative EM algorithm when the class priors are non-stationary.     

动态文本分类中概念漂移问题的解决算法研究

以当前的\"消极学习型分类法\"加\"动态更新训练集\"的组合模式,不足以解决好动态文本分类中的概念漂移问题.为此,受消极分类法基本思想的启发,并借鉴k-NN算法的优点,提出了针对概念漂移问题的\"消极特征选择模式\"的概念和基于此模式的动态文本分类算法.测试结果表明,新算法很好地解决了当前存在的难点问题,具有高可靠性、高实用性等优点.     

一种介词-动词模式的获取方法

基于模式的知识获取方法研究是当前文本知识获取的重点研究之一,如何获得文本知识模式是该研究中的一个重要研究内容。提出一种新的基于介词和动词模式（称为PV模式）的获取方法。首先构造出一个候选的动词介词组合（称为PV组合）,使用统计方法对其进行过滤。度量PV组合好坏有两个标准：一个是模式词的表示能力,另一个是模式词与概念词之间及多个概念词之间的相关性。依据这两个标准构造了6个数值特征,通过训练产生了3个分类器,采用交叉验证的方式估计出3个分类器的精度分别达到0．853,0．862和0．856。这些分类器为从PV组合中自动挑选PV模式提供依据。     

Graphical causal inference and copula regression model for apple keywords by text mining

Apple is a leading company of technological evolution and innovation. This company founded and produced the Apple I computer in 1976. Since then, based on its innovative technologies, Apple has launched creative and innovative products and services such as the iPod, iTunes, the iPhone, the Apple app store, and the iPad. In many fields of academia and business, diverse studies of Apple’s technological innovation strategy have been performed. In this paper, we analyze Apple’s patents to better understand its technological innovation. We collected all applied patents by Apple until now, and applied statistics and text mining for patent analysis. By using graphical causal inference method, we created the causal relations among Apple keywords preprocessed by text mining, and then we carried out the semiparametric Gaussian copula regression model to see how the target response keyword and the predictor keywords are relating to each other. Furthermore, Gaussian copula partial correlation was applied to Apple keywords to find out the detailed dependence structure. By performing these methods, this paper shows the technological trends and relations between Apple’s technologies. This research could make contributions in finding vacant technology areas and central technologies for Apple’s R&D planning.     

Classifying Uncertain and Evolving Data Streams with Distributed Extreme Learning Machine

Conventional classification algorithms are not well suited for the inherent uncertainty, potential concept drift, volume, and velocity of streaming data. Specialized algorithms are needed to obtain e?c...     

面向分布式漂移数据流的集成分类模型

针对大数据环境下分类精度不高的问题,提出了一种面向分布式数据流的集成分类模型.首先,使用微簇模式减少局部节点向中心节点传输的数据量,降低通信代价;然后,使用样本重构算法生成全局分类器的训练样本;最后,提出一种面向漂移数据流的集成分类模型,采用动态分类器和稳定分类器的加权组合策略,使用混合标记策略标记最具代表性的样本以更...     

概念漂移数据流分类中的多源在线迁移学习算法

现有概念漂移处理算法在检测到概念漂移发生后,通常需要在新到概念上重新训练分类器,同时“遗忘”以往训练的分类器。在概念漂移发生初期,由于能够获取到的属于新到概念的样本较少,导致新建的分类器在短时间内无法得到充分训练,分类性能通常较差。进一步,现有的基于在线迁移学习的数据流分类算法仅能使用单个分类器的知识辅助新到概念进行学习,在历史概念与新到概念相似性较差时,分类模型的分类准确率不理想。针对以上问题,文中提出一种能够利用多个历史分类器知识的数据流分类算法——CMOL。CMOL算法采取分类器权重动态调节机制,根据分类器的权重对分类器池进行更新,使得分类器池能够尽可能地包含更多的概念。实验表明,相较于其他相关算法,CMOL算法能够在概念漂移发生时更快地适应新到概念,显示出更高的分类准确率。     

Multilabel classification using heterogeneous ensemble of multi-label classifiers

Multilabel classification is a challenging research problem in which each instance may belong to more than one class. Recently, a considerable amount of research has been concerned with the development of “good” multi-label learning methods. Despite the extensive research effort, many scientific challenges posed by e.g. highly imbalanced training sets and correlation among labels remain to be addressed. The aim of this paper is to use a heterogeneous ensemble of multi-label learners to simultaneously tackle both the sample imbalance and label correlation problems. This is different from the existing work in the sense that we are proposing to combine state-of-the-art multi-label methods by ensemble techniques instead of focusing on ensemble techniques within a multi-label learner. The proposed ensemble approach (EML) is applied to six publicly available multi-label data sets from various domains including computer vision, biology and text using several evaluation criteria. We validate the advocated approach experimentally and demonstrate that it yields significant performance gains when compared with state-of-the art multi-label methods.