基于增量式学习的数据流实时分类模型

传统数据挖掘方法,主要针对静态数据进行挖掘,而对数据流挖掘往往失效。为了解决数据流的数据挖掘问题,提出一种通过改变传统支持向量机增量式学习方法,利用轮转式结构将多分类器按照数据流时间顺序进行组合,并且通过对分类器的优化,可以提高模型对数据流分类的准确率并减少训练时间消耗。实验结果表明,该模型在保证学习精度和推广能力的同时,提高了训练速度,适合于数据流在线分类和在线学的问题。     

Joint learning of cross-modal classifier and factor analysis for multimedia data classification

In this paper, we study the problem of learning from multiple model data for the purpose of document classification. In this problem, each document is composed of two different models of data, i.e., an image and a text. We propose to represent the data of two models by projecting them to a shared data space by using cross-model factor analysis formula and classify them in the shared space by using a linear class label predictor, named cross-model classifier. The parameters of both cross-model classifier and cross-model factor analysis are learned jointly, so that they can regularize the learning of each other. We construct a unified objective function for this learning problem. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projections measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple model document data sets show the advantage of the proposed algorithm over state-of-the-art multimedia data classification methods.     

Accuracy-based learning classifier systems: models, analysis and applications to classification tasks

Recently, Learning Classifier Systems (LCS) and particularly XCS have arisen as promising methods for classification tasks and data mining. This paper investigates two models of accuracy-based learning classifier systems on different types of classification problems. Departing from XCS, we analyze the evolution of a complete action map as a knowledge representation. We propose an alternative, UCS, which evolves a best action map more efficiently. We also investigate how the fitness pressure guides the search towards accurate classifiers. While XCS bases fitness on a reinforcement learning scheme, UCS defines fitness from a supervised learning scheme. We find significant differences in how the fitness pressure leads towards accuracy, and suggest the use of a supervised approach specially for multi-class problems and problems with unbalanced classes. We also investigate the complexity factors which arise in each type of accuracy-based LCS. We provide a model on the learning complexity of LCS which is based on the representative examples given to the system. The results and observations are also extended to a set of real world classification problems, where accuracy-based LCS are shown to perform competitively with respect to other learning algorithms. The work presents an extended analysis of accuracy-based LCS, gives insight into the understanding of the LCS dynamics, and suggests open issues for further improvement of LCS on classification tasks.     

Strength-based learning classifier systems revisited: Effective rule evolution in supervised classification tasks

Learning classifier systems (LCS) are machine learning systems designed to work for both multi-step and single-step decision tasks. The latter case presents an interesting challenge for such algorithms, especially when they are applied to real-world data mining (DM) problems. The present investigation departs from the popular approach of applying accuracy-based LCS to single-step classification and aims to uncover the potential of strength-based LCS in such tasks. Although the latter family of algorithms have often been associated with poor generalization and performance, we aim at alleviating these problems by defining appropriate extensions to the traditional strength-based LCS framework. These extensions are detailed and their effect on system performance is studied through the application of the proposed algorithm on a set of artificial problems, designed to challenge its scalability and generalization abilities. The comparison of the proposed algorithm with UCS, its state-of-the-art accuracy-based counterpart, emphasizes the effects of our extended strength-based approach and validates its competitiveness in multi-class problems with various class distributions. Overall, our work presents an investigation of strength-based LCS in the domain of supervised classification. Our extensive analysis of the learning dynamics involved in these systems provides proof of their potential as real-world DM tools, inducing tractable rule-based classification models, even in the presence of severe class imbalances.     

Dynamic classifier ensemble for positive unlabeled text stream classification

Most of studies on streaming data classification are based on the assumption that data can be fully labeled. However, in real-life applications, it is impractical and time-consuming to manually label the entire stream for training. It is very common that only a small part of positive data and a large amount of unlabeled data are available in data stream environments. In this case, applying the traditional streaming algorithms with straightforward adaptation to positive unlabeled stream may not work well or lead to poor performance. In this paper, we propose a Dynamic Classifier Ensemble method for Positive and Unlabeled text stream (DCEPU) classification scenarios. We address the problem of classifying positive and unlabeled text stream with various concept drift by constructing an appropriate validation set and designing a novel dynamic weighting scheme in the classification phase. Experimental results on benchmark dataset RCV1-v2 demonstrate that the proposed method DCEPU outperforms the existing LELC (Li et?al. 2009b), DVS (with necessary adaption) (Tsymbal et?al. in Inf Fusion 9(1):56?C68, 2008), and Stacking style ensemble-based algorithm (Zhang et?al. 2008b).     

A feedback control structure for on-line learning tasks

This paper addresses adaptive control architectures for systems that respond autonomously to changing tasks. Such systems often have many sensory and motor alternatives and behavior drawn from these produces varying quality solutions. The objective is then to ground behavior in control laws which, combined with resources, enumerate closed-loop behavioral alternatives. Use of such controllers leads to analyzable and predictable composite systems, permitting the construction of abstract behavioral models. Here, discrete event system and reinforcement learning techniques are employed to constrain the behavioral alternatives and to synthesize behavior on-line. To illustrate this, a quadruped robot learning a turning gait subject to safety and kinematic constraints is presented.     

A review of improved extreme learning machine methods for data stream classification

Multimedia Tools and Applications - Classification is a hotspot in data stream mining and has gained increasing interest from various research fields. Compared with traditional data stream...     

Multiple classifier system using classification confidence for texture classification

Multimedia Tools and Applications - This paper proposes a simple yet effective novel classifier fusion strategy for multi-class texture classification. The resulting classification framework is...     

A hybrid neural network system for pattern classification tasks with missing features

A hybrid neural network comprising fuzzy ARTMAP and fuzzy c-means clustering is proposed for pattern classification with incomplete training and test data. Two benchmark problems and a real medical pattern classification tasks are employed to evaluate the effectiveness of the hybrid network. The results are analyzed and compared with those from other methods.     

Robust radar target classifier using artificial neural networks

In this paper an artificial neural network (ANN) based radar target classifier is presented, and its performance is compared with that of a conventional minimum distance classifier. Radar returns from realistic aircraft are synthesized using a thin wire time domain electromagnetic code. The time varying backscattered electric field from each target is processed using both a conventional scheme and an ANN-based scheme for classification purposes. It is found that a multilayer feedforward ANN, trained using a backpropagation learning algorithm, provides a higher percentage of successful classification than the conventional scheme. The performance of the ANN is found to be particularly attractive in an environment of low signal-to-noise ratio. The performance of both methods are also compared when a preemphasis filter is used to enhance the contributions from the high frequency poles in the target response.     

Robust automatic target recognition using learning classifier systems

This work developed and demonstrated a machine learning approach for robust ATR. The primary innovation of this work was the development of an automated way of developing inference rules that can draw on multiple models and multiple feature types to make robust ATR decisions. The key realization is that this “meta learning” problem is one of structural learning, and that it can be conducted independently of parameter learning associated with each model and feature based technique. This was accomplished by using a learning classifier system, which is based on genetics-based machine learning, for the ill conditioned combinatorial problem of structural rule learning, while using statistical and mathematical techniques for parameter learning.This system was tested on MSTAR Public Release SAR data using standard and extended operation conditions. These results were also compared against two baseline classifiers, a PCA based distance classifier and a MSE classifier. The classifiers were evaluated for accuracy (via training set classification) and robustness (via testing set classification). In both cases, the LCS based robust ATR system performed well with accuracy over 99% and robustness over 80%.     

A similarity-based approach for data stream classification

Incremental learning techniques have been used extensively to address the data stream classification problem. The most important issue is to maintain a balance between accuracy and efficiency, i.e., the algorithm should provide good classification performance with a reasonable time response. This work introduces a new technique, named Similarity-based Data Stream Classifier (SimC), which achieves good performance by introducing a novel insertion/removal policy that adapts quickly to the data tendency and maintains a representative, small set of examples and estimators that guarantees good classification rates. The methodology is also able to detect novel classes/labels, during the running phase, and to remove useless ones that do not add any value to the classification process. Statistical tests were used to evaluate the model performance, from two points of view: efficacy (classification rate) and efficiency (online response time). Five well-known techniques and sixteen data streams were compared, using the Friedman’s test. Also, to find out which schemes were significantly different, the Nemenyi’s, Holm’s and Shaffer’s tests were considered. The results show that SimC is very competitive in terms of (absolute and streaming) accuracy, and classification/updating time, in comparison to several of the most popular methods in the literature.     

Sample re-weighting hyper box classifier for multi-class data classification

In this work, we propose two novel classifiers for multi-class classification problems using mathematical programming optimisation techniques. A hyper box-based classifier (Xu & Papageorgiou, 2009) that iteratively constructs hyper boxes to enclose samples of different classes has been adopted. We firstly propose a new solution procedure that updates the sample weights during each iteration, which tweaks the model to favour those difficult samples in the next iteration and therefore achieves a better final solution. Through a number of real world data classification problems, we demonstrate that the proposed refined classifier results in consistently good classification performance, outperforming the original hyper box classifier and a number of other state-of-the-art classifiers.Furthermore, we introduce a simple data space partition method to reduce the computational cost of the proposed sample re-weighting hyper box classifier. The partition method partitions the original dataset into two disjoint regions, followed by training sample re-weighting hyper box classifier for each region respectively. Through some real world datasets, we demonstrate the data space partition method considerably reduces the computational cost while maintaining the level of prediction accuracies.     

A classifier learning system using a coevolution method for deflection yoke misconvergence pattern classification problem

Deflection yoke (DY) is one of the core components of a cathode ray tube (CRT) in a computer monitor or a television that determines the image quality. Once a DY anomaly is found from beam patterns on a display in the production line of CRTs, the remedy process should be performed through three steps: identifying misconvergence types from the anomalous display pattern, adjusting manufacturing process parameters, and fine tuning. This study focuses on discovering a classifier for the identification of DY misconvergence patterns by applying a coevolutionary classification method. The DY misconvergence classification problems may be decomposed into two subproblems, which are feature selection and classifier adaptation. A coevolutionary classification method is designed by coordinating the two subproblems, whose performances are affected by each other. The proposed method establishes a group of partial sub-regions, defined by regional feature set, and then fits a finite number of classifiers to the data pattern by using a genetic algorithm in every sub-region. A cycle of the cooperation loop is completed by evolving the sub-regions based on the evaluation results of the fitted classifiers located in the corresponding sub-regions. The classifier system has been tested with real-field data acquired from the production line of a computer monitor manufacturer in Korea, showing superior performance to other methods such as k-nearest neighbors, decision trees, and neural networks.     

Two-step learning for crowdsourcing data classification

Multimedia Tools and Applications - Crowdsourcing learning (Bonald and Combes 2016; Dawid and Skene, J R Stat Soc: Series C (Appl Stat), 28(1):20–28 1979; Karger et al. 2011; Li et al, IEEE...     

A local neural classifier for the recognition of EEG patternsassociated to mental tasks

This paper proposes a novel and simple local neural classifier for the recognition of mental tasks from on-line spontaneous EEG signals. The proposed neural classifier recognizes three mental tasks from on-line spontaneous EEG signals. Correct recognition is around 70%. This modest rate is largely compensated by two properties, namely low percentage of wrong decisions (below 5%) and rapid responses (every 1/2 s). Interestingly, the neural classifier achieves this performance with a few units, normally just one per mental task. Also, since the subject and his/her personal interface learn simultaneously from each other, subjects master it rapidly (in a few days of moderate training). Finally, analysis of learned EEG patterns confirms that for a subject to operate satisfactorily a brain interface, the latter must fit the individual features of the former.     

Robust learning from bites for data mining

Some methods from statistical machine learning and from robust statistics have two drawbacks. Firstly, they are computer-intensive such that they can hardly be used for massive data sets, say with millions of data points. Secondly, robust and non-parametric confidence intervals for the predictions according to the fitted models are often unknown. A simple but general method is proposed to overcome these problems in the context of huge data sets. An implementation of the method is scalable to the memory of the computer and can be distributed on several processors to reduce the computation time. The method offers distribution-free confidence intervals for the median of the predictions. The main focus is on general support vector machines (SVM) based on minimizing regularized risks. As an example, a combination of two methods from modern statistical machine learning, i.e. kernel logistic regression and ε-support vector regression, is used to model a data set from several insurance companies. The approach can also be helpful to fit robust estimators in parametric models for huge data sets.     

Convergent on-line algorithms for supervised learning in neural networks

We define online algorithms for neural network training, based on the construction of multiple copies of the network, which are trained by employing different data blocks. It is shown that suitable training algorithms can be defined, in a way that the disagreement between the different copies of the network is asymptotically reduced, and convergence toward stationary points of the global error function can be guaranteed. Relevant features of the proposed approach are that the learning rate must be not necessarily forced to zero and that real-time learning is permitted.     

A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine

Multimedia Tools and Applications - The characteristics of the data stream have brought enormous challenges to classification algorithms. Concept drift is the most concerning characteristics, and...     

A distributed system for learning programming on-line

Several Web-based on-line judges or on-line programming trainers have been developed in order to allow students to train their programming skills. However, their pedagogical functionalities in the learning of programming have not been clearly defined. EduJudge is a project which aims to integrate the “UVA On-line Judge”, an existing on-line programming trainer with an important number of problems and users, into an effective educational environment consisting of the e-learning platform Moodle and the competitive learning tool QUESTOURnament. The result is the EduJudge system which allows teachers to apply different pedagogical approaches using a proven e-learning platform, makes problems easy to search through an effective search engine, and provides an automated evaluation of the solutions submitted to these problems. The final objective is to provide new learning strategies to motivate students and present programming as an easy and attractive challenge. EduJudge has been tried and tested in three algorithms and programming courses in three different Engineering degrees. The students’ motivation and satisfaction levels were analysed alongside the effects of the EduJudge system on students’ academic outcomes. Results indicate that both students and teachers found that among other multiple benefits the EduJudge system facilitates the learning process. Furthermore, the experiment also showed an improvement in students’ academic outcomes. It must be noted that the students’ level of satisfaction did not depend on their computer skills or their gender.