一种基于粒子群算法的分类器设计

将粒子群算法应用于数据分类,给出了适用于粒子群算法的分类规则编码,构造了新的分类规则适应度函数来更准确的提取规则集,并通过修改粒子位置更新方程使粒子群算法适于解决分类规则挖掘问题,进而实现了基于粒子群算法的分类器设计。该文进一步用UCI基准数据集对作者提出的粒子群分类器进行了测试,并将几种不同速度与位置更新策略的粒子群算法分类器与遗传算法分类器进行对比,实验结果表明,这种粒子群分类器是一种有效、可行的分类器设计方案。     

A prototype classifier based on gravitational search algorithm

In recent years, heuristic algorithms have been successfully applied to solve clustering and classification problems. In this paper, gravitational search algorithm (GSA) which is one of the newest swarm based heuristic algorithms is used to provide a prototype classifier to face the classification of instances in multi-class data sets. The proposed method employs GSA as a global searcher to find the best positions of the representatives (prototypes). The proposed GSA-based classifier is used for data classification of some of the well-known benchmark sets. Its performance is compared with the artificial bee colony (ABC), the particle swarm optimization (PSO), and nine other classifiers from the literature. The experimental results of twelve data sets from UCI machine learning repository confirm that the GSA can successfully be applied as a classifier to classification problems.     

Hierarchical classification of protein function with ensembles of rules and particle swarm optimisation

This paper focuses on hierarchical classification problems where the classes to be predicted are organized in the form of a tree. The standard top-down divide and conquer approach for hierarchical classification consists of building a hierarchy of classifiers where a classifier is built for each internal (non-leaf) node in the class tree. Each classifier discriminates only between its child classes. After the tree of classifiers is built, the system uses them to classify test examples one class level at a time, so that when the example is assigned a class at a given level, only the child classes need to be considered at the next level. This approach has the drawback that, if a test example is misclassified at a certain class level, it will be misclassified at deeper levels too. In this paper we propose hierarchical classification methods to mitigate this drawback. More precisely, we propose a method called hierarchical ensemble of hierarchical rule sets (HEHRS), where different ensembles are built at different levels in the class tree and each ensemble consists of different rule sets built from training examples at different levels of the class tree. We also use a particle swarm optimisation (PSO) algorithm to optimise the rule weights used by HEHRS to combine the predictions of different rules into a class to be assigned to a given test example. In addition, we propose a variant of a method to mitigate the aforementioned drawback of top-down classification. These three types of methods are compared against the standard top-down hierarchical classification method in six challenging bioinformatics datasets, involving the prediction of protein function. Overall HEHRS with the rule weights optimised by the PSO algorithm obtains the best predictive accuracy out of the four types of hierarchical classification method.     

Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition

In many real-world applications, pattern recognition systems are designed a priori using limited and imbalanced data acquired from complex changing environments. Since new reference data often becomes available during operations, performance could be maintained or improved by adapting these systems through supervised incremental learning. To avoid knowledge corruption and sustain a high level of accuracy over time, an adaptive multiclassifier system (AMCS) may integrate information from diverse classifiers that are guided by a population-based evolutionary optimization algorithm. In this paper, an incremental learning strategy based on dynamic particle swarm optimization (DPSO) is proposed to evolve heterogeneous ensembles of classifiers (where each classifier corresponds to a particle) in response to new reference samples. This new strategy is applied to video-based face recognition, using an AMCS that consists of a pool of fuzzy ARTMAP (FAM) neural networks for classification of facial regions, and a niching version of DPSO that optimizes all FAM parameters such that the classification rate is maximized. Given that diversity within a dynamic particle swarm is correlated with diversity within a corresponding pool of base classifiers, DPSO properties are exploited to generate and evolve diversified pools of FAM classifiers, and to efficiently select ensembles among the pools based on accuracy and particle swarm diversity. Performance of the proposed strategy is assessed in terms of classification rate and resource requirements under different incremental learning scenarios, where new reference data is extracted from real-world video streams. Simulation results indicate the DPSO strategy provides an efficient way to evolve ensembles of FAM networks in an AMCS. Maintaining particle diversity in the optimization space yields a level of accuracy that is comparable to AMCS using reference ensemble-based and batch learning techniques, but requires significantly lower computational complexity than assessing diversity among classifiers in the feature or decision spaces.     

基于量子行为的粒子群优化算法分类规则获取*

介绍了基本的粒子群算法,并针对基本的粒子群算法在收敛性能上的缺陷,提出将具有量子行为的粒子群优化算法应用于数据挖掘学科中的分类规则获取。对加州大学厄文分校的若干数据集模式分类规则进行提取,与其他规则提取方法相比,证明该算法提高了分类规则的正确率以及全局寻优能力。     

Applying particle swarm optimization-based decision tree classifier for cancer classification on gene expression data

BackgroundThe application of microarray data for cancer classification is important. Researchers have tried to analyze gene expression data using various computational intelligence methods.PurposeWe propose a novel method for gene selection utilizing particle swarm optimization combined with a decision tree as the classifier to select a small number of informative genes from the thousands of genes in the data that can contribute in identifying cancers.ConclusionStatistical analysis reveals that our proposed method outperforms other popular classifiers, i.e., support vector machine, self-organizing map, back propagation neural network, and C4.5 decision tree, by conducting experiments on 11 gene expression cancer datasets.     

Exploring comprehensible classification rules from trained neural networks integrated with a time-varying binary particle swarm optimizer

Purpose

Extracting comprehensible classification rules is the most emphasized concept in data mining researches. In order to obtain accurate and comprehensible classification rules from databases, a new approach is proposed by combining advantages of artificial neural networks (ANN) and swarm intelligence.

Method

Artificial neural networks (ANNs) are a group of very powerful tools applied to prediction, classification and clustering in different domains. The main disadvantage of this general purpose tool is the difficulties in its interpretability and comprehensibility. In order to eliminate these disadvantages, a novel approach is developed to uncover and decode the information hidden in the black-box structure of ANNs. Therefore, in this paper a study on knowledge extraction from trained ANNs for classification problems is carried out. The proposed approach makes use of particle swarm optimization (PSO) algorithm to transform the behaviors of trained ANNs into accurate and comprehensible classification rules. Particle swarm optimization with time varying inertia weight and acceleration coefficients is designed to explore the best attribute-value combination via optimizing ANN output function.

Results

The weights hidden in trained ANNs turned into comprehensible classification rule set with higher testing accuracy rates compared to traditional rule based classifiers.     

The effects of combining classifiers with the same training statistics using Bayesian decision rules

Multiple classifier systems (MCS) are attracting increasing interest in the field of pattern recognition and machine learning. Recently, MCS are also being introduced in the remote sensing field where the importance of classifier diversity for image classification problems has not been examined. In this article, Satellite Pour l'Observation de la Terre (SPOT) IV panchromatic and multispectral satellite images are classified into six land cover classes using five base classifiers: contextual classifier, k-nearest neighbour classifier, Mahalanobis classifier, maximum likelihood classifier and minimum distance classifier. The five base classifiers are trained with the same feature sets throughout the experiments and a posteriori probability, derived from the confusion matrix of these base classifiers, is applied to five Bayesian decision rules (product rule, sum rule, maximum rule, minimum rule and median rule) for constructing different combinations of classifier ensembles. The performance of these classifier ensembles is evaluated for overall accuracy and kappa statistics. Three statistical tests, the McNemar's test, the Cochran's Q test and the Looney's F-test, are used to examine the diversity of the classification results of the base classifiers compared to the results of the classifier ensembles. The experimental comparison reveals that (a) significant diversity amongst the base classifiers cannot enhance the performance of classifier ensembles; (b) accuracy improvement of classifier ensembles can only be found by using base classifiers with similar and low accuracy; (c) increasing the number of base classifiers cannot improve the overall accuracy of the MCS and (d) none of the Bayesian decision rules outperforms the others.     

Use of particle swarm optimization for machinery fault detection

A study is presented on the application of particle swarm optimization (PSO) combined with other computational intelligence (CI) techniques for bearing fault detection in machines. The performance of two CI based classifiers, namely, artificial neural networks (ANNs) and support vector machines (SVMs) are compared. The time domain vibration signals of a rotating machine with normal and defective bearings are processed for feature extraction. The extracted features from original and preprocessed signals are used as inputs to the classifiers for detection of machine condition. The classifier parameters, e.g., the number of nodes in the hidden layer for ANNs and the kernel parameters for SVMs are selected along with input features using PSO algorithms. The classifiers are trained with a subset of the experimental data for known machine conditions and are tested using the remaining set of data. The procedure is illustrated using the experimental vibration data of a rotating machine. The roles of the number of features, PSO parameters and CI classifiers on the detection success are investigated. Results are compared with other techniques such as genetic algorithm (GA) and principal component analysis (PCA). The PSO based approach gave a test classification success rate of 98.6–100% which were comparable with GA and much better than with PCA. The results show the effectiveness of the selected features and the classifiers in the detection of the machine condition.     

A greedy classification algorithm based on association rule

Classification and association rule discovery are important data mining tasks. Using association rule discovery to construct classification systems, also known as associative classification, is a promising approach. In this paper, a new associative classification technique, Ranked Multilabel Rule (RMR) algorithm is introduced, which generates rules with multiple labels. Rules derived by current associative classification algorithms overlap in their training objects, resulting in many redundant and useless rules. However, the proposed algorithm resolves the overlapping between rules in the classifier by generating rules that does not share training objects during the training phase, resulting in a more accurate classifier. Results obtained from experimenting on 20 binary, multi-class and multi-label data sets show that the proposed technique is able to produce classifiers that contain rules associated with multiple classes. Furthermore, the results reveal that removing overlapping of training objects between the derived rules produces highly competitive classifiers if compared with those extracted by decision trees and other associative classification techniques, with respect to error rate.     

Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.     

A hybrid particle swarm optimization based fuzzy expert system for the diagnosis of coronary artery disease

This paper presents a particle swarm optimization (PSO)-based fuzzy expert system for the diagnosis of coronary artery disease (CAD). The designed system is based on the Cleveland and Hungarian Heart Disease datasets. Since the datasets consist of many input attributes, decision tree (DT) was used to unravel the attributes that contribute towards the diagnosis. The output of the DT was converted into crisp if–then rules and then transformed into fuzzy rule base. PSO was employed to tune the fuzzy membership functions (MFs). Having applied the optimized MFs, the generated fuzzy expert system has yielded 93.27% classification accuracy. The major advantage of this approach is the ability to interpret the decisions made from the created fuzzy expert system, when compared with other approaches.     

A hybrid network intrusion detection system using simplified swarm optimization (SSO)

The network intrusion detection techniques are important to prevent our systems and networks from malicious behaviors. However, traditional network intrusion prevention such as firewalls, user authentication and data encryption have failed to completely protect networks and systems from the increasing and sophisticated attacks and malwares. In this paper, we propose a new hybrid intrusion detection system by using intelligent dynamic swarm based rough set (IDS-RS) for feature selection and simplified swarm optimization for intrusion data classification. IDS-RS is proposed to select the most relevant features that can represent the pattern of the network traffic. In order to improve the performance of SSO classifier, a new weighted local search (WLS) strategy incorporated in SSO is proposed. The purpose of this new local search strategy is to discover the better solution from the neighborhood of the current solution produced by SSO. The performance of the proposed hybrid system on KDDCup 99 dataset has been evaluated by comparing it with the standard particle swarm optimization (PSO) and two other most popular benchmark classifiers. The testing results showed that the proposed hybrid system can achieve higher classification accuracy than others with 93.3% and it can be one of the competitive classifier for the intrusion detection system.     

Particle swarm optimized multiple regression linear model for data classification

This paper presents a new data classification method based on particle swarm optimization (PSO) techniques. The paper discusses the building of a classifier model based on multiple regression linear approach. The coefficients of multiple regression linear models (MRLMs) are estimated using least square estimation technique and PSO techniques for percentage of correct classification performance comparisons. The mathematical models are developed for many real world datasets collected from UCI machine repository. The mathematical models give the user an insight into how the attributes are interrelated to predict the class membership. The proposed approach is illustrated on many real data sets for classification purposes. The comparison results on the illustrative examples show that the PSO based approach is superior to traditional least square approach in classifying multi-class data sets.     

Rough set feature selection and rule induction for prediction of malignancy degree in brain glioma

The degree of malignancy in brain glioma is assessed based on magnetic resonance imaging (MRI) findings and clinical data before operation. These data contain irrelevant features, while uncertainties and missing values also exist. Rough set theory can deal with vagueness and uncertainty in data analysis, and can efficiently remove redundant information. In this paper, a rough set method is applied to predict the degree of malignancy. As feature selection can improve the classification accuracy effectively, rough set feature selection algorithms are employed to select features. The selected feature subsets are used to generate decision rules for the classification task. A rough set attribute reduction algorithm that employs a search method based on particle swarm optimization (PSO) is proposed in this paper and compared with other rough set reduction algorithms. Experimental results show that reducts found by the proposed algorithm are more efficient and can generate decision rules with better classification performance. The rough set rule-based method can achieve higher classification accuracy than other intelligent analysis methods such as neural networks, decision trees and a fuzzy rule extraction algorithm based on Fuzzy Min-Max Neural Networks (FRE-FMMNN). Moreover, the decision rules induced by rough set rule induction algorithm can reveal regular and interpretable patterns of the relations between glioma MRI features and the degree of malignancy, which are helpful for medical experts.     

Combining models from neural networks and inductive learning algorithms

The knowledge-based artificial neural network (KBANN) is composed of phases involving the expression of domain knowledge, the abstraction of domain knowledge at neural networks, the training of neural networks, and finally, the extraction of rules from trained neural networks. The KBANN attempts to open up the neural network black box and generates symbolic rules with (approximately) the same predictive power as the neural network itself. An advantage of using KBANN is that the neural network considers the contribution of the inputs towards classification as a group, while rule-based algorithms like C5.0 measure the individual contribution of the inputs one at a time as the tree is grown. The knowledge consolidation model (KCM) combines the rules extracted using KBANN (NeuroRule), frequency matrix (which is similar to the Naïve Bayesian technique), and C5.0 algorithm. The KCM can effectively integrate multiple rule sets into one centralized knowledge base. The cumulative rules from other single models can improve overall performance as it can reduce error-term and increase R-square. The key idea in the KCM is to combine a number of classifiers such that the resulting combined system achieves higher classification accuracy and efficiency than the original single classifiers. The aim of KCM is to design a composite system that outperforms any individual classifier by pooling together the decisions of all classifiers. Another advantage of KCM is that it does not need the memory space to store the dataset as only extracted knowledge is necessary in build this integrated model. It can also reduce the costs from storage allocation, memory, and time schedule. In order to verify the feasibility and effectiveness of KCM, personal credit rating dataset provided by a local bank in Seoul, Republic of Korea is used in this study. The results from the tests show that the performance of KCM is superior to that of the other single models such as multiple discriminant analysis, logistic regression, frequency matrix, neural networks, decision trees, and NeuroRule. Moreover, our model is superior to a previous algorithm for the extraction of rules from general neural networks.     

An early software-quality classification based on improved grey relational classifier

The inherent uncertainty and incomplete information of the software development process presents particular challenges for identifying fault-prone modules and providing a preferred model early enough in a development cycle in order to guide software enhancement efforts effectively. Grey relational analysis (GRA) of grey system theory is a well known approach that is utilized for generalizing estimates under small sample and uncertain conditions. This paper examines the potential benefits for providing an early software-quality classification based on improved grey relational classifier. The particle swarm optimization (PSO) approach is adopted to explore the best fit of weights on software metrics in the GRA approach for deriving a classifier with preferred balance of misclassification rates. We have demonstrated our approach by using the data from the medical information system dataset. Empirical results show that the proposed approach provides a preferred balance of misclassification rates than the grey relational classifiers without using PSO. It also outperforms the widely used classifiers of classification and regression trees (CART) and C4.5 approaches.     

Polynomial-based radial basis function neural networks (P-RBF NNs) and their application to pattern classification

Polynomial neural networks have been known to exhibit useful properties as classifiers and universal approximators. In this study, we introduce a concept of polynomial-based radial basis function neural networks (P-RBF NNs), present a design methodology and show the use of the networks in classification problems. From the conceptual standpoint, the classifiers of this form can be expressed as a collection of “if-then” rules. The proposed architecture uses two essential development mechanisms. Fuzzy clustering (Fuzzy C-Means, FCM) is aimed at the development of condition parts of the rules while the corresponding conclusions of the rules are formed by some polynomials. A detailed learning algorithm for the P-RBF NNs is developed. The proposed classifier is applied to two-class pattern classification problems. The performance of this classifier is contrasted with the results produced by the “standard” RBF neural networks. In addition, the experimental application covers a comparative analysis including several previous commonly encountered methods such as standard neural networks, SVM, SOM, PCA, LDA, C4.5, and decision trees. The experimental results reveal that the proposed approach comes with a simpler structure of the classifier and better prediction capabilities.     

A hybrid classifier combining SMOTE with PSO to estimate 5-year survivability of breast cancer patients

In this study, we propose a set of new algorithms to enhance the effectiveness of classification for 5-year survivability of breast cancer patients from a massive data set with imbalanced property. The proposed classifier algorithms are a combination of synthetic minority oversampling technique (SMOTE) and particle swarm optimization (PSO), while integrating some well known classifiers, such as logistic regression, C5 decision tree (C5) model, and 1-nearest neighbor search. To justify the effectiveness for this new set of classifiers, the g-mean and accuracy indices are used as performance indexes; moreover, the proposed classifiers are compared with previous literatures. Experimental results show that the hybrid algorithm of SMOTE + PSO + C5 is the best one for 5-year survivability of breast cancer patient classification among all algorithm combinations. We conclude that, implementing SMOTE in appropriate searching algorithms such as PSO and classifiers such as C5 can significantly improve the effectiveness of classification for massive imbalanced data sets.     

层次化粒子群优化算法及其在分类规则提取中的应用

介绍层次化粒子群优化算法,采用自下而上的方式在层次结构中移动粒子．将此算法应用到分类问题,用于Iris数据集的分类规则提取,并与标准的粒子群优化（Particle Swarm Optimizer,PSO）算法相比较,结果表明提取规则的精度得到提高．