Motor fault detection and diagnosis using a hybrid FMM-CART model with online learning

In this paper, a hybrid online learning model that combines the fuzzy min–max (FMM) neural network and the Classification and Regression Tree (CART) for motor fault detection and diagnosis tasks is described. The hybrid model, known as FMM-CART, incorporates the advantages of both FMM and CART for undertaking data classification (with FMM) and rule extraction (with CART) problems. In particular, the CART model is enhanced with an importance predictor-based feature selection measure. To evaluate the effectiveness of the proposed online FMM-CART model, a series of experiments using publicly available data sets containing motor bearing faults is first conducted. The results (primarily prediction accuracy and model complexity) are analyzed and compared with those reported in the literature. Then, an experimental study on detecting imbalanced voltage supply of an induction motor using a laboratory-scale test rig is performed. In addition to producing accurate results, a set of rules in the form of a decision tree is extracted from FMM-CART to provide explanations for its predictions. The results positively demonstrate the usefulness of FMM-CART with online learning capabilities in tackling real-world motor fault detection and diagnosis tasks.     

Application of fuzzy ARTMAP and fuzzy <Emphasis Type="Italic">c</Emphasis>-means clustering to pattern classification with incomplete data

In this paper, a hybrid neural network that is capable of incremental learning and classification of patterns with incomplete data is proposed. Fuzzy ARTMAP (FAM) is employed as the constituting network for pattern classification while fuzzy c-means (FCM) clustering is used as the underlying algorithm for processing training as well as test samples with missing features. To handle an incomplete training set, FAM is first trained using complete samples only. Missing features of the training samples are estimated and replaced using two FCM-based strategies. Then, network training is conducted using all the complete and estimated samples. To handle an incomplete test set, a non-substitution FCM-based strategy is employed so that a predicted output can be produced rapidly. The performance of the proposed hybrid network is evaluated using a benchmark problem, and its practical applicability is demonstrated using a medical diagnosis task. The results are compared, analysed and quantified statistically with the bootstrap method. Implications of the proposed network for pattern classification tasks with incomplete data are discussed.     

Using artificial neural networks to enhance CART

Accuracy is a critical factor in predictive modeling. A predictive model such as a decision tree must be accurate to draw conclusions about the system being modeled. This research aims at analyzing and improving the performance of classification and regression trees (CART), a decision tree algorithm, by evaluating and deriving a new methodology based on the performance of real-world data sets that were studied. This paper introduces a new approach to tree induction to improve the efficiency of the CART algorithm by combining the existing functionality of CART with the addition of artificial neural networks (ANNs). Trained ANNs are utilized by the tree induction algorithm by generating new, synthetic data, which have been shown to improve the overall accuracy of the decision tree model when actual training samples are limited. In this paper, traditional decision trees developed by the standard CART methodology are compared with the enhanced decision trees that utilize the ANN’s synthetic data generation, or CART+. This research demonstrates the improved accuracies that can be obtained with CART+, which can ultimately improve the knowledge that can be extracted by researchers about a system being modeled.     

A hybrid intelligent system for medical data classification

In this paper, a hybrid intelligent system that consists of the Fuzzy Min–Max neural network, the Classification and Regression Tree, and the Random Forest model is proposed, and its efficacy as a decision support tool for medical data classification is examined. The hybrid intelligent system aims to exploit the advantages of the constituent models and, at the same time, alleviate their limitations. It is able to learn incrementally from data samples (owing to Fuzzy Min–Max neural network), explain its predicted outputs (owing to the Classification and Regression Tree), and achieve high classification performances (owing to Random Forest). To evaluate the effectiveness of the hybrid intelligent system, three benchmark medical data sets, viz., Breast Cancer Wisconsin, Pima Indians Diabetes, and Liver Disorders from the UCI Repository of Machine Learning, are used for evaluation. A number of useful performance metrics in medical applications which include accuracy, sensitivity, specificity, as well as the area under the Receiver Operating Characteristic curve are computed. The results are analyzed and compared with those from other methods published in the literature. The experimental outcomes positively demonstrate that the hybrid intelligent system is effective in undertaking medical data classification tasks. More importantly, the hybrid intelligent system not only is able to produce good results but also to elucidate its knowledge base with a decision tree. As a result, domain users (i.e., medical practitioners) are able to comprehend the prediction given by the hybrid intelligent system; hence accepting its role as a useful medical decision support tool.     

基于神经网络的模糊决策树改进算法

传统决策树通过对特征空间的递归划分寻找决策边界,给出特征空间的“硬”划分。但对于处理大数据和复杂模式问题时,这种精确决策边界降低了决策树的泛化能力。为了让决策树算法获得对不精确知识的自动获取,把模糊理论引进了决策树,并在建树过程中,引入神经网络作为决策树叶节点,提出了一种基于神经网络的模糊决策树改进算法。在神经网络模糊决策树中,分类器学习包含两个阶段：第一阶段采用不确定性降低的启发式算法对大数据进行划分,直到节点划分能力低于真实度阈值[ε]停止模糊决策树的增长;第二阶段对该模糊决策树叶节点利用神经网络做具有泛化能力的分类。实验结果表明,相较于传统的分类学习算法,该算法准确率高,对识别大数据和复杂模式的分类问题能够通过结构自适应确定决策树规模。     

Condition monitoring of induction motors: A review and an application of an ensemble of hybrid intelligent models

In this paper, a review on condition monitoring of induction motors is first presented. Then, an ensemble of hybrid intelligent models that is useful for condition monitoring of induction motors is proposed. The review covers two parts, i.e., (i) a total of nine commonly used condition monitoring methods of induction motors; and (ii) intelligent learning models for condition monitoring of induction motors subject to single and multiple input signals. Based on the review findings, the Motor Current Signature Analysis (MCSA) method is selected for this study owing to its online, non-invasive properties and its requirement of only single input source; therefore leading to a cost-effective condition monitoring method. A hybrid intelligent model that consists of the Fuzzy Min–Max (FMM) neural network and the Random Forest (RF) model comprising an ensemble of Classification and Regression Trees is developed. The majority voting scheme is used to combine the predictions produced by the resulting FMM–RF ensemble (or FMM–RFE) members. A benchmark problem is first deployed to evaluate the usefulness of the FMM–RFE model. Then, the model is applied to condition monitoring of induction motors using a set of real data samples. Specifically, the stator current signals of induction motors are obtained using the MCSA method. The signals are processed to produce a set of harmonic-based features for classification using the FMM–RFE model. The experimental results show good performances in both noise-free and noisy environments. More importantly, a set of explanatory rules in the form of a decision tree can be extracted from the FMM–RFE model to justify its predictions. The outcomes ascertain the effectiveness of the proposed FMM–RFE model in undertaking condition monitoring tasks, especially for induction motors, under different environments.     

南方地区复杂条件下的耕地面积遥感提取方法

针对我国南方地区植被类型复杂、地形复杂和地块破碎等原因导致耕地信息提取精度较低问题,提出了一种面向对象和CART决策树结合的复杂条件下耕地面积提取方法。以广西南宁市隆安县与武鸣县地区为研究区,采用Sentinel-2A影像,结合数字高程数据（Digital Elevation Model,DEM）及归一化植被指数（Normalized Difference Vegetation Index,NDVI）等多源数据,利用面向对象分割技术识别地块信息,然后以地块为单位采用CART（Classification And Regression Tree,CART）决策树分类法,依据不同地类的形状、光谱特征,提取研究区的耕地。结果表明：面向对象的CART决策树分类方法分类总体精度和Kappa系数分别为96.1%和0.94,相比较于未加入面向对象分割的CART决策树耕地信息提取总体精度提高Kappa系数提高0.54,面向对象的分割方法有利于减少复杂背景对耕地提取的影响。基于面向对象的CART决策树分类方法相比较于传统方法对研究区耕地信息的提取有较好的精确性,能够提高耕地信息的提取精度。     

Inducing decision trees with an ant colony optimization algorithm

Decision trees have been widely used in data mining and machine learning as a comprehensible knowledge representation. While ant colony optimization (ACO) algorithms have been successfully applied to extract classification rules, decision tree induction with ACO algorithms remains an almost unexplored research area. In this paper we propose a novel ACO algorithm to induce decision trees, combining commonly used strategies from both traditional decision tree induction algorithms and ACO. The proposed algorithm is compared against three decision tree induction algorithms, namely C4.5, CART and cACDT, in 22 publicly available data sets. The results show that the predictive accuracy of the proposed algorithm is statistically significantly higher than the accuracy of both C4.5 and CART, which are well-known conventional algorithms for decision tree induction, and the accuracy of the ACO-based cACDT decision tree algorithm.     

A hybrid model of fuzzy ARTMAP and genetic algorithm for data classification and rule extraction

A two-stage hybrid model for data classification and rule extraction is proposed. The first stage uses a Fuzzy ARTMAP (FAM) classifier with Q-learning (known as QFAM) for incremental learning of data samples, while the second stage uses a Genetic Algorithm (GA) for rule extraction from QFAM. Given a new data sample, the resulting hybrid model, known as QFAM-GA, is able to provide prediction pertaining to the target class of the data sample as well as to give a fuzzy if-then rule to explain the prediction. To reduce the network complexity, a pruning scheme using Q-values is applied to reduce the number of prototypes generated by QFAM. A ‘don't care’ technique is employed to minimize the number of input features using the GA. A number of benchmark problems are used to evaluate the effectiveness of QFAM-GA in terms of test accuracy, noise tolerance, model complexity (number of rules and total rule length). The results are comparable, if not better, than many other models reported in the literature. The main significance of this research is a usable and useful intelligent model (i.e., QFAM-GA) for data classification in noisy conditions with the capability of yielding a set of explanatory rules with minimum antecedents. In addition, QFAM-GA is able to maximize accuracy and minimize model complexity simultaneously. The empirical outcome positively demonstrate the potential impact of QFAM-GA in the practical environment, i.e., providing an accurate prediction with a concise justification pertaining to the prediction to the domain users, therefore allowing domain users to adopt QFAM-GA as a useful decision support tool in assisting their decision-making processes.     

Incremental learning for online tool condition monitoring using Ellipsoid ARTMAP network model

In this paper, an Ellipsoid ARTMAP (EAM) network model based on incremental learning algorithm is proposed to realize online learning and tool condition monitoring. The main characteristic of EAM model is that hyper-ellipsoid is used for geometric representation of categories which can depict the sample distribution robustly and accurately. Meanwhile, adaptive resonance based strategy can realize the update of the hyper-ellipsoid node locally and monotonically. Therefore, the model has strong incremental learning ability, which guarantees the constructed classifier can learn new knowledge without forgetting the original information. Based on incremental EAM model, a tool condition monitoring system is realized. In this system, features are firstly extracted from the force and vibration signals to depict dynamic features of tool wear process. Then, fast correlation based filter (FCBF) method is introduced to select the minimum redundant features adaptively so as to decrease the feature redundancy and improve classifier robustness. Based on the selected features, EAM based incremental classifier is constructed to realize recognition of the tool wear states. To show the effectiveness of the proposed method, multi-teeth milling experiments of Ti-6Al-4V alloy were carried out. Moreover, to estimate the generation error of the classifiers accurately, a five-fold cross validation method is utilized. By comparison with the commonly used Fuzzy ARTMAP (FAM) classifier, it can be shown that the averaging recognition rate of EAM initial classifier can reach 98.67%, which is higher than FAM. Moreover, the incremental learning ability of EAM is also analyzed and compared with FAM using the new data coming from different cutting passes and tool wear category. The results show that the updated EAM classifier can get higher classification accuracy on the original knowledge while realizing effective online learning of the new knowledge.     

CNN-ELM混合短文本分类模型

针对目前自然语言处理研究中,使用卷积神经网络（CNN）进行短文本分类任务时可以结合不同神经网络结构与分类算法以提高分类性能的问题,提出了一种结合卷积神经网络与极速学习机的CNN-ELM混合短文本分类模型。使用词向量训练构成文本矩阵作为输入数据,然后使用卷积神经网络提取特征并使用Highway网络进行特征优化,最后使用误差最小化极速学习机(EM-ELM)作为分类器完成短文本分类任务。与其他模型相比,该混合模型能够提取更具代表性的特征并能快速准确地输出分类结果。在多种英文数据集上的实验结果表明提出的CNN-ELM混合短文本分类模型比传统机器学习模型与深度学习模型更适合完成短文本分类任务。     

基于模糊优势互补互信息的有序决策树算法

传统决策树算法应用于有序分类任务时存在两个问题：传统决策树算法没有引入序关系,因此无法学习和抽取数据集中的序结构;现实生活中存在大量模糊而非精确的知识,而传统的决策树算法无法处理存在模糊属性取值的数据。针对上述问题,提出了基于模糊优势互补互信息的有序决策树算法。首先,使用优势集表示数据中的序关系,并引入模糊集来计算优势集以形成模糊优势集。模糊优势集不仅能反映数据中的序信息,而且能自动获取不精确知识。然后,在模糊优势集的基础上将互补互信息进行推广,并提出了模糊优势互补互信息。最后,使用模糊优势互补互信息作为启发式,设计出基于模糊优势互补互信息的有序决策树算法。在5个人工数据集及9个现实数据集上的实验结果表明,所提算法在有序分类任务上较经典决策树算法取得了更低的分类误差。     

基于新型不纯度度量的代价敏感随机森林分类器

针对不平衡数据集的有效分类问题,提出一种结合代价敏感学习和随机森林算法的分类器。首先提出了一种新型不纯度度量,该度量不仅考虑了决策树的总代价,还考虑了同一节点对于不同样本的代价差异;其次,执行随机森林算法,对数据集作K次抽样,构建K个基础分类器;然后,基于提出的不纯度度量,通过分类回归树(CART)算法来构建决策树,从而形成决策树森林;最后,随机森林通过投票机制做出数据分类决策。在UCI数据库上进行实验,与传统随机森林和现有的代价敏感随机森林分类器相比,该分类器在分类精度、AUC面积和Kappa系数这3种性能度量上都具有良好的表现。     

Online adaptive decision trees: pattern classification and function approximation

Recently we have shown that decision trees can be trained in the online adaptive (OADT) mode (Basak, 2004), leading to better generalization score. OADTs were bottlenecked by the fact that they are able to handle only two-class classification tasks with a given structure. In this article, we provide an architecture based on OADT, ExOADT, which can handle multiclass classification tasks and is able to perform function approximation. ExOADT is structurally similar to OADT extended with a regression layer. We also show that ExOADT is capable not only of adapting the local decision hyperplanes in the nonterminal nodes but also has the potential of smoothly changing the structure of the tree depending on the data samples. We provide the learning rules based on steepest gradient descent for the new model ExOADT. Experimentally we demonstrate the effectiveness of ExOADT in the pattern classification and function approximation tasks. Finally, we briefly discuss the relationship of ExOADT with other classification models.     

An online pruning strategy for supervised ARTMAP-based neural networks

Identifying an appropriate architecture of an artificial neural network (ANN) for a given task is important because learning and generalisation of an ANN is affected by its structure. In this paper, an online pruning strategy is proposed to participate in the learning process of two constructive networks, i.e. fuzzy ARTMAP (FAM) and fuzzy ARTMAP with dynamic decay adjustment (FAMDDA), and the resulting hybrid networks are called FAM/FAMDDA with temporary nodes (i.e. FAM-T and FAMDDA-T, respectively). FAM-T and FAMDDA-T possess a capability of reducing the network complexity online by removing unrepresentative neurons. The performances of FAM-T and FAMDDA-T are evaluated and compared with those of FAM and FAMDDA using a total of 13 benchmark data sets. To demonstrate the applicability of FAM-T and FAMDDA-T, a real fault detection and diagnosis task in a power plant is tested. The results from both benchmark studies and real-world application show that FAMDDA-T and FAM-T are able to yield satisfactory classification performances, with the advantage of having parsimonious network structures.     

基于改进的代价敏感决策树的网络贷款分类

在网络贷款用户数据集中，贷款成功和贷款失败的用户数量存在着严重的不平衡，传统的机器学习算法在解决该类问题时注重整体分类正确率，导致贷款成功用户的预测精度较低。针对此问题，在代价敏感决策树敏感函数的计算中加入类分布，以减弱正负样本数量对误分类代价的影响，构建改进的代价敏感决策树；以该决策树作为基分类器并以分类准确度作为衡量标准选择表现较好的基分类器，将它们与最后阶段生成的分类器集成得到最终的分类器。实验结果表明，与已有的常用于解决此类问题的算法（如MetaCost算法、代价敏感决策树、AdaCost算法等）相比，改进的代价敏感决策树对网络贷款用户分类可以降低总体的误分类错误率，具有更强的泛化能力。     

Information entropy for ordinal classification

Ordinal classification plays an important role in various decision making tasks.However, little attention is paid to this type of learning tasks compared with general classification learning.Shannon information entropy and the derived measure of mutual information play a fundamental role in a number of learning algorithms including feature evaluation, selection and decision tree construction.These measures are not applicable to ordinal classification for they cannot characterize the consistency of monotonic...     

基于一致性度量属性约简的决策树构建优化研究

针对样本中有无关的、冗余的属性会降低决策树算法的分类精度,本文提出基于一致性度量属性约简后构建决策树的方法。对UCI机器学习数据库中5个两类分类样本离散化后,分别基于粗糙集和一致性度量的属性约简来构建C45和CART决策树,实验表明基于一致性度量属性约简构建的决策树有较高的精度和可行性。     

自组织决策树的联想记忆在线学习模型*

针对多源数据在线学习环境下的联想记忆建模问题,并综合考虑计算高效性、噪声鲁棒性等目标,提出基于自组织决策树的联想记忆在线学习模型.首先根据模式数据内在结构进行类内信息增强和噪声约简,然后基于信息熵增益的决策树算法对约简后数据进行子域划分,最后通过子域关系学习建模多源数据的联想关系.理论分析模型的学习稳定性.实验表明,文中模型在含噪数据在线分类学习和异联想建模问题上具有优良性能.     

A hybrid neural network model for rule generation and its application to process fault detection and diagnosis

In this paper, a hybrid neural network model, based on the integration of fuzzy ARTMAP (FAM) and the rectangular basis function network (RecBFN), which is capable of learning and revealing fuzzy rules is proposed. The hybrid network is able to classify data samples incrementally and, at the same time, to extract rules directly from the network weights for justifying its predictions. With regards to process systems engineering, the proposed network is applied to a fault detection and diagnosis task in a power generation station. Specifically, the efficiency of the network in monitoring the operating conditions of a circulating water (CW) system is evaluated by using a set of real sensor measurements collected from the power station. The rules extracted are analyzed, discussed, and compared with those from a rule extraction method of FAM. From the comparison results, it is observed that the proposed network is able to extract more meaningful rules with a lower degree of rule redundancy and higher interpretability within the neural network framework. The extracted rules are also in agreement with experts’ opinions for maintaining the CW system in the power generation plant.