Robust automatic target recognition using learning classifier systems

This work developed and demonstrated a machine learning approach for robust ATR. The primary innovation of this work was the development of an automated way of developing inference rules that can draw on multiple models and multiple feature types to make robust ATR decisions. The key realization is that this “meta learning” problem is one of structural learning, and that it can be conducted independently of parameter learning associated with each model and feature based technique. This was accomplished by using a learning classifier system, which is based on genetics-based machine learning, for the ill conditioned combinatorial problem of structural rule learning, while using statistical and mathematical techniques for parameter learning.This system was tested on MSTAR Public Release SAR data using standard and extended operation conditions. These results were also compared against two baseline classifiers, a PCA based distance classifier and a MSE classifier. The classifiers were evaluated for accuracy (via training set classification) and robustness (via testing set classification). In both cases, the LCS based robust ATR system performed well with accuracy over 99% and robustness over 80%.     

On Taxonomy and Evaluation of Feature Selection‐Based Learning Classifier System Ensemble Approaches for Data Mining Problems

Ensemble methods aim at combining multiple learning machines to improve the efficacy in a learning task in terms of prediction accuracy, scalability, and other measures. These methods have been applied to evolutionary machine learning techniques including learning classifier systems (LCSs). In this article, we first propose a conceptual framework that allows us to appropriately categorize ensemble‐based methods for fair comparison and highlights the gaps in the corresponding literature. The framework is generic and consists of three sequential stages: a pre‐gate stage concerned with data preparation; the member stage to account for the types of learning machines used to build the ensemble; and a post‐gate stage concerned with the methods to combine ensemble output. A taxonomy of LCSs‐based ensembles is then presented using this framework. The article then focuses on comparing LCS ensembles that use feature selection in the pre‐gate stage. An evaluation methodology is proposed to systematically analyze the performance of these methods. Specifically, random feature sampling and rough set feature selection‐based LCS ensemble methods are compared. Experimental results show that the rough set‐based approach performs significantly better than the random subspace method in terms of classification accuracy in problems with high numbers of irrelevant features. The performance of the two approaches are comparable in problems with high numbers of redundant features.     

Evolving optimum populations with XCS classifier systems

The main goal of the research direction is to extract building blocks of knowledge from a problem domain. Once extracted successfully, these building blocks are to be used in learning more complex problems of the domain, in an effort to produce a scalable learning classifier system (LCS). However, whilst current LCS (and other evolutionary computation techniques) discover good rules, they also create sub-optimum rules. Therefore, it is difficult to separate good building blocks of information from others without extensive post-processing. In order to provide richness in the LCS alphabet, code fragments similar to tree expressions in genetic programming are adopted. The accuracy-based XCS concept is used as it aims to produce maximally general and accurate classifiers, albeit the rule base requires condensation (compaction) to remove spurious classifiers. Serendipitously, this work on scalability of LCS produces compact rule sets that can be easily converted to the optimum population. The main contribution of this work is the ability to clearly separate the optimum rules from others without the need for expensive post-processing for the first time in LCS. This paper identifies that consistency of action in rich alphabets guides LCS to optimum rule sets.     

Predicting Code Smells and Analysis of Predictions: Using Machine Learning Techniques and Software Metrics

Code smell detection is essential to improve software quality, enhancing software maintainability, and decrease the risk of faults and failures in the software system. In this paper, we proposed a code smell prediction approach based on machine learning techniques and software metrics. The local interpretable model-agnostic explanations (LIME) algorithm was further used to explain the machine learning model’s predictions and interpretability. The datasets obtained from Fontana et al. were reformed and used to build binary-label and multi-label datasets. The results of 10-fold cross-validation show that the performance of tree-based algorithms (mainly Random Forest) is higher compared with kernel-based and network-based algorithms. The genetic algorithm based feature selection methods enhance the accuracy of these machine learning algorithms by selecting the most relevant features in each dataset. Moreover, the parameter optimization techniques based on the grid search algorithm significantly enhance the accuracy of all these algorithms. Finally, machine learning techniques have high potential in predicting the code smells, which contribute to detect these smells and enhance the software’s quality.

     

基于平行坐标的贝叶斯可视化分类方法

结合信息可视化与机器学习技术,提出一种基于多元数据平行坐标图表示的贝叶斯可视化分类方法。该方法基于类条件概率密度估计对平行坐标图表示进行优化,最后对变换后的各变量值加权求和,用贝叶斯法则分类。这种方法通过平行坐标来使不可见的数据和算法变得可见,从而易于利用专家领域知识,分类结果容易理解,特别适合应用到疾病诊断等医学领域的模式识别问题。     

An evidential cooperative multi-agent system

The cooperative learning systems (COLS) are an interesting way of research in Artificial Intelligence. This is because an intelligence form can emerge by interacting simple agents in these systems. In literature, we can find many learning techniques, which can be improved by combining them to a cooperative learning, this one can be considered as a special case of bagging. In particular, learning classifier systems (LCS) are adapted to cooperative learning systems because LCS manipulate rules and, hence, knowledge exchange between agents is facilitated. However, a COLS has to use a combination mechanism in order to aggregate information exchanged between agents, this combination mechanism must take in consideration the nature of information manipulated by the agents. In this paper we investigate a cooperative learning system based on the Evidential Classifier System, the cooperative system uses Dempster–Shafer theory as a support to make data fusion. This is due to the fact that the Evidential Classifier System is itself based on this theory. We present some ways to make cooperation by using this architecture and discuss the characteristics of such architecture by comparing the obtained results with those obtained by an individual approach.     

Ensemble Rough Hypercuboid Approach for Classifying Cancers

Cancer classification is the critical basis for patient-tailored therapy. Conventional histological analysis tends to be unreliable because different tumors may have similar appearance. The advances in microarray technology make individualized therapy possible. Various machine learning methods can be employed to classify cancer tissue samples based on microarray data. However, few methods can be elegantly adopted for generating accurate and reliable as well as biologically interpretable rules. In this paper, we introduce an approach for classifying cancers based on the principle of minimal rough fringe. For training rough hypercuboid classifiers from gene expression data sets, the method dynamically evaluates all available genes and sifts the genes with the smallest implicit regions as the dimensions of implicit hypercuboids. An unseen object is predicted to be a certain class if it falls within the corresponding class hypercuboid. Based upon the method, ensemble rough hypercuboid classifiers are subsequently constructed. Experimental results on some open cancer gene expression data sets show that the proposed method is capable of generating accurate and interpretable rules compared with some other machine learning methods. Hence, it is a feasible way of classifying cancer tissues in biomedical applications.     

The journey of graph kernels through two decades

In the real world all events are connected. There is a hidden network of dependencies that governs behavior of natural processes. Without much argument it can be said that, of all the known data-structures, graphs are naturally suitable to model such information. But to learn to use graph data structure is a tedious job as most operations on graphs are computationally expensive, so exploring fast machine learning techniques for graph data has been an active area of research and a family of algorithms called kernel based approaches has been famous among researchers of the machine learning domain. With the help of support vector machines, kernel based methods work very well for learning with Gaussian processes. In this survey we will explore various kernels that operate on graph representations. Starting from the basics of kernel based learning we will travel through the history of graph kernels from its first appearance to discussion of current state of the art techniques in practice.     

基于C5.0决策树算法的西北干旱区土地覆盖分类研究——以甘肃省武威市为例

西北干旱区面积广阔,由于土地利用类型多样,成因复杂,对环境变化敏感、变化过程快、幅度大、景观差异明显等特点,在影像上表现出的“同物异谱”现象明显 |利用常规目视解译、监督非监督分类、人工参与的决策树分类等方法在效率或精度等方面各有其缺陷。采用机器学习C5.0决策树算法,综合利用地物波谱、NDVI、TC、纹理等信息,根据样本数据自动挖掘分类规则并对整个研究区进行地物分类。机器学习的决策树可以挖掘出更多的分类规则,C5.0算法对采样数据的分布没有要求,可以处理离散和连续数据,生成的规则易于理解,分类精度高,可以满足西北干旱区大面积的土地利用/覆被变化制图的需要。     

面向简化规则的集成学习模型及规则约简策略

随着机器学习模型的广泛应用,研究者们逐渐认识到这类方法的局限之处。这些模型大多数为黑盒模型,导致其可解释性较差。为了解决这一问题,以集成学习模型为基础,提出了一种基于规则的可解释模型以及规则约简方法,包括生成优化的随机森林模型、冗余规则的发现和约简等步骤。首先,提出了一种随机森林模型的评价方法,并基于强化学习的思想对随机森林模型的关键参数进行了优化,得到了更具可解释性的随机森林模型。其次,对随机森林模型中提取的规则集进行了冗余消除,得到了更加精简的规则集。在公开数据集上的实验结果表明,生成的规则集在预测准确率和可解释性方面均表现优秀。     

A new variance-based approach for discriminative feature extraction in machine hearing classification using spectrogram features

Machine hearing is an emerging research field that is analogous to machine vision in that it aims to equip computers with the ability to hear and recognise a variety of sounds. It is a key enabler of natural human–computer speech interfacing, as well as in areas such as automated security surveillance, environmental monitoring, smart homes/buildings/cities. Recent advances in machine learning allow current systems to accurately recognise a diverse range of sounds under controlled conditions. However doing so in real-world noisy conditions remains a challenging task. Several front–end feature extraction methods have been used for machine hearing, employing speech recognition features like MFCC and PLP, as well as image-like features such as AIM and SIF. The best choice of feature is found to be dependent upon the noise environment and machine learning techniques used. Machine learning methods such as deep neural networks have been shown capable of inferring discriminative classification rules from less structured front–end features in related domains. In the machine hearing field, spectrogram image features have recently shown good performance for noise-corrupted classification using deep neural networks. However there are many methods of extracting features from spectrograms. This paper explores a novel data-driven feature extraction method that uses variance-based criteria to define spectral pooling of features from spectrograms. The proposed method, based on maximising the pooled spectral variance of foreground and background sound models, is shown to achieve very good performance for robust classification.     

Rolling element bearing fault diagnosis using wavelet transform

This paper is focused on fault diagnosis of ball bearings having localized defects (spalls) on the various bearing components using wavelet-based feature extraction. The statistical features required for the training and testing of artificial intelligence techniques are calculated by the implementation of a wavelet based methodology developed using Minimum Shannon Entropy Criterion. Seven different base wavelets are considered for the study and Complex Morlet wavelet is selected based on minimum Shannon Entropy Criterion to extract statistical features from wavelet coefficients of raw vibration signals. In the methodology, firstly a wavelet theory based feature extraction methodology is developed that demonstrates the information of fault from the raw signals and then the potential of various artificial intelligence techniques to predict the type of defect in bearings is investigated. Three artificial intelligence techniques are used for faults classifications, out of which two are supervised machine learning techniques i.e. support vector machine, learning vector quantization and other one is an unsupervised machine learning technique i.e. self-organizing maps. The fault classification results show that the support vector machine identified the fault categories of rolling element bearing more accurately and has a better diagnosis performance as compared to the learning vector quantization and self-organizing maps.     

Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset

Identification of attacks by a network intrusion detection system (NIDS) is an important task. In signature or rule based detection, the previously encountered attacks are modeled, and signatures/rules are extracted. These rules are used to detect such attacks in future, but in anomaly or outlier detection system, the normal network traffic is modeled. Any deviation from the normal model is deemed to be an outlier/ attack. Data mining and machine learning techniques are widely used in offline NIDS. Unsupervised and supervised learning techniques differ the way NIDS dataset is treated. The characteristic features of unsupervised and supervised learning are finding patterns in data, detecting outliers, and determining a learned function for input features, generalizing the data instances respectively. The intuition is that if these two techniques are combined, better performance may be obtained. Hence, in this paper the advantages of unsupervised and supervised techniques are inherited in the proposed hierarchical model and devised into three stages to detect attacks in NIDS dataset. NIDS dataset is clustered using Dirichlet process (DP) clustering based on the underlying data distribution. Iteratively on each cluster, local denser areas are identified using local outlier factor (LOF) which in turn is discretized into four bins of separation based on LOF score. Further, in each bin the normal data instances are modeled using one class classifier (OCC). A combination of Density Estimation method, Reconstruction method, and Boundary methods are used for OCC model. A product rule combination of the threemethods takes into consideration the strengths of each method in building a stronger OCC model. Any deviation from this model is considered as an attack. Experiments are conducted on KDD CUP’99 and SSENet-2011 datasets. The results show that the proposed model is able to identify attacks with higher detection rate and low false alarms.     

Adaptive Default Hierarchy Formation

Autonomous systems are likely to be required to face situations that cannot be foreseen by their designers. The potential for perpetually novel situations places a premium on mechanisms that allow for automatic adaptation in a general setting. The term reinforcement learning problems (Mendel and McLaren, 1970) generally describes problems where a control system must adapt based on performance-only feedback. This paper considers the learning classifier system (LCS) as an approach to reinforcement learning problems. An LCS is a type of adaptive expert system that uses a knowledge base of production rules in a low-level syntax that can be manipulated by a genetic algorithm (GA) (Holland. 1975; Goldberg, 1989) Genetic algorithms comprise a class of computerized search procedures that are based on the mechanics of natural genetics (Goldberg, 1989; Holland. 1975). An important feature of the LCS paradigm is the possible adaptive formation of default hierarchies (layered sets of default and exception rules) )Holland et al., 1986). This paper examines the problem of default hierarchy formation under the conventional bid-competition method of LCS conflict resolution, and suggests the necessity auction and a separate priority factor as modifications to this method. Simulations show the utility of this method. Final discussion presents conclusions and suggests avenues for further research     

Learning relational structures: Applications in computer vision

We present and compare two new techniques for learning Relational Structures (RSs) as they occur in 2D pattern and 3D object recognition. These techniques, namely, Evidence-Based Networks (EBS-NNets) and Rulegraphs combine techniques from computer vision with those from machine learning and graph matching. The EBS-NNet has the ability to generalize pattern rules from training instances in terms of bounds on both unary (single part) and binary (part relation) numerical features. It also learns the compatibilities between unary and binary feature states in defining different pattern classes. Rulegraphs check this compatibility between unary and binary rules by combining evidence theory with graph theory. The two systems are tested and compared using a number of different pattern and object recognition problems.     

Automated,highly-accurate,bug assignment using machine learning and tossing graphs

Empirical studies indicate that automating the bug assignment process has the potential to significantly reduce software evolution effort and costs. Prior work has used machine learning techniques to automate bug assignment but has employed a narrow band of tools which can be ineffective in large, long-lived software projects. To redress this situation, in this paper we employ a comprehensive set of machine learning tools and a probabilistic graph-based model (bug tossing graphs) that lead to highly-accurate predictions, and lay the foundation for the next generation of machine learning-based bug assignment. Our work is the first to examine the impact of multiple machine learning dimensions (classifiers, attributes, and training history) along with bug tossing graphs on prediction accuracy in bug assignment. We validate our approach on Mozilla and Eclipse, covering 856,259 bug reports and 21 cumulative years of development. We demonstrate that our techniques can achieve up to 86.09% prediction accuracy in bug assignment and significantly reduce tossing path lengths. We show that for our data sets the Naïve Bayes classifier coupled with product–component features, tossing graphs and incremental learning performs best. Next, we perform an ablative analysis by unilaterally varying classifiers, features, and learning model to show their relative importance of on bug assignment accuracy. Finally, we propose optimization techniques that achieve high prediction accuracy while reducing training and prediction time.     

INFGMN – Incremental Neuro-Fuzzy Gaussian mixture network

Accuracy and interpretability are contradictory objectives that conflict in all machine learning techniques and achieving a satisfactory balance between these two criteria is a major challenge. The objective is not only to maximize interpretability, but also to guarantee a high degree of accuracy. This challenge is even greater when it is considered that the model will have to evolve and adapt itself to the dynamics of the underlying environment, i.e. it will have to learn incrementally. Little research has been published about incremental learning using Mamdani–Larsen (ML) fuzzy models under these conditions. This article presents a novel proposal for a Neuro-Fuzzy System (NFS) with an incremental learning capability, the Incremental Neuro-Fuzzy Gaussian Mixture Network (INFGMN), that attempts to generate incremental models that are highly interpretable and precise. The principal characteristics of the INFGMN are as follows: (i) the INFGMN learns incrementally using a single sweep of the training data (each training pattern can be immediately used and discarded); (ii) it is capable of producing reasonable estimates based on few training data; (iii) the learning process can proceed in perpetuity as new training data become available (learning and recalling phases are not separate); (iv) the INFGMN can deal with the Stability-Plasticity dilemma and is unaffected by catastrophic interference (rules are added or removed whenever necessary); (v) the fuzzy rule base is defined automatically and incrementally (new rules are added whenever necessary); and (vi) the INFGMN maintains an ML-type fuzzy rule base that attempts to provide the best trade-off between accuracy and interpretability, thereby dealing with the Accuracy-Interpretability dilemma. The INFGMNs performance in terms of learning and modelling is assessed using a variety of benchmark applications and the results are promising.     

Reducing the number of parameters of a fuzzy system using scaling functions

Learning techniques are tailored for fuzzy systems in order to tune them or even for deriving fuzzy rules from data. However, a compromise between accuracy and interpretability has to be found. Flexible fuzzy systems with a large number of parameters and high degrees of freedom tend to function as black boxes. In this paper, we introduce an interpretation of fuzzy systems that enables us to work with a small number of parameters without loosing flexibility or interpretability. In this way, we can provide a learning algorithm that is efficient and yields accuracy as well as interpretability. Our fuzzy system is based on extremely simple fuzzy sets and transformations using interpretable scaling functions of the input variables.     

知识图谱可解释推理研究综述

面向知识图谱的知识推理旨在通过已有的知识图谱事实,去推断新的事实,进而实现知识库的补全.近年来,尽管基于分布式表示学习的方法在推理任务上取得了巨大的成功,但是他们的黑盒属性使得模型无法为预测出的事实做出解释.所以,如何设计用户可理解、可信赖的推理模型成为了人们关注的问题.从可解释性的基本概念出发,系统梳理了面向知识图谱的可解释知识推理的相关工作,具体介绍了事前可解释推理模型和事后可解释推理模型的研究进展;根据可解释范围的大小,将事前可解释推理模型进一步细分为全局可解释的推理和局部可解释的推理;在事后解释模型中,回顾了推理模型的代表方法,并详细介绍提供事后解释的两类解释方法.此外,还总结了可解释知识推理在医疗、金融领域的应用.随后,对可解释知识推理的现状进行概述,最后展望了可解释知识推理的未来发展方向,以期进一步推动可解释推理的发展和应用.