Generalization in the XCSF classifier system: analysis, improvement, and extension

We analyze generalization in XCSF and introduce three improvements. We begin by showing that the types of generalizations evolved by XCSF can be influenced by the input range. To explain these results we present a theoretical analysis of the convergence of classifier weights in XCSF which highlights a broader issue. In XCSF, because of the mathematical properties of the Widrow-Hoff update, the convergence of classifier weights in a given subspace can be slow when the spread of the eigenvalues of the autocorrelation matrix associated with each classifier is large. As a major consequence, the system's accuracy pressure may act before classifier weights are adequately updated, so that XCSF may evolve piecewise constant approximations, instead of the intended, and more efficient, piecewise linear ones. We propose three different ways to update classifier weights in XCSF so as to increase the generalization capabilities of XCSF: one based on a condition-based normalization of the inputs, one based on linear least squares, and one based on the recursive version of linear least squares. Through a series of experiments we show that while all three approaches significantly improve XCSF, least squares approaches appear to be best performing and most robust. Finally we show how XCSF can be extended to include polynomial approximations.     

Comparing extended classifier system and genetic programming for financial forecasting: an empirical study

As a broad subfield of artificial intelligence, machine learning is concerned with the development of algorithms and techniques that allow computers to learn. These methods such as fuzzy logic, neural networks, support vector machines, decision trees and Bayesian learning have been applied to learn meaningful rules; however, the only drawback of these methods is that it often gets trapped into a local optimal. In contrast with machine learning methods, a genetic algorithm (GA) is guaranteeing for acquiring better results based on its natural evolution and global searching. GA has given rise to two new fields of research where global optimization is of crucial importance: genetic based machine learning (GBML) and genetic programming (GP). This article adopts the GBML technique to provide a three-phase knowledge extraction methodology, which makes continues and instant learning while integrates multiple rule sets into a centralized knowledge base. Moreover, the proposed system and GP are both applied to the theoretical and empirical experiments. Results for both approaches are presented and compared. This paper makes two important contributions: (1) it uses three criteria (accuracy, coverage, and fitness) to apply the knowledge extraction process which is very effective in selecting an optimal set of rules from a large population; (2) the experiments prove that the rule sets derived by the proposed approach are more accurate than GP.     

基于表格查寻学习算法的自适应模糊分类器

以手写体数字识别问题为背景,提出了一种基于表格查寻学习算法的自适应模糊分类 器,并用Matlab给出了自适应模糊分类器的实现,进而对其进行了仿真。仿真结果表明,该自适应模 糊分类器在手写体数字识别的识别性能、利用语言信息、计算复杂性等方面均优于采用BP算法的三 层前馈分类器,体现了自适应模糊处理技术用于模式识别的优越性和潜力。     

Designing a classifier by a layered multi-population genetic programming approach

This paper proposes a method called layered genetic programming (LAGEP) to construct a classifier based on multi-population genetic programming (MGP). LAGEP employs layer architecture to arrange multiple populations. A layer is composed of a number of populations. The results of populations are discriminant functions. These functions transform the training set to construct a new training set. The successive layer uses the new training set to obtain better discriminant functions. Moreover, because the functions generated by each layer will be composed to a long discriminant function, which is the result of LAGEP, every layer can evolve with short individuals. For each population, we propose an adaptive mutation rate tuning method to increase the mutation rate based on fitness values and remaining generations. Several experiments are conducted with different settings of LAGEP and several real-world medical problems. Experiment results show that LAGEP achieves comparable accuracy to single population GP in much less time.     

A linguistic information feed-back-based dynamical fuzzy system (LIFBDFS) with learning algorithm

In this study, the linguistic information feed-back-based dynamical fuzzy system (LIFBDFS) proposed earlier by the authors is first introduced. The principles of α-level sets and backpropagation through time approach are also briefly discussed. We next employ these two methods to derive an explicit learning algorithm for the feedback parameters of the LIFBDFS. With this training algorithm, our LIFBDFS indeed becomes a potential candidate in solving real-time modeling and prediction problems.     

Improving the performance of the BioHEL learning classifier system

The identification of significant attributes is of major importance to the performance of a variety of Learning Classifier Systems including the newly-emerged Bioinformatics-oriented Hierarchical Evolutionary Learning (BioHEL) algorithm. However, the BioHEL fails to deliver on a set of synthetic datasets which are the checkerboard data mixed with Gaussian noises due to the fact the significant attributes were not successfully recognised. To address this issue, a univariate Estimation of Distribution Algorithm (EDA) technique is introduced to BioHEL which primarily builds a probabilistic model upon the outcome of the generalization and specialization operations. The probabilistic model which estimates the significance of each attribute provides guidance for the exploration of the problem space. Experiment evaluations showed that the proposed BioHEL systems achieved comparable performance to the conventional one on a number of real-world small-scale datasets. Research efforts were also made on finding the optimal parameter for the traditional and proposed BioHEL systems.     

A faster convergence and concise interpretability TSK fuzzy classifier deep-wide-based integrated learning

Hierarchical TSK fuzzy system was proposed to approach the exponential growth of IF-THEN rules which named “fuzzy rule explosion”. However, it could not get better performance in few layers for instability of TSK fuzzy system, such that hierarchical TSK fuzzy system suffers from bad interpretability and slow convergence along with too much layers. To get a better solution, this study employs a faster convergence and concise interpretability TSK fuzzy classifier deep-wide-based integrated learning (FCCI-TSK) which has a wide structure to adopt several ensemble units learning in a meantime, and the best performer will be picked up to transfer its learning knowledge to next layer with the help of stacked generalization principle. The ensemble units are integrated by negative correlation learning (NCL). FCCI-TSK adjusts the input of the next layer with a better guidance such that it can quicken the speed of convergence and reduce the number of layers. Besides, leading with guidance, it can achieve higher accuracy and better interpretability with more simple structure. The contributions of this study include: (1) To enhance the performance of fuzzy classifier, we mix NCL and stacked generalization principle together in FCCI-TSK; (2) To overcome the phenomenon of “fuzzy rule explosion”, we adopt deep-wide integrated learning and information discarding to accelerate convergence and obtain concise interpretability in the meantime. Comparing with other 11 algorithms, the results on twelve UCI datasets show that FCCI-TSK has the best performance overall and the convergence of FCCI-TSK is also examined.     

Islanding detection of distributed generation by using multi-gene genetic programming based classifier

This paper proposed a new method for detecting islanding of distributed generation (DG), using Multi-gene Genetic Programming (MGP). Islanding has been a serious concern among power distribution utilities and distributed generation owners, because it poses risks to the safety of utilities’ workers and consumers, and can cause damage to power distribution systems’ equipment. Therefore, a DG must be disconnected as soon as an islanding is detected. In addition, an islanding detection method must have high degree of dependability to correctly discriminate islanding from other events, such as load switching, in order to avoid unnecessary disconnection of the distributed generator. In this context, the novelty of the proposed method is that the MGP is capable of obtaining a set of mathematical and logic functions employed to detect and classify islanding correctly. This is a new approach among the computational intelligent methods proposed for DG islanding detection. The main idea was to use local voltage measurements as input of the method, eliminating the need of complex and expensive communication infrastructure. The method has been trained with several islanding and non-islanding cases, by using a power distribution system comprising five concentrated loads, a synchronous distributed generator and a wind power plant. The results showed that the proposed method was successful in differentiating the islanding events from other disturbances, revealing its great potential to be applied in anti-islanding protection schemes for distributed generation.     

GOFAM: a hybrid neural network classifier combining fuzzy ARTMAP and genetic algorithm

Fuzzy ARTMAP (FAM), which is a supervised model from the adaptive resonance theory (ART) neural network family, is one of the conspicuous neural network classifier. The generalization/performance of FAM is affected by two important factors which are network parameters and presentation order of training data. In this paper we introduce a genetic algorithm to find a better presentation order of training data for FAM. The proposed method which is the combination of genetic algorithm with Fuzzy ARTMAP is called Genetic Ordered Fuzzy ARTMAP (GOFAM). To illustrate the effectiveness of GOFAM, several standard datasets from UCI repository of machine learning databases are experimented. The results are analyzed and compared with those from FAM and Ordered FAM which is used to determine a fixed order of training pattern presentation to FAM. Experimental results demonstrate the performance of GOFAM is much better than performance of Fuzzy ARTMAP and Ordered Fuzzy ARTMAP. In term of network size, GOFAM performs significantly better than FAM and Ordered FAM.     

基于平均奖赏强化学习算法的零阶分类元系统

零阶学习分类元系统ZCS（Zeroth-level Classifier System）作为一种基于遗传的机器学习技术（Genetics-Based Machine Learning）,在解决多步学习问题上,已展现出应用价值。然而标准的ZCS系统采用折扣奖赏强化学习技术,难于适应更为广泛的应用领域。基于ZCS的现有框架,提出了一种采用平均奖赏强化学习技术（R-学习算法）的分类元系统,将ZCS中的折扣奖赏强化学习方法替换为R-学习算法,从而使ZCS一方面可应用于需要优化平均奖赏的问题领域,另一方面则可求解规模较大、需要动作长链支持的多步学习问题。实验显示,在多步学习问题中,该系统可给出满意解,且在维持动作长链,以及克服过泛化问题方面,具有更优的特性。     

基于遗传规划和集成学习的恶意软件检测

近年来恶意软件不断地发展变化,导致单一检测模型的准确率较低,使用集成学习组合多种模型可以提高检测效果,但集成模型中基学习器的准确性和多样性难以平衡。为此,提出一种基于遗传规划的集成模型生成方法,遗传规划可以将特征处理和构建集成模型两个阶段集成到单个程序树中,解决了传统恶意软件集成检测模型难以平衡个体准确率和多样性的问题。该方法以集成模型的恶意软件检出率作为种群进化依据,保证了基学习器的准确性;在构建集成模型时自动选择特征处理方法、分类算法和优化基学习器的超参数,通过输入属性扰动和算法参数扰动增加基学习器的多样性,根据优胜劣汰的思想进化生成具有高准确性和多样性的最优集成模型。在EMBER数据集上的结果表明,最优集成模型的检测准确率达到了98.88%;进一步的分析表明,该方法生成的模型具有较高的多样性和可解释性。     

A genetic fuzzy expert system for automatic question classification in a competitive learning environment

Intelligent tutoring systems are efficient tools to automatically adapt the learning process to the student’s progress and needs. One of the possible adaptations is to apply an adaptive question sequencing system, which matches the difficulty of the questions to the student’s knowledge level. In this context, it is important to correctly classify the questions to be presented to students according to their difficulty level. Many systems have been developed for estimating the difficulty of questions. However the variety in the application environments makes difficult to apply the existing solutions directly to other applications. Therefore, a specific solution has been designed in order to determine the difficulty level of open questions in an automatic and objective way. This solution can be applied to activities with special temporal and running features, as the contests developed through QUESTOURnament, which is a tool integrated into the e-learning platform Moodle. The proposed solution is a fuzzy expert system that uses a genetic algorithm in order to characterize each difficulty level. From the output of the algorithm, it defines the fuzzy rules that are used to classify the questions. Data registered from a competitive activity in a Telecommunications Engineering course have been used in order to validate the system against a group of experts. Results show that the system performs successfully. Therefore, it can be concluded that the system is able to do the questions classification labour in a competitive learning environment.     

A multistage genetic fuzzy classifier for land cover classification from satellite imagery

A linguistic boosted genetic fuzzy classifier (LiBGFC) is proposed in this paper for land cover classification from multispectral images. The LiBGFC is a three-stage process, aiming at effectively tackling the interpretability versus accuracy tradeoff problem. The first stage iteratively generates fuzzy rules, as directed by a boosting algorithm that localizes new rules in uncovered subspaces of the feature space, implicitly preserving the cooperation with previously derived ones. Each rule is able to select the required features, further improving the interpretability of the obtained model. Special provision is taken in the formulation of the fitness function to avoid the creation of redundant rules. A simplification stage follows the first one aiming at further improving the interpretability of the initial rule base, providing a more compact and interpretable solution. Finally, a genetic tuning stage fine tunes the fuzzy sets database improving the classification performance of the obtained model. The LiBGFC is tested using an IKONOS multispectral VHR image, in a lake-wetland ecosystem of international importance. The results indicate the effectiveness of the proposed system in handling multidimensional feature spaces, producing easily understandable fuzzy models.     

A neural fuzzy system with fuzzy supervised learning

A neural fuzzy system learning with fuzzy training data (fuzzy if-then rules) is proposed in this paper. This system is able to process and learn numerical information as well as linguistic information. At first, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use alpha-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, a fuzzy supervised learning algorithm is developed for the proposed system. It extends the normal supervised learning techniques to the learning problems where only linguistic teaching signals are available. The fuzzy supervised learning scheme can train the proposed system with desired fuzzy input-output pairs which are fuzzy numbers instead of the normal numerical values. With fuzzy supervised learning, the proposed system can be used for rule base concentration to reduce the number of rules in a fuzzy rule base. Simulation results are presented to illustrate the performance and applicability of the proposed system.     

FuGeNeSys-a fuzzy genetic neural system for fuzzy modeling

The author has developed a novel approach to fuzzy modeling from input-output data. Using the basic techniques of soft computing, the method allows supervised approximation of multi-input multi-output (MIMO) systems. Typically, a small number of rules are produced. The learning capacity of FuGeNeSys is considerable, as is shown by the numerous applications developed. The paper gives a significant example of how the fuzzy models developed are generally better than those to be found in literature as concerns simplicity and both approximation and classification capabilities     

Available-to-promise based bidding decision by fuzzy mathematical programming and genetic algorithm

The present study attempts to integrate bidding decisions with order promising and production planning to enhance supplier profitability and service level. This study formulates the bid price and production plan as a mixed integer programming model with fuzzy constraints. The fuzzy constraints represent the decision-maker’s subjective judgment regarding the customer’s price tolerance. The proposed model combines the advanced available-to-promise (AATP) concept to find optimum resource allocation and enable accurate estimations of production costs and delivery dates. The proposed solution procedure determines the optimum bid price by striking a compromise between profitability and the possibility to win the contract. This study develops a genetic algorithm to solve this problem, and provides computer simulated experiments to evaluate the performance of the proposed approach.