Extracting Interpretable Fuzzy Rules from RBF Networks

Radial basis function networks and fuzzy rule systems are functionally equivalent under some mild conditions. Therefore, the learning algorithms developed in the field of artificial neural networks can be used to adapt the parameters of fuzzy systems. Unfortunately, after the neural network learning, the structure of the original fuzzy system is changed and interpretability, which is considered to be one of the most important features of fuzzy systems, is usually impaired. This Letter discusses the differences between RBF networks and interpretable fuzzy systems. Based on these discussions, a method for extracting interpretable fuzzy rules from RBF networks is suggested. Simulation examples are given to embody the idea of this paper.     

Structure optimization of neural networks for evolutionary design optimization

We study the use of neural networks as approximate models for the fitness evaluation in evolutionary design optimization. To improve the quality of the neural network models, structure optimization of these networks is performed with respect to two different criteria: One is the commonly used approximation error with respect to all available data, and the other is the ability of the networks to learn different problems of a common class of problems fast and with high accuracy. Simulation results from turbine blade optimizations using the structurally optimized neural network models are presented to show that the performance of the models can be improved significantly through structure optimization.We would like to thank the BMBF, grant LOKI, number 01 IB 001 C, for their financial support of our research.     

Knowledge Incorporation into Neural Networks From Fuzzy Rules

The incorporation of prior knowledge into neural networks can improve neural network learning in several respects, for example, a faster learning speed and better generalization ability. However, neural network learning is data driven and there is no general way to exploit knowledge which is not in the form of data input-output pairs. In this paper, we propose two approaches for incorporating knowledge into neural networks from fuzzy rules. These fuzzy rules are generated based on expert knowledge or intuition. In the first approach, information from the derivative of the fuzzy system is used to regularize the neural network learning, whereas in the second approach the fuzzy rules are used as a catalyst. Simulation studies show that both approaches increase the learning speed significantly.     

A Model for the Dynamic Interaction Between Evolution and Learning

The interaction between learning and evolution has elicited much interest particularly among researchers who use evolutionary algorithms for the optimization of neural structures. In this article, we will propose an extension of the existing models by including a developmental phase – a growth process – of the neural network. In this way, we are able to examine the dynamical interaction between genetic information and information learned during development. Several measures are proposed to quantitatively examine the benefits and the effects of such an overlap between learning and evolution. The proposed model, which is based on the recursive encoding method for structure optimization of neural networks, is applied to the problem domain of time series prediction. Furthermore, comments are made on problem domains which associate growing networks (size) during development with problems of increasing complexity.     

Preference representation using Gaussian functions on a hyperplane in evolutionary multi-objective optimization

Many-objective optimization has attracted much attention in evolutionary multi-objective optimization (EMO). This is because EMO algorithms developed so far often degrade their search ability for optimization problems with four or more objectives, which are frequently referred to as many-objective problems. One of promising approaches to handle many objectives is to incorporate the preference of a decision maker (DM) into EMO algorithms. With the preference, EMO algorithms can focus the search on regions preferred by the DM, resulting in solutions close to the Pareto front around the preferred regions. Although a number of preference-based EMO algorithms have been proposed, it is not trivial for the DM to reflect his/her actual preference in the search. We previously proposed to represent the preference of the DM using Gaussian functions on a hyperplane. The DM specifies the center and spread vectors of the Gaussian functions so as to represent his/her preference. The preference handling is integrated into the framework of NSGA-II. This paper extends our previous work so that obtained solutions follow the distribution of Gaussian functions specified. The performance of our proposed method is demonstrated mainly for benchmark problems and real-world applications with a few objectives in this paper. We also show the applicability of our method to many-objective problems.     

An examination of different fitness and novelty based selection methods for the evolution of neural networks

It has been suggested recently that it is a reasonable abstraction of evolutionary processes to use evolutionary algorithms that select individuals based on the novelty of their behavior instead of their fitness. Here we study the performance of fitness- and novelty-based search on several neuroevolution tasks. We also propose several new algorithms that select both for fit and for novel individuals, but without weighting these two criteria directly against each other. We find that behavioral speciation, behavioral near neutral speciation, and behavioral novelty speciation perform best on most tasks. Pure novelty search, as well as a number of hybrid methods without speciation mechanism, do not perform well on most tasks. Using behavioral criteria for speciation often yields better results than using genetic criteria.     

Functions with noise-induced multimodality: a test for evolutionary robust Optimization-properties and performance analysis

This paper proposes and analyzes a class of test functions for evolutionary robust optimization, the "functions with noise-induced multimodality" (FNIMs). After a motivational introduction gleaned from a real-world optimization problem, the robust optimizer properties of this test class are investigated with respect to different robustness measures. The steady-state behavior of evolution strategies on FNIMs will be investigated empirically. Being based on the empirical results, a subclass of FNIMs is identified which is amenable to an asymptotical performance analysis. The results of this analysis will be used to derive recommendations for the choice of strategy-specific parameters such as population size and truncation ratio.     

A systems approach to evolutionary multiobjective structural optimization and beyond

Multiobjective evolutionary algorithms (MOEAs) have shown to be effective in solving a wide range of test problems. However, it is not straightforward to apply MOEAs to complex real-world problems. This paper discusses the major challenges we face in applying MOEAs to complex structural optimization, including the involvement of time-consuming and multi-disciplinary quality evaluation processes, changing environments, vagueness in formulating criteria formulation, and the involvement of multiple sub-systems. We propose that the successful tackling of all these aspects give birth to a systems approach to evolutionary design optimization characterized by considerations at four levels, namely, the system property level, temporal level, spatial level and process level. Finally, we suggest a few promising future research topics in evolutionary structural design that consist in the necessary steps towards a life-like design approach, where design principles found in biological systems such as self-organization, self-repair and scalability play a central role.     

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.