Genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this paper, we propose and investigate a new category of neurofuzzy networks—fuzzy polynomial neural networks (FPNN) endowed with fuzzy set-based polynomial neurons (FSPNs) We develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms (GAs) in particular. The conventional FPNNs developed so far are based on the mechanisms of self-organization, fuzzy neurocomputing, and evolutionary optimization. The design of the network exploits the FSPNs as well as the extended group method of data handling (GMDH). Let us stress that in the previous development strategies some essential parameters of the networks (such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables) being available within the network are provided by the designer in advance and kept fixed throughout the overall development process. This restriction may hamper a possibility of developing an optimal architecture of the model. The design proposed in this study addresses this issue. The augmented and genetically developed FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional FPNNs. The GA-based design procedure being applied at each layer of the FPNN leads to the selection of the most suitable nodes (or FSPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation in which we use a number of modeling benchmarks—synthetic and experimental data being commonly used in fuzzy or neurofuzzy modeling. The obtained results demonstrate the superiority of the proposed networks over the models existing in the references.     

Granular Neural Networks and Their Development Through Context-Based Clustering and Adjustable Dimensionality of Receptive Fields

In this study, we present a new architecture of a granular neural network and provide a comprehensive design methodology as well as elaborate on an algorithmic setup supporting its development. The proposed neural network relates to a broad category of radial basis function neural networks (RBFNNs) in the sense that its topology involves a collection of receptive fields. In contrast to the standard architectures encountered in RBFNNs, here we form individual receptive fields in subspaces of the original input space rather than in the entire input space. These subspaces could be different for different receptive fields. The architecture of the network is fully reflective of the structure encountered in the training data which are granulated with the aid of clustering techniques. More specifically, the output space is granulated with use of K-means clustering while the information granules in the multidimensional input space are formed by using the so-called context-based fuzzy C-means, which takes into account the structure being already formed in the output space. The innovative development facet of the network involves a dynamic reduction of dimensionality of the input space in which the information granules are formed in the subspace of the overall input space which is formed by selecting a suitable subset of input variables so that this subspace retains the structure of the entire space. As this search is of combinatorial character, we use the technique of genetic optimization [genetic algorithms (GAs), to be more specific] to determine the optimal input subspaces. A series of numeric studies exploiting synthetic data and data coming from the Machine Learning Repository, University of California at Irvine, provide a detailed insight into the nature of the algorithm and its parameters as well as offer some comparative analysis.     

Hybrid fuzzy set-based polynomial neural networks and their development with the aid of genetic optimization and information granulation

We introduce a new architecture of feed-forward neural networks called hybrid fuzzy set-based polynomial neural networks (HFSPNNs) that are composed of heterogeneous feed-forward neural networks such as polynomial neural networks (PNNs) and fuzzy set-based polynomial neural networks (FSPNNs). We develop their comprehensive design methodology by embracing mechanisms of genetic optimization and information granulation. The construction of information granulation-driven HFSPNN exploits fundamental technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms (GAs). The architecture of the resulting information granulation-driven genetically optimized HFSPNN results from a synergistic usage of the hybrid system generated by combining original fuzzy set-based polynomial neurons (FSPNs)-based FSPNN with polynomial neurons (PNs)-based PNN. The design of the conventional genetically optimized HFPNN exploits the extended Group Method of Data Handling (GMDH) whose some essential parameters of the network being tuned with the use of genetic algorithms throughout the overall development process. Two general optimization mechanisms are explored. First, the structural optimization is realized via GAs while the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gHFSPNN is quantified through extensive experimentation where we considered a number of modeling benchmarks (synthetic and experimental data already experimented with in fuzzy or neurofuzzy modeling).     

A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering

In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP²N²), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs).The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models.We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space.We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling.     

Design of optimized fuzzy cascade controllers by means of Hierarchical Fair Competition-based Genetic Algorithms

In this study, we present a design of an optimized fuzzy cascade controller based on Hierarchical Fair Competition-based Genetic Algorithms (HFCGA) for a rotary inverted pendulum system. In this system, one controls the movement of a pendulum through the adjustment of a rotating arm. The objective is to control the position of the rotating arm and to make the pendulum maintain the unstable equilibrium point at vertical position. To control the system, we design a fuzzy cascade controller scheme which consists of two fuzzy controllers arrange in a cascaded topology. The parameters of the controller are optimized by means of the HFCGA algorithm. The fuzzy cascade scheme comprises two controllers located in two loops. An inner loop controller governs the position of the rotating arm while an outer controller modifies a set point of the inner controller implied by the changes of the angle of pendulum. The HFCGA being a computationally effective scheme of the Parallel Genetic Algorithm (PGA) has been developed to eliminate an effect of premature convergence encountered in Serial Genetic Algorithms (SGA). It has emerged as an effective optimization vehicle to deal with very large search spaces. A comparative analysis involving computing simulations and practical experiment demonstrates that the proposed HFCGA based fuzzy cascade controller comes with superb performance in comparison with the conventional Linear Quadratic Regulator (LQR) controller as well as HFCGA-based PD cascade controller.     

Fuzzy polynomial neural networks: hybrid architectures of fuzzy modeling

We introduce a concept of fuzzy polynomial neural networks (FPNNs), a hybrid modeling architecture combining polynomial neural networks (PNNs) and fuzzy neural networks (FNNs). The development of the FPNNs dwells on the technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms. The structure of the FPNN results from a synergistic usage of FNN and PNN. FNNs contribute to the formation of the premise part of the rule-based structure of the FPNN. The consequence part of the FPNN is designed using PNNs. The structure of the PNN is not fixed in advance as it usually takes place in the case of conventional neural networks, but becomes organized dynamically to meet the required approximation error. We exploit a group method of data handling (GMDH) to produce this dynamic topology of the network. The performance of the FPNN is quantified through experimentation that exploits standard data already used in fuzzy modeling. The obtained experimental results reveal that the proposed networks exhibit high accuracy and generalization capabilities in comparison to other similar fuzzy models.     

The Design of Genetically Optimized Self-Organizing Neural Networks with Polynomial and Fuzzy Polynomial Neurons

In this study, we introduce and investigate a class of neural architectures of self-organizing neural networks (SONNs) that is based on a genetically optimized multilayer perceptron with polynomial neurons (PNs) or fuzzy polynomial neurons (FPNs), develop a comprehensive design methodology involving mechanisms of genetic optimization, and carry out a series of numeric experiments. We distinguish between two kinds of SONN architectures: (a) PN-based and (b) FPN-based SONNs. The augmented genetically optimized SONN (gSONN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one encountered in the conventional SONN. The genetic algorithm (GA)-based design procedure being applied at each layer of SONN leads to the selection of preferred nodes (PNs or FPNs) with specific local characteristics (such as the number of input variables, the order of the polynomial, and a collection of the specific subset of input variables) available within the network.     

Fuzzy set-oriented neural networks based on fuzzy polynomial inference and dynamic genetic optimization

In this paper, we introduce a new topology and offer a comprehensive design methodology of fuzzy set-based neural networks (FsNNs). The proposed architecture of the FsNNs is based on the fuzzy polynomial neurons formed through a collection of ‘if-then’ fuzzy rules, fuzzy inference, and polynomials with the extended structure of the premise and the consequence parts of fuzzy rules being formed within the networks. Three different forms of regression polynomials (namely constant, linear, and quadratic) are used in the consequence part of the rules. In order to build an optimal FsNN, the underlying structural and parametric optimization is supported by a dynamic search-based genetic algorithm (GA), which forms an optimal solution through successive adjustments (refinements) of the search range. The structure optimization involves the determination of the input variables included in the premise part and the order of the polynomial forming the consequence part of the rules. In the study, we explore two types of optimization methodologies, namely a simultaneous tuning and a separate tuning. GAs are global optimizers; however, when being used in their generic version, they often lead to a significant computing overhead caused by the need to explore an excessively large search space. To eliminate this shortcoming and increase the effectiveness of the optimization itself, we introduce a dynamic search-based GA that results in a rapid convergence while narrowing down the search to a limited region of the search space. We exploit this optimization mechanism to be completed both at the structural as well as the parametric level. To evaluate the performance of the proposed FsNN, we offer a suite of several representative numerical examples.     

Polynomial-based radial basis function neural networks (P-RBF NNs) and their application to pattern classification

Polynomial neural networks have been known to exhibit useful properties as classifiers and universal approximators. In this study, we introduce a concept of polynomial-based radial basis function neural networks (P-RBF NNs), present a design methodology and show the use of the networks in classification problems. From the conceptual standpoint, the classifiers of this form can be expressed as a collection of “if-then” rules. The proposed architecture uses two essential development mechanisms. Fuzzy clustering (Fuzzy C-Means, FCM) is aimed at the development of condition parts of the rules while the corresponding conclusions of the rules are formed by some polynomials. A detailed learning algorithm for the P-RBF NNs is developed. The proposed classifier is applied to two-class pattern classification problems. The performance of this classifier is contrasted with the results produced by the “standard” RBF neural networks. In addition, the experimental application covers a comparative analysis including several previous commonly encountered methods such as standard neural networks, SVM, SOM, PCA, LDA, C4.5, and decision trees. The experimental results reveal that the proposed approach comes with a simpler structure of the classifier and better prediction capabilities.     

Polynomial neural networks architecture: analysis and design

In this study, we introduce and investigate a class of neural architectures of Polynomial Neural Networks (PNNs), discuss a comprehensive design methodology and carry out a series of numeric experiments. Two kinds of PNN architectures, namely a basic PNN and a modified PNN architecture are discussed. Each of them comes with two types such as the generic and the advanced type. The essence of the design procedure dwells on the Group Method of Data Handling. PNN is a flexible neural architecture whose structure is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but becomes dynamically meaning that the network grows over the training period. In this sense, PNN is a self-organizing network. A comparative analysis shows that the proposed PNN are models with higher accuracy than other fuzzy models.