IG-based genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this study, we introduce and investigate a new topology of fuzzy-neural networks—fuzzy polynomial neural networks (FPNN) that is based on a genetically optimized multiplayer perceptron with fuzzy set-based polynomial neurons (FSPNs). We also develop a comprehensive design methodology involving mechanisms of genetic optimization and information granulation. In the sequel, the genetically optimized FPNN (gFPNN) is formed with the use of fuzzy set-based polynomial neurons (FSPNs) composed of fuzzy set-based rules through the process of information granulation. This granulation is realized with the aid of the C-means clustering (C-Means). The design procedure applied in the construction of each layer of an FPNN deals with its structural optimization involving the selection of the most suitable nodes (or FSPNs) with specific local characteristics (such as the number of input variable, the order of the polynomial, the number of membership functions, and a collection of specific subset of input variables) and address main aspects of parametric optimization. Along this line, two general optimization mechanisms are explored. The structural optimization is realized via genetic algorithms (GAs) and HCM method whereas in case of the parametric optimization we proceed with a standard least square estimation (learning). Through the consecutive process of structural and parametric optimization, a flexible neural network is generated in a dynamic fashion. The performance of the designed networks is quantified through experimentation where we use two modeling benchmarks already commonly utilized within the area of fuzzy or neurofuzzy modeling.     

Genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this paper, we propose and investigate a new category of neurofuzzy networks—fuzzy polynomial neural networks (FPNN) endowed with fuzzy set-based polynomial neurons (FSPNs) We develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms (GAs) in particular. The conventional FPNNs developed so far are based on the mechanisms of self-organization, fuzzy neurocomputing, and evolutionary optimization. The design of the network exploits the FSPNs as well as the extended group method of data handling (GMDH). Let us stress that in the previous development strategies some essential parameters of the networks (such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables) being available within the network are provided by the designer in advance and kept fixed throughout the overall development process. This restriction may hamper a possibility of developing an optimal architecture of the model. The design proposed in this study addresses this issue. The augmented and genetically developed FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional FPNNs. The GA-based design procedure being applied at each layer of the FPNN leads to the selection of the most suitable nodes (or FSPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation in which we use a number of modeling benchmarks—synthetic and experimental data being commonly used in fuzzy or neurofuzzy modeling. The obtained results demonstrate the superiority of the proposed networks over the models existing in the references.     

Evolutionary design of hybrid self-organizing fuzzy polynomial neural networks with the aid of information granulation

We introduce a new architecture of information granulation-based and genetically optimized Hybrid Self-Organizing Fuzzy Polynomial Neural Networks (HSOFPNN). Such networks are based on genetically optimized multi-layer perceptrons. We develop their comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The architecture of the resulting HSOFPNN combines fuzzy polynomial neurons (FPNs) that are located at the first layer of the network with polynomial neurons (PNs) forming the remaining layers of the network. The augmented version of the HSOFPNN, “IG_gHSOFPNN”, for brief, embraces the concept of information granulation and subsequently exhibits higher level of flexibility and leads to simpler architectures and rapid convergence speed to optimal structure in comparison with the HSOFPNNs and SOFPNNs.

The GA-based design procedure being applied at each layer of HSOFPNN leads to the selection of preferred nodes of the network (FPNs or PNs) whose local characteristics (such as the number of input variables, the order of the polynomial, a collection of the specific subset of input variables, the number of membership functions for each input variable, and the type of membership function) can be easily adjusted. In the sequel, two general optimization mechanisms are explored. The structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is afterwards carried out in the setting of a standard least square method-based learning. The obtained results demonstrate a superiority of the proposed networks over the existing fuzzy and neural models.     

Fuzzy polynomial neural networks: hybrid architectures of fuzzy modeling

We introduce a concept of fuzzy polynomial neural networks (FPNNs), a hybrid modeling architecture combining polynomial neural networks (PNNs) and fuzzy neural networks (FNNs). The development of the FPNNs dwells on the technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms. The structure of the FPNN results from a synergistic usage of FNN and PNN. FNNs contribute to the formation of the premise part of the rule-based structure of the FPNN. The consequence part of the FPNN is designed using PNNs. The structure of the PNN is not fixed in advance as it usually takes place in the case of conventional neural networks, but becomes organized dynamically to meet the required approximation error. We exploit a group method of data handling (GMDH) to produce this dynamic topology of the network. The performance of the FPNN is quantified through experimentation that exploits standard data already used in fuzzy modeling. The obtained experimental results reveal that the proposed networks exhibit high accuracy and generalization capabilities in comparison to other similar fuzzy models.     

Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO

In this paper the optimization of type-2 fuzzy inference systems using genetic algorithms (GAs) and particle swarm optimization (PSO) is presented. The optimized type-2 fuzzy inference systems are used to estimate the type-2 fuzzy weights of backpropagation neural networks. Simulation results and a comparative study among neural networks with type-2 fuzzy weights without optimization of the type-2 fuzzy inference systems, neural networks with optimized type-2 fuzzy weights using genetic algorithms, and neural networks with optimized type-2 fuzzy weights using particle swarm optimization are presented to illustrate the advantages of the bio-inspired methods. The comparative study is based on a benchmark case of prediction, which is the Mackey-Glass time series (for τ = 17) problem.     

Fuzzy set-oriented neural networks based on fuzzy polynomial inference and dynamic genetic optimization

In this paper, we introduce a new topology and offer a comprehensive design methodology of fuzzy set-based neural networks (FsNNs). The proposed architecture of the FsNNs is based on the fuzzy polynomial neurons formed through a collection of ‘if-then’ fuzzy rules, fuzzy inference, and polynomials with the extended structure of the premise and the consequence parts of fuzzy rules being formed within the networks. Three different forms of regression polynomials (namely constant, linear, and quadratic) are used in the consequence part of the rules. In order to build an optimal FsNN, the underlying structural and parametric optimization is supported by a dynamic search-based genetic algorithm (GA), which forms an optimal solution through successive adjustments (refinements) of the search range. The structure optimization involves the determination of the input variables included in the premise part and the order of the polynomial forming the consequence part of the rules. In the study, we explore two types of optimization methodologies, namely a simultaneous tuning and a separate tuning. GAs are global optimizers; however, when being used in their generic version, they often lead to a significant computing overhead caused by the need to explore an excessively large search space. To eliminate this shortcoming and increase the effectiveness of the optimization itself, we introduce a dynamic search-based GA that results in a rapid convergence while narrowing down the search to a limited region of the search space. We exploit this optimization mechanism to be completed both at the structural as well as the parametric level. To evaluate the performance of the proposed FsNN, we offer a suite of several representative numerical examples.     

A new approach to radial basis function-based polynomial neural networks: analysis and design

In this study, we introduce a new topology of radial basis function-based polynomial neural networks (RPNNs) that is based on a genetically optimized multi-layer perceptron with radial polynomial neurons (RPNs). This paper offers a comprehensive design methodology involving various mechanisms of optimization, especially fuzzy C-means (FCM) clustering and particle swarm optimization (PSO). In contrast to the typical architectures encountered in polynomial neural networks (PNNs), our main objective is to develop a topology and establish a comprehensive design strategy of RPNNs: (a) The architecture of the proposed network consists of radial polynomial neurons (RPN). These neurons are fully reflective of the structure encountered in numeric data, which are granulated with the aid of FCM clustering. RPN dwells on the concepts of a collection of radial basis function and the function-based nonlinear polynomial processing. (b) The PSO-based design procedure being applied to each layer of the RPNN leads to the selection of preferred nodes of the network whose local parameters (such as the number of input variables, a collection of the specific subset of input variables, the order of the polynomial, the number of clusters of FCM clustering, and a fuzzification coefficient of the FCM method) are properly adjusted. The performance of the RPNN is quantified through a series of experiments where we use several modeling benchmarks, namely a synthetic three-dimensional data and learning machine data (computer hardware data, abalone data, MPG data, and Boston housing data) already used in neuro-fuzzy modeling. A comparative analysis shows that the proposed RPNN exhibits higher accuracy in comparison with some previous models available in the literature.     

A Design of Genetically Oriented Fuzzy Relation Neural Networks (FrNNs) Based on the Fuzzy Polynomial Inference Scheme

In this paper, we introduce new architectures of genetically oriented fuzzy relation neural networks (FrNNs) and offer a comprehensive design methodology that supports their development. The proposed FrNNs are based on “if–then”-rule-based networks, with the extended structure of the premise and the consequence parts of the individual rules. We consider two types of the FrNN topologies, which are called FrNN-I and FrNN-II here, depending upon the usage of inputs in the premise and the consequence of fuzzy rules. Three different forms of regression polynomials (namely, constant, linear, and quadratic) are used to construct the consequence of the rules. In order to develop optimal FrNNs, the structure and the parameters are optimized using genetic algorithms (GAs). The proposed methodology is compared when the two development strategies, with separate and simultaneous optimization schemes that involve structure and parameters, are carried out. Given the large search space associated with these FrNN models, we enhance the search capabilities of the GAs by introducing the dynamic variants of genetic optimization. It fully exploits the processing capabilities of the FrNNs by supporting their structural and parametric optimization. To evaluate the performance of the proposed FrNNs, we exploit a suite of several representative numerical examples. A comparative analysis shows that the FrNNs exhibit higher accuracy and predictive capabilities as well as better modeling stability, when compared with some other models that exist in the literature.      

Structural developments of fuzzy systems with the aid of information granulation

We introduce a design procedure for fuzzy systems using the concept of information granulation and genetic optimization. Information granulation and resulting information granules themselves become an important design aspect of fuzzy models. By accommodating the formalism of fuzzy sets, the model is geared towards capturing relationship between information granules (fuzzy sets) rather than concentrating on plain numeric data. Information granulation realized with the use of the standard C-Means clustering helps determine the initial values of the parameters of the fuzzy models. This in particular concerns such essential components of the rules as the initial apexes of the membership functions standing in the premise part of the fuzzy rules and the initial values of the polynomial functions standing in the consequence part. The initial parameters are afterwards tuned with the aid of the genetic algorithms (GAs) and the least square method (LSM). The overall design methodology arises as a hybrid development process involving structural and parametric optimization. Especially, genetic algorithms and C-Means are used to generate the structurally as well as parametrically optimized fuzzy model. To identify the structure and estimate parameters of the fuzzy model we exploit the methodologies such as joint and successive method realized by means of genetic algorithms. The proposed model is evaluated using experimental data and its performance is contrasted with the behavior of the fuzzy models available in the literature.     

Intelligent computing approach to solve the nonlinear Van der Pol system for heartbeat model

In this work, an intelligent computing algorithm is developed for finding the approximate solution of heart model based on nonlinear Van der Pol (VdP)-type second-order ordinary differential equations (ODEs) using feed-forward artificial neural networks (FF-ANNs) optimized with genetic algorithms (GAs) hybrid through interior-point algorithm (IPA). The mathematical modeling of the system is constructed using FF-ANN models by defining an unsupervised error and unknown weights; the networks are tuned globally with GAs, and local refinement of the results is made with IPA. Design scheme is applied to study the VdP heart dynamics model by varying the pulse shape modification factor, damping coefficients and external forcing factor while keeping the fixed value of the ventricular contraction period. The results of the proposed algorithm are compared with reference numerical solutions of Adams method to establish its correctness. Multiple independent runs are performed for the scheme, and results of statistical analyses in terms of mean absolute deviation, root-mean-square error and Nash–Sutcliffe efficiency illustrate its applicability, effectiveness and reliability.

     

An improvement on genetic-based learning method for fuzzy artificial neural networks

Fuzzy artificial neural networks (FANNs), which are the generalizations of artificial neural networks (ANNs), refer to connectionist systems in which all inputs, outputs, weights and biases may be fuzzy values. This paper proposes a two-phase learning method for FANNs, which reduces the generated error based on genetic algorithms (GAs). The optimization process is held on the alpha cuts of each fuzzy weight. Global optimized values of the alpha cuts at zero and one levels are obtained in the first phase and optimal values of several other alpha cuts are obtained in the second phase. Proposed method is shown to be superior in terms of generated error and executed time when compared with basic GA-based algorithms.     

A genetic-based neuro-fuzzy approach for modeling and control ofdynamical systems

Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of a FNN is composed of three phases. The first phase is used to find the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches.     

Self-organizing neurofuzzy networks based on evolutionary fuzzy granulation

Experimental software datasets describing software projects in terms of their complexity and development time have been a subject of intensive modeling. A number of various modeling methodologies and modeling designs have been proposed including such development frameworks as neural networks, fuzzy and neurofuzzy models. In this study, we introduce a concept of self-organizing neurofuzzy networks (SONFN), a hybrid modeling architecture combining neurofuzzy networks (NFN) and polynomial neural networks (PNN). For these networks we develop a comprehensive design methodology. The construction of SONFNs takes advantage of the well-established technologies of computational intelligence (CI), namely fuzzy sets, neural networks and genetic algorithms. The architecture of the SONFN results from a synergistic usage of NFNs and PNNs. NFN contributes to the formation of the premise part of the rule-based structure of the SONFN. The consequence part of the SONFN is designed using PNNs. We discuss two types of SONFN architectures whose taxonomy is based on the NFN scheme being applied to the premise part of SONFN. We introduce a comprehensive learning algorithm. It is shown that this network exhibits a dynamic structure as the number of its layers as well as the number of nodes in each layer of the SONFN are not predetermined (as this is the case in a popular topology of a multilayer perceptron). The experimental results include a well-known NASA dataset concerning software cost estimation.     

A hybrid approach based on neural networks and genetic algorithms for detecting temporal patterns in stock markets

This study investigates the effectiveness of a hybrid approach based on the artificial neural networks (ANNs) for time series properties, such as the adaptive time delay neural networks (ATNNs) and the time delay neural networks (TDNNs), with the genetic algorithms (GAs) in detecting temporal patterns for stock market prediction tasks. Since ATNN and TDNN use time-delayed links of the network into a multi-layer feed-forward network, the topology of which grows by on layer at every time step, it has one more estimate of the number of time delays in addition to several control variables of the ANN design. To estimate these many aspects of the ATNN and TDNN design, a general method based on trial and error along with various heuristics or statistical techniques is proposed. However, for the reason that determining the number of time delays or network architectural factors in a stand-alone mode does not guarantee the illuminating improvement of the performance for building the ATNN and TDNN model, we apply GAs to support optimization of the number of time delays and network architectural factors simultaneously for the ATNN and TDNN model. The results show that the accuracy of the integrated approach proposed for this study is higher than that of the standard ATNN, TDNN and the recurrent neural network (RNN).     

Soft computing for crisis management and political decision making: the use of genetically evolved fuzzy cognitive maps

This paper examines the use of fuzzy cognitive maps (FCMs) as a technique for modeling political and strategic issues situations and supporting the decision-making process in view of an imminent crisis. Its object domain is soft computing using as its basic elements different methods from the areas of fuzzy logic, cognitive maps, neural networks and genetic algorithms. FCMs, more specifically, use notions borrowed from artificial intelligence and combine characteristics of both fuzzy logic and neural networks, in the form of dynamic models that describe a given political setting. The present work proposes the use of the genetically evolved certainty neuron fuzzy cognitive map (GECNFCM) as an extension of certainty neuron fuzzy cognitive maps (CNFCMs) aiming at overcoming the main weaknesses of the latter, namely the recalculation of the weights corresponding to each concept every time a new strategy is adopted. This novel technique combines CNFCMs with genetic algorithms (GAs), the advantage of which lies with their ability to offer the optimal solution without a problem-solving strategy, once the requirements are defined. Using a multiple scenario analysis we demonstrate the value of such a hybrid technique in the context of a model that reflects the political and strategic complexity of the Cyprus issue, as well as the uncertainties involved in it. The issue has been treated on a purely technical level, with distances carefully kept concerning all sides involved in it.     

Online elicitation of Mamdani-type fuzzy rules via TSK-based generalized predictive control

Many synergies have been proposed between soft-computing techniques, such as neural networks (NNs), fuzzy logic (FL), and genetic algorithms (GAs), which have shown that such hybrid structures can work well and also add more robustness to the control system design. In this paper, a new control architecture is proposed whereby the on-line generated fuzzy rules relating to the self-organizing fuzzy logic controller (SOFLC) are obtained via integration with the popular generalized predictive control (GPC) algorithm using a Takagi-Sugeno-Kang (TSK)-based controlled autoregressive integrated moving average (CARIMA) model structure. In this approach, GPC replaces the performance index (PI) table which, as an incremental model, is traditionally used to discover, amend, and delete the rules. Because the GPC sequence is computed using predicted future outputs, the new hybrid approach rewards the time-delay very well. The new generic approach, named generalized predictive self-organizing fuzzy logic control (GPSOFLC), is simulated on a well-known nonlinear chemical process, the distillation column, and is shown to produce an effective fuzzy rule-base in both qualitative (minimum number of generated rules) and quantitative (good rules) terms.     

A study on hybrid random signal-based learning and its applications

In this paper, we propose three effective hybrid random signal-based learning (RSL) algorithms which are a combination of RSL with simulated annealing (SA) and a genetic algorithm (GA) to obtain a global solution that can be used in combinatorial optimization problems. GAs are becoming more popular because of their relative simplicity and robustness. GAs are global search techniques for non-linear optimization, but they are not good at fine-tuning solutions. RSL is similar to the reinforcement learning of neural networks using random signals. It can find an accurate solution in local search space. However, it is poor at hill-climbing, whereas simulated annealing has the ability to perform probabilistic hill-climbing. Therefore, combining them yields effective hybrid algorithms, i.e. hybrid RSL algorithms, with the merits of both. To check the generalization ability of the proposed algorithms, the optimizations of several benchmark test functions are considered, while the optimization of a fuzzy logic controller for the inverted pendulum is detailed to show the applicability of the proposed algorithms to fuzzy control.     

Neuro-heuristic computational intelligence for solving nonlinear pantograph systems

We present a neuro-heuristic computing platform for finding the solution for initial value problems (IVPs) of nonlinear pantograph systems based on functional differential equations (P-FDEs) of different orders. In this scheme, the strengths of feed-forward artificial neural networks (ANNs), the evolutionary computing technique mainly based on genetic algorithms (GAs), and the interior-point technique (IPT) are exploited. Two types of mathematical models of the systems are constructed with the help of ANNs by defining an unsupervised error with and without exactly satisfying the initial conditions. The design parameters of ANN models are optimized with a hybrid approach GA–IPT, where GA is used as a tool for effective global search, and IPT is incorporated for rapid local convergence. The proposed scheme is tested on three different types of IVPs of P-FDE with orders 1–3. The correctness of the scheme is established by comparison with the existing exact solutions. The accuracy and convergence of the proposed scheme are further validated through a large number of numerical experiments by taking different numbers of neurons in ANN models.     

A novel parallel hybrid intelligence optimization algorithm for a function approximation problem

A novel parallel hybrid intelligence optimization algorithm (PHIOA) is proposed based on combining the merits of particle swarm optimization with genetic algorithms. The PHIOA uses the ideas of selection, crossover and mutation from genetic algorithms (GAs) and the update velocity and situation of particle swarm optimization (PSO) under the independence of PSO and GAs. The proposed algorithm divides the individuals into two equation groups according to their fitness values. The subgroup of the top fitness values is evolved by GAs and the other subgroup is evolved by the PSO algorithm. The optimal number is selected as a global optimum at every circulation which shows better results than both PSO and GAs, then improves the overall performance of the algorithm. The PHIOA is used to optimize the structure and parameters of the fuzzy neural network. Finally, the experimental results have demonstrated the superiority of the proposed PHIOA to search the global optimal solution. The PHIOA can improve the error accuracy while speeding up the convergence process, and effectively avoid the premature convergence to compare with the existing methods.