Genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this paper, we propose and investigate a new category of neurofuzzy networks—fuzzy polynomial neural networks (FPNN) endowed with fuzzy set-based polynomial neurons (FSPNs) We develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms (GAs) in particular. The conventional FPNNs developed so far are based on the mechanisms of self-organization, fuzzy neurocomputing, and evolutionary optimization. The design of the network exploits the FSPNs as well as the extended group method of data handling (GMDH). Let us stress that in the previous development strategies some essential parameters of the networks (such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables) being available within the network are provided by the designer in advance and kept fixed throughout the overall development process. This restriction may hamper a possibility of developing an optimal architecture of the model. The design proposed in this study addresses this issue. The augmented and genetically developed FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional FPNNs. The GA-based design procedure being applied at each layer of the FPNN leads to the selection of the most suitable nodes (or FSPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation in which we use a number of modeling benchmarks—synthetic and experimental data being commonly used in fuzzy or neurofuzzy modeling. The obtained results demonstrate the superiority of the proposed networks over the models existing in the references.     

Hybrid fuzzy set-based polynomial neural networks and their development with the aid of genetic optimization and information granulation

We introduce a new architecture of feed-forward neural networks called hybrid fuzzy set-based polynomial neural networks (HFSPNNs) that are composed of heterogeneous feed-forward neural networks such as polynomial neural networks (PNNs) and fuzzy set-based polynomial neural networks (FSPNNs). We develop their comprehensive design methodology by embracing mechanisms of genetic optimization and information granulation. The construction of information granulation-driven HFSPNN exploits fundamental technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms (GAs). The architecture of the resulting information granulation-driven genetically optimized HFSPNN results from a synergistic usage of the hybrid system generated by combining original fuzzy set-based polynomial neurons (FSPNs)-based FSPNN with polynomial neurons (PNs)-based PNN. The design of the conventional genetically optimized HFPNN exploits the extended Group Method of Data Handling (GMDH) whose some essential parameters of the network being tuned with the use of genetic algorithms throughout the overall development process. Two general optimization mechanisms are explored. First, the structural optimization is realized via GAs while the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gHFSPNN is quantified through extensive experimentation where we considered a number of modeling benchmarks (synthetic and experimental data already experimented with in fuzzy or neurofuzzy modeling).     

Evolutionary design of hybrid self-organizing fuzzy polynomial neural networks with the aid of information granulation

We introduce a new architecture of information granulation-based and genetically optimized Hybrid Self-Organizing Fuzzy Polynomial Neural Networks (HSOFPNN). Such networks are based on genetically optimized multi-layer perceptrons. We develop their comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The architecture of the resulting HSOFPNN combines fuzzy polynomial neurons (FPNs) that are located at the first layer of the network with polynomial neurons (PNs) forming the remaining layers of the network. The augmented version of the HSOFPNN, “IG_gHSOFPNN”, for brief, embraces the concept of information granulation and subsequently exhibits higher level of flexibility and leads to simpler architectures and rapid convergence speed to optimal structure in comparison with the HSOFPNNs and SOFPNNs.

The GA-based design procedure being applied at each layer of HSOFPNN leads to the selection of preferred nodes of the network (FPNs or PNs) whose local characteristics (such as the number of input variables, the order of the polynomial, a collection of the specific subset of input variables, the number of membership functions for each input variable, and the type of membership function) can be easily adjusted. In the sequel, two general optimization mechanisms are explored. The structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is afterwards carried out in the setting of a standard least square method-based learning. The obtained results demonstrate a superiority of the proposed networks over the existing fuzzy and neural models.     

Fuzzy polynomial neural networks: hybrid architectures of fuzzy modeling

We introduce a concept of fuzzy polynomial neural networks (FPNNs), a hybrid modeling architecture combining polynomial neural networks (PNNs) and fuzzy neural networks (FNNs). The development of the FPNNs dwells on the technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms. The structure of the FPNN results from a synergistic usage of FNN and PNN. FNNs contribute to the formation of the premise part of the rule-based structure of the FPNN. The consequence part of the FPNN is designed using PNNs. The structure of the PNN is not fixed in advance as it usually takes place in the case of conventional neural networks, but becomes organized dynamically to meet the required approximation error. We exploit a group method of data handling (GMDH) to produce this dynamic topology of the network. The performance of the FPNN is quantified through experimentation that exploits standard data already used in fuzzy modeling. The obtained experimental results reveal that the proposed networks exhibit high accuracy and generalization capabilities in comparison to other similar fuzzy models.     

Structural developments of fuzzy systems with the aid of information granulation

We introduce a design procedure for fuzzy systems using the concept of information granulation and genetic optimization. Information granulation and resulting information granules themselves become an important design aspect of fuzzy models. By accommodating the formalism of fuzzy sets, the model is geared towards capturing relationship between information granules (fuzzy sets) rather than concentrating on plain numeric data. Information granulation realized with the use of the standard C-Means clustering helps determine the initial values of the parameters of the fuzzy models. This in particular concerns such essential components of the rules as the initial apexes of the membership functions standing in the premise part of the fuzzy rules and the initial values of the polynomial functions standing in the consequence part. The initial parameters are afterwards tuned with the aid of the genetic algorithms (GAs) and the least square method (LSM). The overall design methodology arises as a hybrid development process involving structural and parametric optimization. Especially, genetic algorithms and C-Means are used to generate the structurally as well as parametrically optimized fuzzy model. To identify the structure and estimate parameters of the fuzzy model we exploit the methodologies such as joint and successive method realized by means of genetic algorithms. The proposed model is evaluated using experimental data and its performance is contrasted with the behavior of the fuzzy models available in the literature.     

Fuzzy set-oriented neural networks based on fuzzy polynomial inference and dynamic genetic optimization

In this paper, we introduce a new topology and offer a comprehensive design methodology of fuzzy set-based neural networks (FsNNs). The proposed architecture of the FsNNs is based on the fuzzy polynomial neurons formed through a collection of ‘if-then’ fuzzy rules, fuzzy inference, and polynomials with the extended structure of the premise and the consequence parts of fuzzy rules being formed within the networks. Three different forms of regression polynomials (namely constant, linear, and quadratic) are used in the consequence part of the rules. In order to build an optimal FsNN, the underlying structural and parametric optimization is supported by a dynamic search-based genetic algorithm (GA), which forms an optimal solution through successive adjustments (refinements) of the search range. The structure optimization involves the determination of the input variables included in the premise part and the order of the polynomial forming the consequence part of the rules. In the study, we explore two types of optimization methodologies, namely a simultaneous tuning and a separate tuning. GAs are global optimizers; however, when being used in their generic version, they often lead to a significant computing overhead caused by the need to explore an excessively large search space. To eliminate this shortcoming and increase the effectiveness of the optimization itself, we introduce a dynamic search-based GA that results in a rapid convergence while narrowing down the search to a limited region of the search space. We exploit this optimization mechanism to be completed both at the structural as well as the parametric level. To evaluate the performance of the proposed FsNN, we offer a suite of several representative numerical examples.     

A new approach to radial basis function-based polynomial neural networks: analysis and design

In this study, we introduce a new topology of radial basis function-based polynomial neural networks (RPNNs) that is based on a genetically optimized multi-layer perceptron with radial polynomial neurons (RPNs). This paper offers a comprehensive design methodology involving various mechanisms of optimization, especially fuzzy C-means (FCM) clustering and particle swarm optimization (PSO). In contrast to the typical architectures encountered in polynomial neural networks (PNNs), our main objective is to develop a topology and establish a comprehensive design strategy of RPNNs: (a) The architecture of the proposed network consists of radial polynomial neurons (RPN). These neurons are fully reflective of the structure encountered in numeric data, which are granulated with the aid of FCM clustering. RPN dwells on the concepts of a collection of radial basis function and the function-based nonlinear polynomial processing. (b) The PSO-based design procedure being applied to each layer of the RPNN leads to the selection of preferred nodes of the network whose local parameters (such as the number of input variables, a collection of the specific subset of input variables, the order of the polynomial, the number of clusters of FCM clustering, and a fuzzification coefficient of the FCM method) are properly adjusted. The performance of the RPNN is quantified through a series of experiments where we use several modeling benchmarks, namely a synthetic three-dimensional data and learning machine data (computer hardware data, abalone data, MPG data, and Boston housing data) already used in neuro-fuzzy modeling. A comparative analysis shows that the proposed RPNN exhibits higher accuracy in comparison with some previous models available in the literature.     

Fuzzy polynomial neural networks for approximation of the compressive strength of concrete

The main purpose of this paper is to develop fuzzy polynomial neural networks (FPNN) to predict the compressive strength of concrete. Two different architectures of FPNN are addressed (Type1 and Type2) and their training methods are discussed. In this research, the proposed FPNN is a combination of fuzzy neural networks (FNNs) and polynomial neural networks (PNNs). Here, while the FNN demonstrates the premises (If-Part) of the fuzzy model, the PNN is implemented as its consequence (Then-Part). To enhance the performance of the network, back propagation (BP), and list square error (LSE) algorithms are utilized for the tuning of the system.Six different FPNN architectures are constructed, trained, and tested using the experimental data of 458 different concrete mix-designs collected from three distinct sources. The data are organized in a format of six input parameters of concrete ingredients and one output as 28-day compressive strength of the mix-design. Using root means square (RMS) and correlation factors (CFs), the models are evaluated and compared with training and testing data pairs. The results show that FPNN-Type1 has strong potential as a feasible tool for prediction of the compressive strength of concrete mix-design. However, the FPNN-Type2 is recognized as unfeasible model to this purpose.     

Advanced self-organizing polynomial neural network

In this paper, we introduce a concept of advanced self-organizing polynomial neural network (Adv_SOPNN). The SOPNN is a flexible neural architecture whose structure is developed through a modeling process. But the SOPNN has a fatal drawback; it cannot be constructed for nonlinear systems with few input variables. To relax this limitation of the conventional SOPNN, we combine a fuzzy system and neural networks with the SOPNN. Input variables are partitioned into several subspaces by the fuzzy system or neural network, and these subspaces are utilized as new input variables to the SOPNN architecture. Two types of the advanced SOPNN are obtained by combining not only the fuzzy rules of a fuzzy system with SOPNN but also the nodes in a hidden layer of neural networks with SOPNN into one methodology. The proposed method is applied to the nonlinear system with two inputs, which cannot be identified by conventional SOPNN to show the performance of the advanced SOPNN. The results show that the proposed method is efficient for systems with limited data set and a few input variables and much more accurate than other modeling methods with respect to identification error.     

A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering

In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP²N²), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs).The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models.We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space.We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling.     

Fuzzy polynomial neurons as neurofuzzy processing units

In this study, we introduce and study fuzzy polynomial neurons (FPNs) being regarded as generic processing units in neurofuzzy computing. The underlying topology of FPNs is formed through fuzzy rules, fuzzy inference and polynomials. Each polynomial offers a nonlinear mapping and is centred around a modal value of the corresponding membership functions defined in the input space of the neuron. The adjustable order of the polynomial is essential when addressing the level of nonlinearity to be handled in the approximation problem. We demonstrate that fuzzy polynomial neurons form a certain class of functional neurons and afterwards discuss their properties and an overall design process. Furthermore, these neurons are discussed in the context of universal approximation and universal approximators     

Evolutionary fuzzy modeling

This study is concerned with a general methodology of identification of fuzzy models. Unlike numeric models, fuzzy models operate at a level of information granules - fuzzy sets - and this aspect brings up an important design requirement of transparency of the model. We propose a three-phase development framework by distinguishing between structural and parametric optimization processes. The underlying topology of the model dwells on fuzzy neural networks - architectures governed by fuzzy logic and equipped with parametric flexibility. Two general optimization mechanisms are explored: the structural optimization is realized via genetic programming whereas for the ensuing detailed parametric optimization we proceed with gradient-based learning. The main advantages of this approach are discussed in detail. The study is illustrated with the aid of a numeric example that provides a detailed insight into the performance of the fuzzy models and quantifies crucial design issues.     

A polynomial fuzzy neural network for modeling and control

This paper introduces a novel neurofuzzy system based on polynomial fuzzy neural network (PFNN) architecture. A PFNN consists of a set of if-then rules with appropriate membership functions (MFs) whose parameters are optimized via a hybrid genetic algorithm. A polynomial neural network is employed in the defuzzification scheme to improve output performance and to select appropriate rules. A performance criterion for model selection is defined to overcome the overfitting problem in the modeling procedure. For a performance assessment of the PFNN inference system, two well-known problems are employed for a comparison with other methods. The results of these comparisons show that the PFNN inference system out-performs the other methods and exhibits robustness characteristics. This work was presented in part at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization

In many real-world problems involving pattern recognition, system identification and modeling, control, decision making, and forecasting of time-series, available data are quite often of uncertain nature. An interesting alternative is to employ type-2 fuzzy sets, which augment fuzzy models with expressive power to develop models, which efficiently capture the factor of uncertainty. The three-dimensional membership functions of type-2 fuzzy sets offer additional degrees of freedom that make it possible to directly and more effectively account for model’s uncertainties. Type-2 fuzzy logic systems developed with the aid of evolutionary optimization forms a useful modeling tool subsequently resulting in a collection of efficient “If-Then” rules.The type-2 fuzzy neural networks take advantage of capabilities of fuzzy clustering by generating type-2 fuzzy rule base, resulting in a small number of rules and then optimizing membership functions of type-2 fuzzy sets present in the antecedent and consequent parts of the rules. The clustering itself is realized with the aid of differential evolution.Several examples, including a benchmark problem of identification of nonlinear system, are considered. The reported comparative analysis of experimental results is used to quantify the performance of the developed networks.     

RSPOP: rough set-based pseudo outer-product fuzzy rule identification algorithm

System modeling with neuro-fuzzy systems involves two contradictory requirements: interpretability verses accuracy. The pseudo outer-product (POP) rule identification algorithm used in the family of pseudo outer-product-based fuzzy neural networks (POPFNN) suffered from an exponential increase in the number of identified fuzzy rules and computational complexity arising from high-dimensional data. This decreases the interpretability of the POPFNN in linguistic fuzzy modeling. This article proposes a novel rough set-based pseudo outer-product (RSPOP) algorithm that integrates the sound concept of knowledge reduction from rough set theory with the POP algorithm. The proposed algorithm not only performs feature selection through the reduction of attributes but also extends the reduction to rules without redundant attributes. As many possible reducts exist in a given rule set, an objective measure is developed for POPFNN to correctly identify the reducts that improve the inferred consequence. Experimental results are presented using published data sets and real-world application involving highway traffic flow prediction to evaluate the effectiveness of using the proposed algorithm to identify fuzzy rules in the POPFNN using compositional rule of inference and singleton fuzzifier (POPFNN-CRI(S)) architecture. Results showed that the proposed rough set-based pseudo outer-product algorithm reduces computational complexity, improves the interpretability of neuro-fuzzy systems by identifying significantly fewer fuzzy rules, and improves the accuracy of the POPFNN.     

基于分段多项式李雅普诺夫函数的模糊系统的稳定性分析

本文采用最大型分段多项式李雅普诺夫函数研究了多项式模糊系统的闭环稳定性问题.首先,本文设计了与分段李雅普诺夫函数对应的切换模糊控制器,提出了多项式模糊模型稳定的平方和条件,同时证明了最大型分段多项式李雅普诺夫函数在函数切换点的稳定性.然后,设计了相应的路径跟踪优化算法,对本文非凸的稳定条件进行迭代求解.最后,通过两个算例进行仿真与比较,说明并验证了本文所提出结论的可行有效性.     

基于F-SVMs的多模型建模方法

针对全局模型难以精确描述复杂工业过程的问题,提出一种基于模糊支持向量机(F-SVMs)的多模型(F-SVMs MM)建模方法。用模糊支持向量分类算法(F-SVC)对输入数据进行预处理,得到多模型模糊隶属度;用模糊支持回归算法(F-SVR)建立多模型(MM)估计器。应用该方法对pH中和滴定过程进行建模,仿真结果表明,F-SVMs MM跟踪性能好、泛化能力强,比USOCPN方法和标准支持向量机(SVMs)方法具有更好的性能和推广能力。     

Numerical solution of a fuzzy system of linear equations with polynomial parametric form

A systems of linear equations are used in many fields of science and industry, such as control theory and image processing, and solving a fuzzy linear system of equations is now a necessity. In this work we try to solve a fuzzy system of linear equations having fuzzy coefficients and crisp variables using a polynomial parametric form of fuzzy numbers.     

A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.     

正则模糊神经网络是模糊值函数的泛逼近器

通过分析多元模糊值Bernstein多项式的近似特性，证明了4层前向正则模糊神经网络（FNN）的逼近性能，该类网络构成了模糊值函数的一类泛逼近器，即在欧氏空间的任何紧集上，任意连续模糊值函数能被这类FNN逼近到任意精度，最后通过实例给出了实现这种近似的具体步骤。