Implicit rule-based fuzzy-neural networks using the identification algorithm of GA hybrid scheme based on information granulation

This paper proposes an identification method for nonlinear models realized in the form of implicit rule-based fuzzy-neural networks (FNN). The design of the model dwells on the technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithm. The FNN modeling and identification environment realizes parameter estimation through a synergistic usage of clustering techniques, genetic optimization and a complex search method. An HCM (Hard C-Means) clustering algorithm helps determine an initial location (parameters) of the membership functions of the information granules to be used in this fuzzy model. The parameters such as apexes of membership functions, learning rates, and momentum coefficients are then adjusted using the optimization algorithm of a GA hybrid scheme. The proposed GA hybrid scheme combines GA with the improved complex method to guarantee both global optimization and local convergence. An aggregate objective function (performance index) is used in the model design in order to achieve a sound balance between its approximation and generalization abilities. The proposed type of the model is experimented with several time series data (gas furnace, sewage treatment process, and NOx emission process data of gas turbine power plant).     

用HCM聚类和遗传算法实现多级模糊神经网络

模糊集理论适用于一些实验数据中不确定性和模糊性的建模问题,而模糊推理系统拥有模糊IF-THEN格式的结构化知识表示,但缺少适应性。神经网络本身具有对外部很强的适应性和从过去数据中学习的机制,但基于线性推理的模糊神经网络(FNN)模型作为模糊推理方法不能得到存在于参数间的最终关系,也不能影响接着发生的模糊集合。因此,我们提出了一个多级模糊神经网络(Multi-FNN),使用硬C均值聚类和进化模糊颗粒,利用处理为近似推理的一个线性推理,获得信息微粒和模糊集之间的关系。     

DCT-Yager FNN: a novel Yager-based fuzzy neural network with the discrete clustering technique.

Earlier clustering techniques such as the modified learning vector quantization (MLVQ) and the fuzzy Kohonen partitioning (FKP) techniques have focused on the derivation of a certain set of parameters so as to define the fuzzy sets in terms of an algebraic function. The fuzzy membership functions thus generated are uniform, normal, and convex. Since any irregular training data is clustered into uniform fuzzy sets (Gaussian, triangular, or trapezoidal), the clustering may not be exact and some amount of information may be lost. In this paper, two clustering techniques using a Kohonen-like self-organizing neural network architecture, namely, the unsupervised discrete clustering technique (UDCT) and the supervised discrete clustering technique (SDCT), are proposed. The UDCT and SDCT algorithms reduce this data loss by introducing nonuniform, normal fuzzy sets that are not necessarily convex. The training data range is divided into discrete points at equal intervals, and the membership value corresponding to each discrete point is generated. Hence, the fuzzy sets obtained contain pairs of values, each pair corresponding to a discrete point and its membership grade. Thus, it can be argued that fuzzy membership functions generated using this kind of a discrete methodology provide a more accurate representation of the actual input data. This fact has been demonstrated by comparing the membership functions generated by the UDCT and SDCT algorithms against those generated by the MLVQ, FKP, and pseudofuzzy Kohonen partitioning (PFKP) algorithms. In addition to these clustering techniques, a novel pattern classifying network called the Yager fuzzy neural network (FNN) is proposed in this paper. This network corresponds completely to the Yager inference rule and exhibits remarkable generalization abilities. A modified version of the pseudo-outer product (POP)-Yager FNN called the modified Yager FNN is introduced that eliminates the drawbacks of the earlier network and yi- elds superior performance. Extensive experiments have been conducted to test the effectiveness of these two networks, using various clustering algorithms. It follows that the SDCT and UDCT clustering algorithms are particularly suited to networks based on the Yager inference rule.     

Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks

A key step of using gradient descend methods to develop learning algorithms of a regular feedforward fuzzy neural network (FNN) is to differentiate max-min functions, which contain max(/spl or/) and min(/spl and/) operations. The paper aims at several objectives. First, investigate further the differentiation of /spl or/-/spl and/ functions. Second, employ general fuzzy numbers, which include triangular and trapezoidal fuzzy numbers as special cases to define a three-layer regular FNN. The general fuzzy numbers related can be approximately determined by their corresponding finite level sets. So, we can approximately represent the input-output (I/O) relationship of the regular FNN as functions of the endpoints of all finite level sets. Third, a fuzzy back-propagation algorithm is presented. And to speed up the convergence of the learning algorithm, a fuzzy conjugate gradient algorithm for fuzzy weights and biases is developed, furthermore, the convergence of the algorithm is analyzed, systematically. Finally, some real simulations demonstrate the efficiency of our learning algorithms. The regular FNN is applied to the approximate realization of fuzzy inference rules and fuzzy functions defined on given compact sets.     

新型模糊神经网络模型

分析了已有模糊神经网络模型结构与学习算法的特点,针对它们收敛速度慢、全局逼近能力差等不足,提出了一种新型的模糊神经网络模型,其在模糊化层实现了隶属函数的合成,且结构简单、推理层只有两个节点。实验结果表明该模型具有收敛速度快、全局逼近能力强的优点,具有一定的实用价值。     

Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

Applying fuzzy neural network to estimate software development effort

The ability to accurately and consistently estimate software development efforts is required by the project managers in planning and conducting software development activities. Since software effort drivers are vague and uncertain, software effort estimates, especially in the early stages of the development life cycle, are prone to a certain degree of estimation errors. A software effort estimation model which adopts a fuzzy inference method provides a solution to fit the uncertain and vague properties of software effort drivers. The present paper proposes a fuzzy neural network (FNN) approach for embedding artificial neural network into fuzzy inference processes in order to derive the software effort estimates. Artificial neural network is utilized to determine the significant fuzzy rules in fuzzy inference processes. We demonstrated our approach by using the 63 historical project data in the well-known COCOMO model. Empirical results showed that applying FNN for software effort estimates resulted in slightly smaller mean magnitude of relative error (MMRE) and probability of a project having a relative error of less than or equal to 0.25 (Pred(0.25)) as compared with the results obtained by just using artificial neural network and the original model. The proposed model can also provide objective fuzzy effort estimation rule sets by adopting the learning mechanism of the artificial neural network.     

Gaussian kernel-based fuzzy inference systems for high dimensional regression

We propose a novel architecture for a higher order fuzzy inference system (FIS) and develop a learning algorithm to build the FIS. The consequent part of the proposed FIS is expressed as a nonlinear combination of the input variables, which can be obtained by introducing an implicit mapping from the input space to a high dimensional feature space. The proposed learning algorithm consists of two phases. In the first phase, the antecedent fuzzy sets are estimated by the kernel-based fuzzy c-means clustering. In the second phase, the consequent parameters are identified by support vector machine whose kernel function is constructed by fuzzy membership functions and the Gaussian kernel. The performance of the proposed model is verified through several numerical examples generally used in fuzzy modeling. Comparative analysis shows that, compared with the zero-order fuzzy model, first-order fuzzy model, and polynomial fuzzy model, the proposed model exhibits higher accuracy, better generalization performance, and satisfactory robustness.     

The construction of fuzzy least squares estimators in fuzzy linear regression models

A new concept and method of imposing imprecise (fuzzy) input and output data upon the conventional linear regression model is proposed. Under the considerations of fuzzy parameters and fuzzy arithmetic operations (fuzzy addition and multiplication), we propose a fuzzy linear regression model which has the similar form as that of conventional one. We conduct the h-level (conventional) linear regression models of fuzzy linear regression model for the sake of invoking the statistical techniques in (conventional) linear regression analysis for real-valued data. In order to determine the sign (nonnegativity or nonpositivity) of fuzzy parameters, we perform the statistical testing hypotheses and evaluate the confidence intervals. Using the least squares estimators obtained from the h-level linear regression models, we can construct the membership functions of fuzzy least squares estimators via the form of “Resolution Identity” which is well-known in fuzzy sets theory. In order to obtain the membership degree of any given estimate taken from the fuzzy least squares estimator, optimization problems have to be solved. We also provide two computational procedures to deal with those optimization problems.     

Structural developments of fuzzy systems with the aid of information granulation

We introduce a design procedure for fuzzy systems using the concept of information granulation and genetic optimization. Information granulation and resulting information granules themselves become an important design aspect of fuzzy models. By accommodating the formalism of fuzzy sets, the model is geared towards capturing relationship between information granules (fuzzy sets) rather than concentrating on plain numeric data. Information granulation realized with the use of the standard C-Means clustering helps determine the initial values of the parameters of the fuzzy models. This in particular concerns such essential components of the rules as the initial apexes of the membership functions standing in the premise part of the fuzzy rules and the initial values of the polynomial functions standing in the consequence part. The initial parameters are afterwards tuned with the aid of the genetic algorithms (GAs) and the least square method (LSM). The overall design methodology arises as a hybrid development process involving structural and parametric optimization. Especially, genetic algorithms and C-Means are used to generate the structurally as well as parametrically optimized fuzzy model. To identify the structure and estimate parameters of the fuzzy model we exploit the methodologies such as joint and successive method realized by means of genetic algorithms. The proposed model is evaluated using experimental data and its performance is contrasted with the behavior of the fuzzy models available in the literature.     

A fuzzy set based solution method for multiobjective optimal design problem of mechanical and structural systems using functional-link net

The principle of solving multiobjective optimization problems with fuzzy sets theory is studied. Membership function is the key to introduce the fuzzy sets theory to multiobjective optimization. However, it is difficult to determine membership functions in engineering applications. On the basis of rapid quadratic optimization in the learning of weights, simplification in hardware as well as in computational procedures of functional-link net, discrete membership functions are used as sample training data. When the network converges, the continuous membership functions implemented with the network. Membership functions based on functional-link net have been used in multiobjective optimization. An example is given to illustrate the method.     

Construction of a neuron-fuzzy classification model based on feature-extraction approach

In this paper, a Feature-Extraction Neuron-Fuzzy Classification Model (FENFCM) is proposed that enables the extraction of feature variables and provides the classification results. The proposed classification model synergistically integrates a standard fuzzy inference system and a neural network with supervised learning. The FENFCM automatically generates the fuzzy rules from the numerical data and triangular functions that are used as membership functions both in the feature extraction unit and in the inference unit. To adapt the proposed FENFCM, two modificatory algorithms are applied. First, we utilize Evolutionary Programming (EP) to determine the distribution of fuzzy sets for each feature variable of the feature extraction unit. Second, the Weight Revised Algorithm (WRA) is used to regulate the weight grade of the principal output node of the inference unit. Finally, the proposed FENFCM is validated using two benchmark data sets: the Wine database and the Iris database. Computer simulation results demonstrate that the proposed classification model can provide a sufficiently high classification rate in comparison with that of other models proposed in the literature.     

A genetic approach to modeling fuzzy systems based on information granulation and successive generation-based evolution method

In this paper, we introduce and investigate a new category of fuzzy inference systems based on information granulation and genetic optimization used to system identification. We show the applications of such systems to identification of nonlinear systems. The formal framework of information granulation and resulting information granules themselves become an important design facet of the fuzzy models. By embracing fuzzy sets, the model is geared towards capturing essential relationship between information granules rather than concentrating on plain numeric data. Information granulation realized with the use of the commonly exploited C-Means clustering helps determine the initial values of the parameters of the fuzzy models. This in particular concerns such essential components of the rules as the initial apexes of the membership functions standing in the premise part of the fuzzy rules and the points of the polynomial functions standing in the consequence part. The initial apexes (center points) of the membership functions based on C-Means algorithm are tuned with the aid of the genetic algorithm (GA), while the tuned apexes are also used to adjust the points of the consequent polynomials (conclusions) of the rules. In particular, the initial apexes of the membership functions and the initial points of the consequent polynomials are adjusted and updated every time through successive evolution process. The overall design methodology involves a hybrid structural and parametric optimization. Genetic algorithms and C-Means clustering are used to optimize the model with respect to its structure and parameters. To determine the structure and estimate the values of the parameters of the fuzzy model we consider the successive tuning method with generation-based evolution by means of genetic algorithms. The model is evaluated with the use of numerical experimentation and its quality is compared with respect to some other fuzzy models already encountered in the literature.     

一种新的基于神经模糊推理网络的复杂系统模糊辨识方法

针对基于输入输出数据的复杂系统的模糊辨识问题,提出了一种新的神经模糊推理网络及相应的学习算法．学习算法被应用于系统的结构辨识与参数辨识．在结构辨识阶段,介绍了一种新的直接从输入输出数据中抽取和优化模糊规则的学习算法;在参数辨识阶段,提出和推导了一种非监督学习和监督学习相结合的混合式学习算法,实现模糊隶属函数的初步调整和优化．仿真结果表明,本文的方法可以同时满足对辨识精度、收敛速度、可读性和规则数的要求．     

Genetic tuning of fuzzy inference within fuzzy classifier systems

Abstract: In generating a suitable fuzzy classifier system, significant effort is often placed on the determination and the fine tuning of the fuzzy sets. However, in such systems little thought is given to the way in which membership functions are combined within the fuzzy rules. Often traditional fuzzy inference strategies are used which consequently provide no control over how strongly or weakly the inference is applied within these rules. Furthermore such strategies will allow no interaction between grades of membership. A number of theoretical fuzzy inference operators have been proposed for both regression and classification problems but they have not been investigated in the context of real-world applications. In this paper we propose a novel genetic algorithm framework for optimizing the strength of fuzzy inference operators concurrently with the tuning of membership functions for a given fuzzy classifier system. Each fuzzy system is generated using two well-established decision tree algorithms: C4.5 and CHAID. This will enable both classification and regression problems to be addressed within the framework. Each solution generated by the genetic algorithm will produce a set of fuzzy membership functions and also determine how strongly the inference will be applied within each fuzzy rule. We investigate several theoretical proven fuzzy inference techniques (T-norms) in the context of both classification and regression problems. The methodology proposed is applied to a number of real-world data sets in order to determine the effects of the simultaneous tuning of membership functions and inference parameters on the accuracy and robustness of fuzzy classifiers.     

应用模糊神经网络预测油田产量

为了研究受多变量、时变和不确定因素影响的油田产量预测问题,将模糊逻辑推理技术与人工神经网络相结合,构建具有模糊逻辑推理和学习功能的模糊神经网络(FNN)系统。该系统基于现有的油田开发历史数据,建立相应的规则集,使用神经网络的训练方法(如梯度下降学习算法),在训练过程中调整参数,并自适应增加规则,以使系统的输出最佳地逼近于目标样本。通过对某油田的实际开发历史数据的拟合与测试,结果表明该模糊神经网络能够较精确地预测未来的油产量,与常规的BP神经网络相比,其预测精度更高、训练速度更快。因此,基于模糊神经网络(FNN)的油田产量预测方法研究具有较好的实际应用价值。     

Class imbalance learning via a fuzzy total margin based support vector machine

The classification of imbalanced data is a major challenge for machine learning. In this paper, we presented a fuzzy total margin based support vector machine (FTM-SVM) method to handle the class imbalance learning (CIL) problem in the presence of outliers and noise. The proposed method incorporates total margin algorithm, different cost functions and the proper approach of fuzzification of the penalty into FTM-SVM and formulates them in nonlinear case. We considered an excellent type of fuzzy membership functions to assign fuzzy membership values and got six FTM-SVM settings. We evaluated the proposed FTM-SVM method on two artificial data sets and 16 real-world imbalanced data sets. Experimental results show that the proposed FTM-SVM method has higher G_Mean and F_Measure values than some existing CIL methods. Based on the overall results, we can conclude that the proposed FTM-SVM method is effective for CIL problem, especially in the presence of outliers and noise in data sets.     

Fuzzy linear regression model based on fuzzy scalar product

The new concept and method of imposing imprecise (fuzzy) input and output data upon the conventional linear regression model is proposed in this paper. We introduce the fuzzy scalar (inner) product to formulate the fuzzy linear regression model. In order to invoke the conventional approach of linear regression analysis for real-valued data, we transact the α-level linear regression models of the fuzzy linear regression model. We construct the membership functions of fuzzy least squares estimators via the form of “Resolution Identity” which is a well-known formula in fuzzy sets theory. In order to obtain the membership value of any given least squares estimate taken from the fuzzy least squares estimator, we transform the original problem into the optimization problems. We also provide two computational procedures to solve the optimization problems.     

A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing

This paper proposes a recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) for dynamic system processing. An RSEIT2FNN incorporates type-2 fuzzy sets in a recurrent neural fuzzy system in order to increase the noise resistance of a system. The antecedent parts in each recurrent fuzzy rule in the RSEIT2FNN are interval type-2 fuzzy sets, and the consequent part is of the Takagi-Sugeno-Kang (TSK) type with interval weights. The antecedent part of RSEIT2FNN forms a local internal feedback loop by feeding the rule firing strength of each rule back to itself. The TSK-type consequent part is a linear model of exogenous inputs. The RSEIT2FNN initially contains no rules; all rules are learned online via structure and parameter learning. The structure learning uses online type-2 fuzzy clustering. For the parameter learning, the consequent part parameters are tuned by a rule-ordered Kalman filter algorithm to improve learning performance. The antecedent type-2 fuzzy sets and internal feedback loop weights are learned by a gradient descent algorithm. The RSEIT2FNN is applied to simulations of dynamic system identifications and chaotic signal prediction under both noise-free and noisy conditions. Comparisons with type-1 recurrent fuzzy neural networks validate the performance of the RSEIT2FNN.     

A genetic-based neuro-fuzzy approach for modeling and control ofdynamical systems

Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of a FNN is composed of three phases. The first phase is used to find the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches.