Generalized neurofuzzy network modeling algorithms usingBezier-Bernstein polynomial functions and additive decomposition

This paper introduces a new neurofuzzy model construction algorithm for nonlinear dynamic systems based upon basis functions that are Bezier-Bernstein polynomial functions. This paper is generalized in that it copes with n-dimensional inputs by utilising an additive decomposition construction to overcome the curse of dimensionality associated with high n. This new construction algorithm also introduces univariate Bezier-Bernstein polynomial functions for the completeness of the generalized procedure. Like the B-spline expansion based neurofuzzy systems, Bezier-Bernstein polynomial function based neurofuzzy networks hold desirable properties such as nonnegativity of the basis functions, unity of support, and interpretability of basis function as fuzzy membership functions, moreover with the additional advantages of structural parsimony and Delaunay input space partition, essentially overcoming the curse of dimensionality associated with conventional fuzzy and RBF networks. This new modeling network is based on additive decomposition approach together with two separate basis function formation approaches for both univariate and bivariate Bezier-Bernstein polynomial functions used in model construction. The overall network weights are then learnt using conventional least squares methods. Numerical examples are included to demonstrate the effectiveness of this new data based modeling approach.     

Learning associations between natural groups of input and output with a neurofuzzy structure

A new modeling approach that finds the associations between natural groups of input and output is proposed. In the new method, input and output are clustered separately by means of Fuzzy C-Means (FCM) algorithm. Then, the learning algorithm identifies the fuzzy rules by relating the resulting fuzzy sets in input and output spaces by using a neurofuzzy architecture. A modified version of classical simulated annealing algorithm is used in order to identify the relative weights of system input variables. The proposed approach is applied to a highly nonlinear function and successful result is achieved.     

Neurofuzzy network based self-tuning control with offset eliminating

The design of nonlinear controllers involves first selecting the input and then determining the nonlinear functions for the controllers. Since systems described by smooth nonlinear functions can be approximated by linear models in the neighbourhood of the selected operating points, the input of the nonlinear controller at these operating points can be chosen to be identical to those of the local linear controllers. Following this approach, it is proposed that the input of the nonlinear controller are similarly chosen, and that the local linear controllers are designed based on the integrating and k-incremental suboptimal control laws for their ability to remove offsets. Neurofuzzy networks are used to implement the nonlinear controllers for their ability to approximate nonlinear functions with arbitrary accuracy, and to be trained from experimental data. These nonlinear controllers are referred to as neurofuzzy controllers for convenience. As the integrating and k-incremental control laws have also been applied to implement self-tuning controllers, the proposed neurofuzzy controllers can also be interpreted as self-tuning nonlinear controllers. The training target for the neurofuzzy controllers is derived, and online training of the neurofuzzy controllers using a simplified recursive least squares (SRLS) method is presented. It is shown that using the SRLS method, computing time to train the neurofuzzy controllers can be drastically reduced and the ability to track varying dynamics improved. The performance of the neurofuzzy controllers and their ability to remove offsets are demonstrated by two simulation examples involving a linear and a nonlinear system, and a case study involving the control of the drum water level in the boiler of a power generation system.     

Robust neurofuzzy rule base knowledge extraction and estimation using subspace decomposition combined with regularization and D-optimality.

A new robust neurofuzzy model construction algorithm has been introduced for the modeling of a priori unknown dynamical systems from observed finite data sets in the form of a set of fuzzy rules. Based on a Takagi-Sugeno (T-S) inference mechanism a one to one mapping between a fuzzy rule base and a model matrix feature subspace is established. This link enables rule based knowledge to be extracted from matrix subspace to enhance model transparency. In order to achieve maximized model robustness and sparsity, a new robust extended Gram-Schmidt (G-S) method has been introduced via two effective and complementary approaches of regularization and D-optimality experimental design. Model rule bases are decomposed into orthogonal subspaces, so as to enhance model transparency with the capability of interpreting the derived rule base energy level. A locally regularized orthogonal least squares algorithm, combined with a D-optimality used for subspace based rule selection, has been extended for fuzzy rule regularization and subspace based information extraction. By using a weighting for the D-optimality cost function, the entire model construction procedure becomes automatic. Numerical examples are included to demonstrate the effectiveness of the proposed new algorithm.     

A unified neurofuzzy model for classification

This work proposes a unified neurofuzzy modelling scheme. To begin with, the initial fuzzy base construction method is based on fuzzy clustering utilising a Gaussian mixture model (GMM) combined with the analysis of covariance (ANOVA) decomposition in order to obtain more compact univariate and bivariate membership functions over the subspaces of the input features. The mean and covariance of the Gaussian membership functions are found by the expectation maximisation (EM) algorithm with the merit of revealing the underlying density distribution of system inputs. The resultant set of membership functions forms the basis of the generalised fuzzy model (GFM) inference engine. The model structure and parameters of this neurofuzzy model are identified via the supervised subspace orthogonal least square (OLS) learning. Finally, instead of providing deterministic class label as model output by convention, a logistic regression model is applied to present the classifier’s output, in which the sigmoid type of logistic transfer function scales the outputs of the neurofuzzy model to the class probability. Experimental validation results are presented to demonstrate the effectiveness of the proposed neurofuzzy modelling scheme.     

Nature-inspired optimal tuning of input membership functions of Takagi-Sugeno-Kang fuzzy models for Anti-lock Braking Systems

This paper suggests a synergy of fuzzy logic and nature-inspired optimization in terms of the nature-inspired optimal tuning of the input membership functions of a class of Takagi-Sugeno-Kang (TSK) fuzzy models dedicated to Anti-lock Braking Systems (ABSs). A set of TSK fuzzy models is proposed by a novel fuzzy modeling approach for ABSs. The fuzzy modeling approach starts with the derivation of a set of local state-space models of the nonlinear ABS process by the linearization of the first-principle process model at ten operating points. The TSK fuzzy model structure and the initial TSK fuzzy models are obtained by the modal equivalence principle in terms of placing the local state-space models in the rule consequents of the TSK fuzzy models. An operating point selection algorithm to guide modeling is proposed, formulated on the basis of ranking the operating points according to their importance factors, and inserted in the third step of the fuzzy modeling approach. The optimization problems are defined such that to minimize the objective functions expressed as the average of squared modeling errors over the time horizon, and the variables of these functions are a part of the parameters of the input membership functions. Two representative nature-inspired algorithms, namely a Simulated Annealing (SA) algorithm and a Particle Swarm Optimization (PSO) algorithm, are implemented to solve the optimization problems and to obtain optimal TSK fuzzy models. The validation and the comparison of SA and PSO and of the new TSK fuzzy models are carried out for an ABS laboratory equipment. The real-time experimental results highlight that the optimized TSK fuzzy models are simple and consistent with both training data and validation data and that these models outperform the initial TSK fuzzy models.     

A computation-efficient on-line training algorithm for neurofuzzy networks

Neurofuzzy networks are often used to model linear or nonlinear processes, as they can provide some insights into the underlying processes and can be trained using experimental data. As the training of the networks involves intensive computation, it is often performed off line. However, it is well known that neurofuzzy networks trained off line may not be able to cope successully with time-varying processes. To overcome this problem, the weights of the networks are trained on line. In this paper, an on-line training algorithm with a computation time that is linear in the number of weights is derived by making full use of the local change property of neurofuzzy networks. It is shown that the estimated weights converge to that obtained from the least-squares method, and that the range of the input domain can be extended without retraining the network. Furthermore, it has a better ability in tracking time-varying systems than the recursive least-squares method, since in the proposed algorithm a positive definite submatrix is added to the relevant part of the covariance matrix. The performance of the proposed algorithm is illustrated by simulation examples and compared with that obtained using the recursive least-squares method.     

A Kernel-Based Two-Class Classifier for Imbalanced Data Sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm     

基于支持向量机的非线性离散动力学系统控制

提出了一种用支持向量机辨识系统状态空间模型的非线性离散动力学系统控制新方法. 在本方法中, 采用最小二乘支持向量机在每一个工作点辨识非线性系统的局部最优线性化模型. 针对该模型, 采用常规的线性控制方法在每个工作点设计局部线性控制器, 并在整个控制任务的每个工作点重复此设计过程.用该方法对两个典型的非线性离散系统采用极点配置技术进行了仿真验证, 结果显示系统对参考输入具有满意的跟踪性能, 证明该方法是有效和可行的.     

非线性动态系统神经模糊建模与内模/PID双重控制系统设计

非线性动态系统的内模控制要求建立精确的对象正模型和逆模型,这对于大多数实际对象是难以做到.提出了基于一类神经模糊模型的非线性动态系统建模方法,并在此基础上研究了基于神经模糊模型的非线性系统的内模控制设计.基于输入输出数据辨识的对象正模型和逆模型存在着模型失配问题,导致神经模糊内模控制范围变窄和控制鲁棒性降低,为了改善系统的性能,提出了神经模糊内模控制与PID控制结合的双重控制策略.对CSTR的反应物浓度控制研究表明,双重控制策略能有效地拓宽系统可控范围,改善系统性能.仿真结果证明该控制策略简单而有效.     

Sparse modeling using orthogonal forward regression with PRESS statistic and regularization

The paper introduces an efficient construction algorithm for obtaining sparse linear-in-the-weights regression models based on an approach of directly optimizing model generalization capability. This is achieved by utilizing the delete-1 cross validation concept and the associated leave-one-out test error also known as the predicted residual sums of squares (PRESS) statistic, without resorting to any other validation data set for model evaluation in the model construction process. Computational efficiency is ensured using an orthogonal forward regression, but the algorithm incrementally minimizes the PRESS statistic instead of the usual sum of the squared training errors. A local regularization method can naturally be incorporated into the model selection procedure to further enforce model sparsity. The proposed algorithm is fully automatic, and the user is not required to specify any criterion to terminate the model construction procedure. Comparisons with some of the existing state-of-art modeling methods are given, and several examples are included to demonstrate the ability of the proposed algorithm to effectively construct sparse models that generalize well.     

Developing soft sensors using hybrid soft computing methodology: a neurofuzzy system based on rough set theory and genetic algorithms

This paper presents a hybrid soft computing modeling approach, a neurofuzzy system based on rough set theory and genetic algorithms (GA). To solve the curse of dimensionality problem of neurofuzzy system, rough set is used to obtain the reductive fuzzy rule set. Both the number of condition attributes and rules are reduced. Genetic algorithm is used to obtain the optimal discretization of continuous attributes. The fuzzy system is then represented via an equivalent artificial neural network (ANN). Because the initial parameter of the ANN is reasonable, the convergence of the ANN training is fast. After the rules are reduced, the structure size of the ANN becomes small, and the ANN is not fully weight-connected. The neurofuzzy approach based on RST and GA has been applied to practical application of building a soft sensor model for estimating the freezing point of the light diesel fuel in fluid catalytic cracking unit.     

Frequency domain subspace identification of a tape servo system

As capacity demands for magnetic tape storage systems grow, servo actuator design for tracking data on high density tape media presents new modeling and control design challenges. In this paper a frequency weighted subspace identification algorithm is presented for control relevant model estimation of a tape servo actuator. Common to other subspace identification methods, the proposed algorithm is based on linear algebra techniques providing means for model order selection and model computation. The proposed subspace identification also allows for frequency dependent weightings to emphasize frequency data around the cross-over frequency to find models relevant for control design. The algorithm is applied to data obtained from a tape storage device, demonstrating model order selection and the estimation of servo actuator dynamics with control relevant model fit criteria.     

Tensor Product‐Based Model Transformation and Optimal Controller Design for High Order Nonlinear Singularly Perturbed Systems

This paper investigates the optimization of a general class of nonlinear singularly perturbed systems. Tensor product‐based modeling, algebraic properties, and singular perturbation theory played a significant role in the formulation of the derived reduced model. The obtained reduced output is a nonlinear state and input dependent vector. In this case, solving the optimal control problem when minimizing a quadratic performance index becomes more difficult because the weighting matrices vary as functions of states. So the reformulation of the initial problem is needed and the resolution is discussed from an analytical point of view. A two‐step controller design procedure is suggested and an algorithm is proposed for the calculus of the gain matrices of the nonlinear control law. The Simulation results are presented to demonstrate the effectiveness of the approach and the power of the implemented algorithm.     

Identification of Neurofuzzy Models Using GTLS Parameter Estimation

In this paper, nonlinear system identification utilizing generalized total least squares (GTLS) methodologies in neurofuzzy systems is addressed. The problem involved with the estimation of the local model parameters of neurofuzzy networks is the presence of noise in measured data. When some or all input channels are subject to noise, the GTLS algorithm yields consistent parameter estimates. In addition to the estimation of the parameters, the main challenge in the design of these local model networks is the determination of the region of validity for the local models. The method presented in this paper is based on an expectation–maximization algorithm that uses a residual from the GTLS parameter estimation for proper partitioning. The performance of the resulting nonlinear model with local parameters estimated by weighted GTLS is a product both of the parameter estimation itself and the associated residual used for the partitioning process. The applicability and benefits of the proposed algorithm are demonstrated by means of illustrative examples and an automotive application.      

Multiple model bank selection based on nonlinearity measure and H-gap metric

This paper provides a systematic method for model bank selection in multi-linear model analysis for nonlinear systems by presenting a new algorithm which incorporates a nonlinearity measure and a modified gap based metric. This algorithm is developed for off-line use, but can be implemented for on-line usage. Initially, the nonlinearity measure analysis based on the higher order statistic (HOS) and the linear cross correlation methods are used for decomposing the total operating space into several regions with linear models. The resulting linear models are then used to construct the primary model bank. In order to avoid unnecessary linear local models in the primary model bank, a gap based metric is introduced and applied in order to merge similar linear local models. In order to illustrate the usefulness of the proposed algorithm, two simulation examples are presented: a pH neutralization plant and a continuous stirred tank reactor (CSTR).     

Genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this paper, we propose and investigate a new category of neurofuzzy networks—fuzzy polynomial neural networks (FPNN) endowed with fuzzy set-based polynomial neurons (FSPNs) We develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms (GAs) in particular. The conventional FPNNs developed so far are based on the mechanisms of self-organization, fuzzy neurocomputing, and evolutionary optimization. The design of the network exploits the FSPNs as well as the extended group method of data handling (GMDH). Let us stress that in the previous development strategies some essential parameters of the networks (such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables) being available within the network are provided by the designer in advance and kept fixed throughout the overall development process. This restriction may hamper a possibility of developing an optimal architecture of the model. The design proposed in this study addresses this issue. The augmented and genetically developed FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional FPNNs. The GA-based design procedure being applied at each layer of the FPNN leads to the selection of the most suitable nodes (or FSPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation in which we use a number of modeling benchmarks—synthetic and experimental data being commonly used in fuzzy or neurofuzzy modeling. The obtained results demonstrate the superiority of the proposed networks over the models existing in the references.     

Self-organizing neurofuzzy networks based on evolutionary fuzzy granulation

Experimental software datasets describing software projects in terms of their complexity and development time have been a subject of intensive modeling. A number of various modeling methodologies and modeling designs have been proposed including such development frameworks as neural networks, fuzzy and neurofuzzy models. In this study, we introduce a concept of self-organizing neurofuzzy networks (SONFN), a hybrid modeling architecture combining neurofuzzy networks (NFN) and polynomial neural networks (PNN). For these networks we develop a comprehensive design methodology. The construction of SONFNs takes advantage of the well-established technologies of computational intelligence (CI), namely fuzzy sets, neural networks and genetic algorithms. The architecture of the SONFN results from a synergistic usage of NFNs and PNNs. NFN contributes to the formation of the premise part of the rule-based structure of the SONFN. The consequence part of the SONFN is designed using PNNs. We discuss two types of SONFN architectures whose taxonomy is based on the NFN scheme being applied to the premise part of SONFN. We introduce a comprehensive learning algorithm. It is shown that this network exhibits a dynamic structure as the number of its layers as well as the number of nodes in each layer of the SONFN are not predetermined (as this is the case in a popular topology of a multilayer perceptron). The experimental results include a well-known NASA dataset concerning software cost estimation.     

Improved feature selection algorithm with fuzzy-rough sets on compact computational domain

The aim of this paper is to provide an efficient input feature selection algorithm for modeling of systems based on modified definition of fuzzy-rough sets. Some of the critical issues concerning the complexity and convergence of the feature selection algorithm are discussed in detail. Based on some natural properties of fuzzy t-norm and t-conorm operators, the concept of fuzzy-rough sets on compact computational domain is put forward, which is then utilized to construct improved Fuzzy-Rough Feature Selection algorithm. Various mathematical properties of this new definition of fuzzy-rough sets are discussed from pattern classification viewpoint. Speedup factor as high as 622 has been achieved with proposed algorithm compared to recently proposed FRSAR, with improved model performance on selected set of features.