M-estimator and D-optimality model construction using orthogonal forward regression.

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.     

Robust neurofuzzy rule base knowledge extraction and estimation using subspace decomposition combined with regularization and D-optimality.

A new robust neurofuzzy model construction algorithm has been introduced for the modeling of a priori unknown dynamical systems from observed finite data sets in the form of a set of fuzzy rules. Based on a Takagi-Sugeno (T-S) inference mechanism a one to one mapping between a fuzzy rule base and a model matrix feature subspace is established. This link enables rule based knowledge to be extracted from matrix subspace to enhance model transparency. In order to achieve maximized model robustness and sparsity, a new robust extended Gram-Schmidt (G-S) method has been introduced via two effective and complementary approaches of regularization and D-optimality experimental design. Model rule bases are decomposed into orthogonal subspaces, so as to enhance model transparency with the capability of interpreting the derived rule base energy level. A locally regularized orthogonal least squares algorithm, combined with a D-optimality used for subspace based rule selection, has been extended for fuzzy rule regularization and subspace based information extraction. By using a weighting for the D-optimality cost function, the entire model construction procedure becomes automatic. Numerical examples are included to demonstrate the effectiveness of the proposed new algorithm.     

Nonlinear model structure design and construction using orthogonal least squares and D-optimality design

A very efficient learning algorithm for model subset selection is introduced based on a new composite cost function that simultaneously optimizes the model approximation ability and model robustness and adequacy. The derived model parameters are estimated via forward orthogonal least squares, but the model subset selection cost function includes a D-optimality design criterion that maximizes the determinant of the design matrix of the subset to ensure the model robustness, adequacy, and parsimony of the final model. The proposed approach is based on the forward orthogonal least square (OLS) algorithm, such that new D-optimality-based cost function is constructed based on the orthogonalization process to gain computational advantages and hence to maintain the inherent advantage of computational efficiency associated with the conventional forward OLS approach. Illustrative examples are included to demonstrate the effectiveness of the new approach.     

Double objective optimal multivariable ripple-free deadbeat control

This paper presents novel results on optimal multivariable deadbeat control. Given a discrete-time, stable, linear, time invariant plant model, we give a simple parameterization of all stabilizing ripple-free deadbeat controllers of a given order. The free parameter is then optimized in the sense that a quadratic index is kept minimal. The optimality criterion has the advantage of accounting for both tracking performance and magnitude of the control effort. The proposed design procedure is simple to use and allows the tuning of the controller with a scalar weighting factor. Simulation results are included to illustrate the effectiveness of the proposed design algorithm.     

Identification of weighting function using deterministic input

In this paper, a procedure for identifying the weighting function of a system is proposed. The procedure uses a new type of criterion involving a priori information about the weighting function as well as the integral of the squared response error. The optimal weighting function applied in this criterion can be determined by solving the optimal tracking problem, provided that the output of a linear, free dynamical system is supplied to the plant as an input. The criterion function is also valid from the estimation theory. The computational algorithm of the procedure is presented herein. Numerical computations of several exampes have shown that the proposed procedure gives satisfactory results even for the short-time observed data.     

A neurofuzzy network knowledge extraction and extended Gram-Schmidt algorithm for model subspace decomposition

This paper introduces a new neurofuzzy model construction and parameter estimation algorithm from observed finite data sets, based on a Takagi-Sugeno (T-S) inference mechanism and a new extended Gram-Schmidt orthogonal decomposition algorithm, for the modeling of a priori unknown dynamical systems in the form of a set of fuzzy rules. The paper introduces a one to one mapping between a fuzzy rule-base and a model matrix feature subspace. Hence, rule-based knowledge can be extracted to enhance model transparency. Model transparency is explored by the derivation of an equivalence between an A-optimality experimental design criterion of the weighting matrix and the average model output sensitivity to the fuzzy rule. The A-optimality experimental design criterion of the weighting matrices of fuzzy rules is used to construct an initial model rule-base. An extended Gram-Schmidt algorithm is then developed to estimate the parameter vector for each rule. This new algorithm decomposes the model rule-bases via an orthogonal subspace decomposition approach, so as to enhance model transparency with the capability of interpreting the derived rule-base energy level.     

A Kernel-Based Two-Class Classifier for Imbalanced Data Sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm     

面向小目标图像的快速核密度估计图像阈值分割算法

针对当前小目标图像阈值分割研究工作面临的难题,提出了快速核密 度估计图像阈值分割新方法.首先给出了基于加权核密度估计器的概率计算模 型,通过引入二阶Renyi熵作为阈值选取准则,提出了基于核密度估计的图像阈 值分割算法 (Kernel density estimator based image thresholding algorithm, KDET), 然后通过引入快速压缩集密度估计 (Fast reduced set density estimator, FRSDE)技术,得到核密度估计的 稀疏权系数表示形式,提出快速核密度估计图像阈值分割算法fastKDET,并从 理论上对相关性质进行了深入探讨.实验表明,本文算法对小目标图像 阈值分割问题具有更广泛的适应性,并且对参数变化不敏感.     

Using the response surface methodology and the D-optimality criterion in crashworthiness related problems

The aim of this paper is to determine the efficient number of experimental points when using the response surface methodology in crashworthiness problems. The D-optimality criterion is used as experimental design method. Two application models have been studied, one square tube and one front rail from Saab Automobile AB. Both models were fully parameterized in the preprocessor LS-INGRID but only two design variables were used. The optimization package LS-OPT was used to determine the design of experiments using the D-optimality criterion. Both models were subjected to an impact into a rigid wall and the simulations were carried out using LS-DYNA. A general recommendation is to to use 1.5 times the minimum number of experimental points. A more specialized recommendation is for linear surfaces 1.5, elliptic surfaces 2.2 and for quadratic surfaces 1.6 times the minimum number of experimental points.     

Training data development with the D-optimality criterion

The importance of using optimum experimental design (OED) concepts when selecting data for training a neural network is highlighted in this paper. We demonstrate that an optimality criterion borrowed from another field; namely the D-optimality criterion used in OED, can be used to enhance the training value of a small training data set. This is important in cases where resources are limited, and collecting data is expensive, hazardous, or time consuming. The analysis results in the cases considered indicate that even with a small set of training examples, so long as the training data set was chosen according to the D-optimality criterion, the network was able to generalize, and as a result, was able to fit complex surfaces     

Robust nonlinear model identification methods using forward regression

In this correspondence new robust nonlinear model construction algorithms for a large class of linear-in-the-parameters models are introduced to enhance model robustness via combined parameter regularization and new robust structural selective criteria. In parallel to parameter regularization, we use two classes of robust model selection criteria based on either experimental design criteria that optimizes model adequacy, or the predicted residual sums of squares (PRESS) statistic that optimizes model generalization capability, respectively. Three robust identification algorithms are introduced, i.e., combined A- and D-optimality with regularized orthogonal least squares algorithm, respectively; and combined PRESS statistic with regularized orthogonal least squares algorithm. A common characteristic of these algorithms is that the inherent computation efficiency associated with the orthogonalization scheme in orthogonal least squares or regularized orthogonal least squares has been extended such that the new algorithms are computationally efficient. Numerical examples are included to demonstrate effectiveness of the algorithms.     

径向基函数神经网络的一种两级学习方法

建立RBF(radial basis function)神经网络模型关键在于确定网络隐中心向量、基宽度参数和隐节点数.为设计结构简单,且具有良好泛化性能径向基网络结构,本文提出了一种RBF网络的两级学习新设计方法.该方法在下级由正则化正交最小二乘法与D-最优试验设计结合算法自动构建结构节俭的RBF网络模型;在上级通过粒子群优化算法优选结合算法中影响网络泛化性能的3个学习参数,即基宽度参数、正则化系数和D-最优代价系数的最佳参数组合.仿真实例表明了该方法的有效性.     

Non-diagonal H∞ weighting function design: Exploiting spatio-temporal deformations in precision motion control

Model-based control design requires a careful specification of performance and robustness requirements. In typical norm-based control designs, performance and robustness requirements are specified in a scalar optimization criterion, even for complex multivariable systems. This paper aims to develop a novel approach for the formulation of this optimization criterion for multivariable motion systems that exhibit spatio-temporal deformations. To achieve this, characteristics of the underlying system are exploited to design multivariable weighting functions. In contrast to pre-existing approaches, which typically lead to diagonal weighting functions, the proposed approach enables the design of non-diagonal weighting functions. Extensive experimental results confirm that the proposed procedure can significantly improve the performance of an industrial motion system compared to earlier approaches.     

Toward the training of feed-forward neural networks with the D-optimum input sequence

The problem under consideration is to obtain a measurement schedule for training neural networks. This task is perceived as an experimental design in a given design space that is obtained in such a way as to minimize the difference between the neural network and the system being considered. This difference can be expressed in many different ways and one of them, namely, the D-optimality criterion is used in this paper. In particular, the paper presents a unified and comprehensive treatment of this problem by discussing the existing and previously unpublished properties of the optimum experimental design (OED) for neural networks. The consequences of the above properties are discussed as well. A hybrid algorithm that can be used for both the training and data development of neural networks is another important contribution of this paper. A careful analysis of the algorithm is presented and its comprehensive convergence analysis with the help of the Lyapunov method are given. The paper contains a number of numerical examples that justify the application of the OED theory for neural networks. Moreover, an industrial application example is given that deals with the valve actuator.     

Integrated feature architecture selection

In this paper, we present an integrated approach to feature and architecture selection for single hidden layer-feedforward neural networks trained via backpropagation. In our approach, we adopt a statistical model building perspective in which we analyze neural networks within a nonlinear regression framework. The algorithm presented in this paper employs a likelihood-ratio test statistic as a model selection criterion. This criterion is used in a sequential procedure aimed at selecting the best neural network given an initial architecture as determined by heuristic rules. Application results for an object recognition problem demonstrate the selection algorithm's effectiveness in identifying reduced neural networks with equivalent prediction accuracy.     

Design of sensitivity-reducing compensators using observers

In this paper the design of compensators for output feedback systems which satisfy a sensitivity, reduction criterion is considered for the case when the dynamic compensator is an observer. Using the conditions for comparison sensitivity design of output feedback systems derived by Naeije and Bosgra [4], it is shown that for arbitrary stable state-observers (full or reduced order) there exist feedback gains multiplying the system output and observer states for which the system is stable and the sensitivity reduction criterion is satisfied. Use of observers enables direct control over some of the feedback system eigenvalues and leads to a useful interpretation of the sensitivity weighting matrix. A design procedure is described and illustrated by an example of an aircraft control. The results are graphically compared with results obtained by a conventional design procedure.     

A control synthesis methodology for uncertain plants with random initial conditions

This paper proposes a controller design procedure to satisfy practical control specifications for uncertain plants with random initial conditions. A discrete-time linear optimal controller structure is employed and a standard mathematical programming algorithm is used to compute the slate weighting matrix Q such that the specifications are satisfied. The design procedure is demonstrated via an example dealing with the design of a flight control system     

Robust LQG optimal controller design for multivariable discrete saturating systems with noise spectral uncertainties and non-linear time-varying unmodelled dynamics

A design procedure is proposed for robust linear-quadratic-gaussian (LQG) optimal controller synthesis against noise spectral uncertainties, non-linear time-varying (NLTV) unmodelled dynamics in discrete saturating systems. A robust stability criterion is derived for multivariable stochastic discrete-time systems with NLTV unmodelled dynamics and constrained actuators. An algorithm based on the robust stabilization creterion is presented for synthesing a robust controller not only to minimize the least favourable cost functional but also to satisfy the robust stabilization criterion by specifying an appropriate weighting scalar in the cost functional. A necessary and sufficient condition for the solvability of such a robust stabilization problem is derived by means of the Nevanlinna-Pick interpolation theory. The Wiener Z-domain solution for controller synthesis, the saddle point theory, and the properties of Schur operator (Class S) are employed to treat this problem. Finally, a numerical example is given to illustrate the results.     

Two-Stage Method for Diagonal Recurrent Neural Network Identification of a High-Power Continuous Microwave Heating System

This paper proposes a diagonal recurrent neural network (DRNN) based identification scheme to handle the complexity and nonlinearity of high-power continuous microwave heating system (HPCMHS). The new DRNN design involves a two-stage training process that couples an efficient forward model selection technique with gradient-based optimization. In the first stage, an impact recurrent network structure is obtained by a fast recursive algorithm in a stepwise forward procedure. To ensure stability, update rules are further developed using Lyapunov stability criterion to tune parameters of reduced size model at the second stage. The proposed approach is tested with an experimental regression problem and a practical HPCMHS identification, and the results are compared with four typical network models. The results show that the new design demonstrates improved accuracy and model compactness with reduced computational complexity over the existing methods.

     

Multi-objective design of state feedback controllers using reinforced quantum-behaved particle swarm optimization

A novel and generic multi-objective design paradigm is proposed which utilizes quantum-behaved PSO (QPSO) for deciding the optimal configuration of the LQR controller for a given problem considering a set of competing objectives. There are three main contributions introduced in this paper as follows. (1) The standard QPSO algorithm is reinforced with an informed initialization scheme based on the simulated annealing algorithm and Gaussian neighborhood selection mechanism. (2) It is also augmented with a local search strategy which integrates the advantages of memetic algorithm into conventional QPSO. (3) An aggregated dynamic weighting criterion is introduced that dynamically combines the soft and hard constraints with control objectives to provide the designer with a set of Pareto optimal solutions and lets her to decide the target solution based on practical preferences. The proposed method is compared against a gradient-based method, seven meta-heuristics, and the trial-and-error method on two control benchmarks using sensitivity analysis and full factorial parameter selection and the results are validated using one-tailed T-test. The experimental results suggest that the proposed method outperforms opponent methods in terms of controller effort, measures associated with transient response and criteria related to steady-state.