Shrinkage estimation in general linear models

We propose a James-Stein-type shrinkage estimator for the parameter vector in a general linear model when it is suspected that some of the parameters may be restricted to a subspace. The James-Stein estimator is shown to demonstrate asymptotically superior risk performance relative to the conventional least squares estimator under quadratic loss. An extensive simulation study based on a multiple linear regression model and a logistic regression model further demonstrates the improved performance of this James-Stein estimator in finite samples. The application of this new estimator is illustrated using Ontario newborn infants data spanning four fiscal years.     

On resampling and uncertainty estimation in Linear System Identification

Linear System Identification yields a nominal model parameter, which minimizes a specific criterion based on the single input-output data set. Here we investigate the utility of various methods for estimating the probability distribution of this nominal parameter using only the data from this single experiment. The results are compared to the actual parameter distribution generated by many Monte Carlo runs of the data-collection experiment. The methods considered are collectively known as resampling schemes, which include Subsampling, the Jackknife, and the Bootstrap. The broad aim is to generate an empirical parameter distribution function via the construction of a large number of new data records from the original single set of data, based on an assumption that this data is representative of all possible data, and then to run the parameter estimator on each of these new records to develop the distribution function. The performance of these schemes is evaluated on a difficult, almost unidentifiable system, and compared to the standard results based on asymptotic normality. In addition to the exploration of this example as a means to evaluate the strengths and weaknesses of these resampling schemes, some new theoretical results are proven and demonstrated for Subsampling schemes.     

A Kernel-Based Two-Class Classifier for Imbalanced Data Sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm     

Maximum Likelihood Estimation of the Cox–Ingersoll–Ross Model Using Particle Filters

This paper shows how to build in a computationally efficient way a maximum simulated likelihood procedure to estimate the Cox–Ingersoll–Ross model from multivariate time series. The advantage of this estimator is that it takes into account the exact likelihood function while avoiding the huge computational burden associated with MCMC methods and without the ad hoc assumption that certain bond yields are measured without error. The proposed methodology is implemented and tested on simulated data. For realistic parameter values the estimator seems to have good small sample properties, compared to the popular quasi maximum likelihood approach, even using moderate simulation sizes. The effect of simulation errors does not seem to undermine the estimation procedure.     

自校正解耦信息融合Wiener状态估值器

对含未知噪声方差阵的多传感器系统,用现代时间序列分析方法.基于滑动平均(MA)新息模型的在线辨识和求解相关函数矩阵方程组,可得到估计噪声方差阵估值器,进而在按分量标量加权线性最小方差最优信息融合则下,提出了自校正解耦信息融合Wiener状态估值器.它的精度比每个局部自校正Wiener状态估值器精度高.它实现了状态分量的解耦局部Wiener估值器和解耦融合Wiener估值器.证明了它的收敛性,即若MA新息模型参数估计是一致的,则它将收敛于噪声统计已知时的最优解耦信息融合Wiener状态估值器,因而它具有渐近最优性.一个带3传感器的目标跟踪系统的仿真例子说明了其有效性.     

Development of moving window state and parameter estimators under maximum likelihood and Bayesian frameworks

Estimation of slowly varying model parameters/unmeasured disturbances is of paramount importance in process monitoring, fault diagnosis, model based advanced control and online optimization. The conventional approach to estimate drifting parameters is to artificially model them as a random walk process and estimate them simultaneously with the states. However, this may lead to a poorly conditioned problem, where the tuning of the random walk model becomes a non-trivial exercise. In this work, the moving window parameter estimator of Huang et al. [1] is recast as a moving window maximum likelihood (ML) estimator. The state can be estimated within the window using any recursive Bayesian estimator. It is assumed that, when the model parameters are perfectly known, the innovation sequence generated by the chosen Bayesian estimator is a Gaussian white noise process and is further used to construct a likelihood function that treats the model parameters as unknowns. This leads to a well conditioned problem where the only tuning parameter is the length of the moving window, which is much easier to select than selecting the covariance of the random walk model. The ML formulation is further modified to develop a maximum a posteriori (MAP) cost function by including arrival cost for the parameter. Efficacy of the proposed ML and MAP formulations has been demonstrated by conducting simulation studies and experimental evaluation. Analysis of the simulation and experimental results reveals that the proposed moving window ML and MAP estimators are capable of tracking the drifting parameters/unmeasured disturbances fairly accurately even when the measurements are available at multiple rates and with variable time delays.     

Moving data window gradient‐based iterative algorithm of combined parameter and state estimation for bilinear systems

The combined iterative parameter and state estimation problem is considered for bilinear state‐space systems with moving average noise in this paper. There are the product terms of state variables and control variables in bilinear systems, which makes it difficult for the parameter and state estimation. By designing a bilinear state estimator based on the Kalman filtering, the states are estimated using the input‐output data. Furthermore, a moving data window (MDW) is introduced, which can update the dynamical data by removing the oldest data and adding the newest measurement data. A state estimator‐based MDW gradient‐based iterative (MDW‐GI) algorithm is proposed to estimate the unknown states and parameters jointly. Moreover, given the extended gradient‐based iterative (EGI) algorithm as a comparison, the MDW‐GI algorithm can reduce the impact of noise to parameter estimation and improve the parameter estimation accuracy. The numerical simulation examples validate the effectiveness of the proposed algorithm.     

Self-tuning weighted measurement fusion white noise deconvolution estimator and its convergence analysis

White noise deconvolution or input white noise estimation has a wide range of applications including oil seismic exploration, communication, signal processing, and state estimation. For the multisensor linear discrete time-invariant stochastic systems with correlated measurement noises, and with unknown ARMA model parameters and noise statistics, the on-line AR model parameter estimator based on the Recursive Instrumental Variable (RIV) algorithm, the on-line MA model parameter estimator based on Gevers–Wouters algorithm and the on-line noise statistic estimator by using the correlation method are presented. Using the Kalman filtering method, a self-tuning weighted measurement fusion white noise deconvolution estimator is presented based on the self-tuning Riccati equation. It is proved that the self-tuning fusion white noise deconvolution estimator converges to the optimal fusion steady-state white noise deconvolution estimator in a realization by using the dynamic error system analysis (DESA) method, so that it has the asymptotic global optimality. The simulation example for a 3-sensor system with the Bernoulli–Gaussian input white noise shows its effectiveness.     

Model based bootstrap methods for interval censored data

The performance of model based bootstrap methods for constructing point-wise confidence intervals around the survival function with interval censored data is investigated. It is shown that bootstrapping from the nonparametric maximum likelihood estimator of the survival function is inconsistent for the current status model. A model based smoothed bootstrap procedure is proposed and proved to be consistent. In fact, a general framework for proving the consistency of any model based bootstrap scheme in the current status model is established. In addition, simulation studies are conducted to illustrate the (in)-consistency of different bootstrap methods in mixed case interval censoring. The conclusions in the interval censoring model would extend more generally to estimators in regression models that exhibit non-standard rates of convergence.     

Tuning SISO offset-free Model Predictive Control based on ARX models

In this paper, we present a tuning methodology for a simple offset-free SISO Model Predictive Controller (MPC) based on autoregressive models with exogenous inputs (ARX models). ARX models simplify system identification as they can be identified from data using convex optimization. Furthermore, the proposed controller is simple to tune as it has only one free tuning parameter. These two features are advantageous in predictive process control as they simplify industrial commissioning of MPC. Disturbance rejection and offset-free control is important in industrial process control. To achieve offset-free control in face of unknown disturbances or model-plant mismatch, integrators must be introduced in either the estimator or the regulator. Traditionally, offset-free control is achieved using Brownian disturbance models in the estimator. In this paper we achieve offset-free control by extending the noise model with a filter containing an integrator. This filter is a first order ARMA model. By simulation and analysis, we argue that it is independent of the parameterization of the underlying linear plant; while the tuning of traditional disturbance models is system dependent. Using this insight, we present MPC for SISO systems based on ARX models combined with the first order filter. We derive expressions for the closed-loop variance of the unconstrained MPC based on a state space representation in innovation form and use these expressions to develop a tuning procedure for the regulator. We establish formal equivalence between GPC and state space based off-set free MPC. By simulation we demonstrate this procedure for a third order system. The offset-free ARX MPC demonstrates satisfactory set point tracking and rejection of an unmeasured step disturbance for a simulated furnace with a long time delay.     

A dynamic state estimator for power system dynamic security assessment

A global modularized dynamic state estimator is formulated to provide the data which will be required for future dynamic security assessment and dynamic security enhancement applications. The dynamic state estimator is global because it is capable of estimating small and large dynamic fluctuations in voltage angle and frequency for an entire area. The dynamic state estimator is composed of the sum of the static state estimate, obtained by using present hardware and algorithms and a modularized dynamic state estimate based on a linearized classical transient stability model with a stochastic load model. This dynamic state estimate component is modularized to (1) eliminate the need to measure or model external system generation and (2) to permit a reduction in computation requirements for (a) updating the linearized power system dynamic model and (b) for computing the state estimate. The modularization, which is accomplished by decoupling the linearized dynamic model for each subregion by measuring the power injections on lines connecting the subregion to the rest of the power system, causes the dynamic state estimate to be locally referenced. A global referencing procedure is proposed and discussed. A linearized stochastic model for the Michigan Electric Coordinated System is developed to illustrate the procedures proposed for developing the stochastic load model and determining the constant gain approximation for the governor turbine energy system dynamics. A summary of results on the performance of the Kalman state estimator is presented.     

Absolute penalty and shrinkage estimation in partially linear models

In the context of a partially linear regression model, shrinkage semiparametric estimation is considered based on the Stein-rule. In this framework, the coefficient vector is partitioned into two sub-vectors: the first sub-vector gives the coefficients of interest, i.e., main effects (for example, treatment effects), and the second sub-vector is for variables that may or may not need to be controlled. When estimating the first sub-vector, the best estimate may be obtained using either the full model that includes both sub-vectors, or the reduced model which leaves out the second sub-vector. It is demonstrated that shrinkage estimators which combine two semiparametric estimators computed for the full model and the reduced model outperform the semiparametric estimator for the full model. Using the semiparametric estimate for the reduced model is best when the second sub-vector is the null vector, but this estimator suffers seriously from bias otherwise. The relative dominance picture of suggested estimators is investigated. In particular, suitability of estimating the nonparametric component based on the B-spline basis function is explored. Further, the performance of the proposed estimators is compared with an absolute penalty estimator through Monte Carlo simulation. Lasso and adaptive lasso were implemented for simultaneous model selection and parameter estimation. A real data example is given to compare the proposed estimators with lasso and adaptive lasso estimators.     

Parameter estimator suitable for some bad samples

A parameter estimator is proposed based on the concept of two-objective optimization, which may be much better than the best linear unbiased estimator in the case of some bad samples, such as a very small sample or an approximately multi-collinear sample. This is illustrated by certain theoretical analyses and simulation studies. A comparison of this estimator with other related ones is given to show the new features of our approach.     

A semi-implicit approach for fast parameter estimation by means of the extended Kalman filter

An assessment is performed of the impact of the time-integration scheme on data assimilation based on the extended Kalman filter approach. The usage of the implicit Euler scheme, which leads to unconditionally stable model updates for the whole range of time steps, results in a less accurate estimation of model parameters, which can be overcome using a parameter estimator based on the explicit Euler scheme. However, the latter limits the maximum time step which may lead to an unacceptable increase in computational load. The alternative can be found using a semi-implicit estimator, where a model is updated using the implicit Euler scheme, whereas the propagation of the error covariance matrix in time is based on the explicit time integration scheme. The improved properties of the proposed algorithm are demonstrated on a one-dimensional problem of permeability estimation in a porous medium for a single phase oil flow.     

Improved Liu-type estimator in partial linear model

In this article, a Liu-type estimation is proposed for the vector-parameter in a partial linear model. This new estimator can be regarded as generalization of the restricted least-squares estimator, the restricted ridge estimator and the restricted Liu estimator. We also obtain the asymptotic distributional bias and risk of these estimators and we also discuss some properties of the new estimator. The selection of the tuning parameter in the proposed estimator is also presented. Finally, a simulation study is presented to explain the performance of the new estimator.     

一类不规则损失输出数据系统的自校正PID控制算法

针对输出误差模型描述的不规则损失输出数据系统,提出一种自校正PID控制算法。首先结合辅助模型辨识思想和最小二乘原理设计参数估计器,在线估计系统参数,并设计一个损失输出估计器来估算采样间损失输出,使控制系统得到一个具有与期望输出相同采样周期的反馈信号;再根据系统参数的估计值实时调整自校正PID控制器参数,使系统实际输出跟踪期望输出,实现自校正PID控制。仿真结果验证了该算法的有效性。     

Robust accelerated failure time regression

Robust estimators for accelerated failure time models with asymmetric (or symmetric) error distribution and censored observations are proposed. It is assumed that the error model belongs to a log-location-scale family of distributions and that the mean response is the parameter of interest. Since scale is a main component of mean, scale is not treated as a nuisance parameter. A three steps procedure is proposed. In the first step, an initial high breakdown point S estimate is computed. In the second step, observations that are unlikely under the estimated model are rejected or down weighted. Finally, a weighted maximum likelihood estimate is computed. To define the estimates, functions of censored residuals are replaced by their estimated conditional expectation given that the response is larger than the observed censored value. The rejection rule in the second step is based on an adaptive cut-off that, asymptotically, does not reject any observation when the data are generated according to the model. Therefore, the final estimate attains full efficiency at the model, with respect to the maximum likelihood estimate, while maintaining the breakdown point of the initial estimator. Asymptotic results are provided. The new procedure is evaluated with the help of Monte Carlo simulations. Two examples with real data are discussed.     

一种基于小生境遗传算法的迟滞非线性系统参数识别方法

工程中存在着大量的具有迟滞非线性恢复力的结构与构件,但迟滞非线性系统既是非线性的,又是非解析的,造成其参数识别十分困难,阻碍了迟滞非线性模型在工程中的应用．本文提出了一种基于小生境遗传算法的迟滞非线性系统参数识别方法,该方法在遗传算法中引入了新的参数——个体活动半径．利用本算法对一木结构剪力墙的BW模型参数进行识别,识别结果误差较小,验证了算法的有效性。     

Fixed-interval smoothing for a linear distributed parameter system

The fixed-interval smoothing estimator for a linear distributed parameter system with a noisy observation at discrete points on the spatial domain or its boundary is derived from the Wiener-Hopf theoretical viewpoint. The present derivation of the smoothing estimator is based on the Wiener-Hopf theory. Thus, using the Wiener-Hopf equations both for the optimal smoothing problem and for the optimal filtering problem, the optimal fixed-interval smoothing estimator which corresponds to the results obtained by using Kalman's limiting procedure is derived. Finally, for the convenience of the numerical computation, we rewrite the results for the optimal fixed-interval smoothing estimator and the error covariance function by using the Fourier expansion method.     

Nonlinear system identification using two-dimensional wavelet-based state-dependent parameter models

This article presents a nonlinear system identification approach that uses a two-dimensional (2-D) wavelet-based state-dependent parameter (SDP) model. In this method, differing from our previous approach, the SDP is a function with respect to two different state variables, which is realised by the use of a 2-D wavelet series expansion. Here, an optimised model structure selection is accomplished using a PRESS-based procedure in conjunction with orthogonal decomposition (OD) to avoid any ill-conditioning problems associated with the parameter estimation. Two simulation examples are provided to demonstrate the merits of the proposed approach.