On the estimation of the linear relation when the error variances are known

The problem of consistent estimation in measurement error models in a linear relation with not necessarily normally distributed measurement errors is considered. Three possible estimators which are constructed as different combinations of the estimators arising from direct and inverse regression are considered. The efficiency properties of these three estimators are derived and the effect of non-normally distributed measurement errors is analyzed. A Monte-Carlo experiment is conducted to study the performance of these estimators in finite samples.     

Fuzzy entropy in fuzzy weighted linear regression model under linear restrictions with simulation study

In fuzzy set theory, it is well known that a fuzzy number can be uniquely determined through its position and entropy. Hence, by using the concept of fuzzy entropy the estimators of the fuzzy regression coefficients may be estimated. In the present communication, a fuzzy linear regression (FLR) model with some restrictions in the form of prior information has been considered. The estimators of regression coefficients have been obtained with the help of fuzzy entropy for the restricted/unrestricted FLR model by assigning some weights in the distance function. Some numerical examples have also been provided in order to illustrate the proposed model along with the obtained weighted estimators. Further, in order to compare the performance of unrestricted estimator and restricted estimator, a simulation study has been conducted by using two fundamental criteria of dominance – mean squared error matrix (MSEM) and absolute bias.     

Parametric (modified least squares) and non-parametric (Theil-Sen) linear regressions for predicting biophysical parameters in the presence of measurement errors

Remote sensing often involves the estimation of in situ quantities from remote measurements. Linear regression, where there are no non-linear combinations of regressors, is a common approach to this prediction problem in the remote sensing community. A review of recent remote sensing articles using univariate linear regression indicates that in the majority of cases, ordinary least squares (OLS) linear regression has been applied, with approximately half the articles using the in situ observations as regressors and the other half using the inverse regression with remote measurements as regressors. OLS implicitly assume an underlying normal structural data model to arrive at unbiased estimates of the response. OLS regression can be a biased predictor in the presence of measurement errors when the regression problem is based on a functional rather than structural data model. Parametric (Modified Least Squares) and non-parametric (Theil-Sen) consistent predictors are given for linear regression in the presence of measurement errors together with analytical approximations of their prediction confidence intervals. Three case studies involving estimation of leaf area index from nadir reflectance estimates are used to compare these unbiased estimators with OLS linear regression. A comparison to Geometric Mean regression, a standardized version of Reduced Major Axis regression, is also performed. The Theil-Sen approach is suggested as a potential replacement of OLS for linear regression in remote sensing applications. It offers simplicity in computation, analytical estimates of confidence intervals, robustness to outliers, testable assumptions regarding residuals and requires limited a priori information regarding measurement errors.     

System identification from noisy measurements of inputs and outputs

In this paper a system identification method is described for the case of measurement errors on inputs and outputs. The method gives consistent estimates of the parameters and in case of normal measurement errors maximum likelihood estimates are obtained. More specific statistical properties of the estimators are also provided. Furthermore, the sensitivity of the results with respect to the assumptions is studied.     

Fast robust estimation of prediction error based on resampling

Robust estimators of the prediction error of a linear model are proposed. The estimators are based on the resampling techniques cross-validation and bootstrap. The robustness of the prediction error estimators is obtained by robustly estimating the regression parameters of the linear model and by trimming the largest prediction errors. To avoid the recalculation of time-consuming robust regression estimates, fast approximations for the robust estimates of the resampled data are used. This leads to time-efficient and robust estimators of prediction error.     

Optic flow field segmentation and motion estimation using a robustgenetic partitioning algorithm

Optic flow motion analysis represents an important family of visual information processing techniques in computer vision. Segmenting an optic flow field into coherent motion groups and estimating each underlying motion is a very challenging task when the optic flow field is projected from a scene of several independently moving objects. The problem is further complicated if the optic flow data are noisy and partially incorrect. In this paper, the authors present a novel framework for determining such optic flow fields by combining the conventional robust estimation with a modified genetic algorithm. The baseline model used in the development is a linear optic flow motion algorithm due to its computational simplicity. The statistical properties of the generalized linear regression (GLR) model are thoroughly explored and the sensitivity of the motion estimates toward data noise is quantitatively established. Conventional robust estimators are then incorporated into the linear regression model to suppress a small percentage of gross data errors or outliers. However, segmenting an optic flow field consisting of a large portion of incorrect data or multiple motion groups requires a very high robustness that is unattainable by the conventional robust estimators. To solve this problem, the authors propose a genetic partitioning algorithm that elegantly combines the robust estimation with the genetic algorithm by a bridging genetic operator called self-adaptation     

Estimation fusion for distributed multi-sensor systems with uncertain cross-correlations

This paper addresses the estimation fusion problem in distributed multi-sensor systems with uncertain cross-covariance among local estimation errors. A robust linear estimation fusion method is proposed in the sense of minimising the worst mean square error of the fused estimator over the uncertain normalised cross-covariances (NCC). The weighted coefficient matrices of the fused estimator can be obtained by solving a semi-definite programming problem. This estimation fusion method is suitable for the situations with completely unknown NCC or partly known NCC. Two fusion estimators for the uncertain NCC with partly known prior information are presented. Some numerical simulations are provided to show the good performance of the proposed estimators.     

Application of multiple regression analysis to identification of time-varying linear dynamic systems

A technique for determining the dynamic response characteristics of time-varying linear systems is presented. A model composed of a parallel connection of filters whose impulse responses are orthogonalized exponential functions is used. The input to the system is fed to each of the filters. The coefficients of linear regression of the system output on each of the filter outputs are determined. Time-varying characteristics are measured by determining successively the coefficients from short samples of the input and output signals of the system. Many of the statistical properties of regression coefficients are known and can be used to estimate the length of sample of input and output signals required to determine the coefficients with given variance. The measurement technique has been implemented on a high speed digital computer. Results obtained by applying the technique to measurement of a variety of digitally simulated filters and to measurement of human pilot control characteristics are presented.     

Finite population estimation under generalized linear model assistance

Finite population estimation is the overall goal of sample surveys. When information regarding auxiliary variables are available, one may take advantage of general regression estimators (GREG) to improve sample estimates precision. GREG estimators may be derived when the relationship between interest and auxiliary variables is represented by a normal linear model. However, in some cases, such as when estimating class frequencies or counting processes means, Bernoulli or Poisson models are more suitable than linear normal ones. This paper focuses on building regression type estimators under a model-assisted approach, for the general case in which the relationship between interest and auxiliary variables may be suitably described by a generalized linear model. The finite population distribution of the variable of interest is viewed as if generated by a member of the exponential family, which includes Bernoulli, Poisson, gamma and inverse Gaussian distributions, among others. The resulting estimator is a generalized linear model regression estimator (GEREG). Its general form and basic statistical properties are presented and studied analytically and empirically, using Monte Carlo simulation experiments. Three applications are presented in which the GEREG estimator shows better performance than the GREG one.     

Stochastic system identification with noisy input-outputmeasurements using polyspectra

Two new classes of parametric, frequency domain approaches are proposed for estimation of the parameters of scalar, linear “errors-in-variables” models, i.e., linear systems where measurements of both input and output of the system are noise contaminated. The first approach consists of linear estimators where using the bispectrum or the integrated polyspectrum of the input and the cross-bispectrum or the integrated cross-polyspectrum, the system transfer function is first estimated at a number of frequencies exceeding one-half the number of unknown parameters. The estimated transfer function is then used to estimate the unknown parameters using an overdetermined linear system of equations. In the second class of approaches, quadratic transfer function matching criteria are optimized by using the results of the linear estimators as initial guesses. Both classes of the parameter estimators are shown to be consistent in any measurement noise that has symmetric probability density function when the bispectral approaches are used. The proposed parameter estimators are shown to be consistent in Gaussian measurement noise when trispectral approaches are used     

On modified parameter estimators for identification and adaptive control. A unified framework and some new schemes

A key assumption in the development of system identification and adaptive control schemes is the availability of a regression model which is linear in the unknown parameters (of the plant and/or the controller). Applying standard— e.g., gradient descent-based—parameter estimators leads to a linear time-varying equation for the parameter errors, whose stability relies on the usually stringent persistency of excitation assumption. As suggested in Kreisselmeier (1977) and Lion (1967), with the inclusion of linear filters, it is possible to generate alternative regression models, whose parameter error equations have different stability properties. In Duarte and, Narendra (1989), Panteley, Ortega,and Moya, (2002) and Slotine and Li, (1989) estimators that combine tracking and identification errors, to generate new parameter error equations, were proposed. The main objectives of this paper are: first, based on the two key developments mentioned above, provide a unified framework for the analysis and design of parameter estimators and, in particular, show that they lie at the core of some modified schemes recently proposed in the literature. Second, extend the realm of application of these estimators to the class of nonlinear systems considered in Panteley et al. (2002). Third, use this framework to propose some new schemes with relaxed conditions for convergence and improved transient performance. Particular attention is given to the task of obviating the persistency of excitation assumption, which is rarely verified in applications and is, certainly not, the only way to ensure robustness of the schemes.     

滑油泵试验器流量修正参数确定方法研究

为提高低压大流量滑油泵试验器流量计量的准确性,针对计量过程中流量影响因素恒定控制难的问题,分析了流量计量过程的特性,确定了影响流量计量精度的主要因素:滑油泵出口压力、滑油温度和滑油泵转速~([1])。为消除这三种因素对计量的影响,提出了带回归系数的流量补偿方法。为确定回归参数,采用多元线性回归算法~([2]),建立多元线性回归模型,并结合大量的实验数据,利用Madab对回归模型求解,最终求得流量回归系数并应用到控制系统中。实践证明,该方法提高了试验器的计量精度,减少了实验次数,对两种标准泵计量20余次,计量误差均在±0.3 L之內,满足使用条件。     

A new MM algorithm for constrained estimation in the proportional hazards model

The constrained estimation in Cox’s model for the right-censored survival data is studied and the asymptotic properties of the constrained estimators are derived by using the Lagrangian method based on Karush–Kuhn–Tucker conditions. A novel minorization–maximization (MM) algorithm is developed for calculating the maximum likelihood estimates of the regression coefficients subject to box or linear inequality restrictions in the proportional hazards model. The first M-step of the proposed MM algorithm is to construct a surrogate function with a diagonal Hessian matrix, which can be reached by utilizing the convexity of the exponential function and the negative logarithm function. The second M-step is to maximize the surrogate function with a diagonal Hessian matrix subject to box constraints, which is equivalent to separately maximizing several one-dimensional concave functions with a lower bound and an upper bound constraint, resulting in an explicit solution via a median function. The ascent property of the proposed MM algorithm under constraints is theoretically justified. Standard error estimation is also presented via a non-parametric bootstrap approach. Simulation studies are performed to compare the estimations with and without constraints. Two real data sets are used to illustrate the proposed methods.     

Estimating the error variance after a pre-test for an interval restriction on the coefficients

This paper considers the estimation of the error variance after a pre-test of an interval restriction on the coefficients. We derive the exact finite sample risks of the interval restricted and pre-test estimators of the error variance, and examine the risk properties of the estimators to model misspecification through the omission of relevant regressors. It is found that the pre-test estimator performs better than the interval restricted estimator in terms of the risk properties in a large region of the parameter space; moreover, its risk performance is more robust with respect to the degrees of model misspecification. Furthermore, we propose a bootstrap procedure for estimating the risks of the estimators, to overcome the difficulty of computing the exact risks.     

Effects of measurement errors on psychometric measurements in ergonomics studies: Implications for correlations,ANOVA, linear regression,factor analysis,and linear discriminant analysis

This paper aims to demonstrate the effects of measurement errors on psychometric measurements in ergonomics studies. A variety of sources can cause random measurement errors in ergonomics studies and these errors can distort virtually every statistic computed and lead investigators to erroneous conclusions. The effects of measurement errors on five most widely used statistical analysis tools have been discussed and illustrated: correlation; ANOVA; linear regression; factor analysis; linear discriminant analysis. It has been shown that measurement errors can greatly attenuate correlations between variables, reduce statistical power of ANOVA, distort (overestimate, underestimate or even change the sign of) regression coefficients, underrate the explanation contributions of the most important factors in factor analysis and depreciate the significance of discriminant function and discrimination abilities of individual variables in discrimination analysis. The discussions will be restricted to subjective scales and survey methods and their reliability estimates. Other methods applied in ergonomics research, such as physical and electrophysiological measurements and chemical and biomedical analysis methods, also have issues of measurement errors, but they are beyond the scope of this paper. As there has been increasing interest in the development and testing of theories in ergonomics research, it has become very important for ergonomics researchers to understand the effects of measurement errors on their experiment results, which the authors believe is very critical to research progress in theory development and cumulative knowledge in the ergonomics field.     

Identification of multivariable stochastic linear systems viapolyspectral analysis given noisy input-output time-domain data

The paper considers the problem of identification of unknown parameters of multivariable, linear “errors-invariables” models. Attention is focused on frequency-domain approaches where the integrated polyspectrum (bispectrum or trispectrum) of the input and the integrated cross-polyspectrum, respectively, of the given time-domain input-output data are exploited. Two new classes of parametric frequency-domain approaches are proposed and analyzed. An integrated polyspectrum-based persistence of excitation condition on system input is defined. Both classes of the parameter estimators are shown to be strongly consistent in any measurement noise sequences with vanishing bispectra when integrated bispectrum-based approaches are used. The proposed parameter estimators are shown to be strongly consistent in Gaussian measurement noise when integrated trispectrum-based approaches are used. The input to the system need not be a linear process but must have nonvanishing bispectrum or trispectrum     

Parameter estimation for linear discrete-time models with randomcoefficients

The problem of parameter estimation in linear discrete-time systems with random coefficients is discussed. In particular, the maximum-likelihood estimators and their consistency for the defined structure of the model are derived. The estimators have a structure similar to that of the least square estimators for the linear discrete-time system with constant coefficients     

Differential games with imperfect state information

Nondeterministic differential games of imperfect information are considered, with particular emphasis on the case of a linear system, a quadratic cost functional, and independent white Gaussian noises additively corrupting the observable output measurements. Solutions are presented for a number of particular cases of this problem, including those in which one of the two controllers has either no information or, under certain additional restrictions, perfect measurements of the state vector. In each case the optimal control for each controller is shown to be closely related to that which would result by assuming a separation theorem to hold. Furthermore, the various terms in the resulting optimal cost are shown to be readily assignable to the appropriate contributing source, such as the optimal cost that would result if the problem were instead a deterministic one with perfect information, the effect of estimation errors, or the effect of measurement errors.     

On the performance of the flexible maximum entropy distributions within partially adaptive estimation

The partially adaptive estimation based on the assumed error distribution has emerged as a popular approach for estimating a regression model with non-normal errors. In this approach, if the assumed distribution is flexible enough to accommodate the shape of the true underlying error distribution, the efficiency of the partially adaptive estimator is expected to be close to the efficiency of the maximum likelihood estimator based on knowledge of the true error distribution. In this context, the maximum entropy distributions have attracted interest since such distributions have a very flexible functional form and nest most of the statistical distributions. Therefore, several flexible MaxEnt distributions under certain moment constraints are determined to use within the partially adaptive estimation procedure and their performances are evaluated relative to well-known estimators. The simulation results indicate that the determined partially adaptive estimators perform well for non-normal error distributions. In particular, some can be useful in dealing with small sample sizes. In addition, various linear regression applications with non-normal errors are provided.     

On the estimation of a large sparse Bayesian system: The Snaer program

The Snaer program calculates the posterior mean and variance of variables on some of which we have data (with precisions), on some we have prior information (with precisions), and on some prior indicator ratios (with precisions) are available. The variables must satisfy a number of exact restrictions. The system is both large and sparse. Two aspects of the statistical and computational development are a practical procedure for solving a linear integer system, and a stable linearization routine for ratios. The numerical method for solving large sparse linear least-squares estimation problems is tested and found to perform well, even when the n×k design matrix is large (nk=O(10⁸)).