Stochastic Regression Model with Dependent Disturbances

In this paper, we consider the estimation of the coefficient of a stochastic regression model whose explanatory variables and disturbances are permitted to exhibit short-memory or long-memory dependence. Three estimators of the coefficient are proposed. A variety of their asymptotics are illuminated under various assumptions on the explanatory variables and the disturbances. Numerical studies of the theoretical results are given. They show some unexpected aspects of the asymptotics of the three estimators.     

Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter estimation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust estimators are maximum likelihood estimators or M-estimators. These are commonly of two types, Huber estimators and Hampel estimators. The former significantly reduces the effect of large outliers whereas the latter nullifies their effect. In particular, these two estimators can be evaluated through the use of an influence function, which quantifies the effect of an observation on the estimated statistic. Here, the influence function must be bounded and finite for an estimator to be robust. For the Hampel estimators the influence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their influence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any exploratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estimators are also tuned to the particular observed system data through an iterative procedure based on the Akaike information criterion, (AIC). This approach is not easily afforded by the Huber estimators and this can have a significant impact on the estimation. The resulting approach is incorporated within an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples for data reconciliation.     

ROBUST FITTING OF INARCH MODELS

We discuss robust M‐estimation of INARCH models for count time series. These models assume the observation at each point in time to follow a Poisson distribution conditionally on the past, with the conditional mean being a linear function of previous observations. This simple linear structure allows us to transfer M‐estimators for autoregressive models to this situation, with some simplifications being possible because the conditional variance given the past equals the conditional mean. We investigate the performance of the resulting generalized M‐estimators using simulations. The usefulness of the proposed methods is illustrated by real data examples.     

基于岭估计的化工过程测量置信水平研究(英文)

Ordinary least squares （OLS） algorithm is widely applied in process measurement, because the sensor model used to estimate unknown parameters can be approximated through multivariate linear model. However, with few or noisy data or multi-collinearity, unbiased OLS leads to large variance. Biased estimators, especially ridge es-timator, have been introduced to improve OLS by trading bias for variance. Ridge estimator is feasible as an esti-mator with smaller variance. At the same confidence level, with additive noise as the normal random variable, the less variance one estimator has, the shorter the two-sided symmetric confidence interval is. However, this finding is limited to the unbiased estimator and few studies analyze and compare the confidence levels between ridge estima-tor and OLS. This paper derives the matrix of ridge parameters under necessary and sufficient conditions based on which ridge estimator is superior to OLS in terms of mean squares error matrix, rather than mean squares error. Then the confidence levels between ridge estimator and OLS are compared under the condition of OLS fixed sym-metric confidence interval, rather than the criteria for evaluating the validity of different unbiased estimators. We conclude that the confidence level of ridge estimator can not be directly compared with that of OLS based on the criteria available for unbiased estimators, which is verified by a simulation and a laboratory scale experiment on a single parameter measurement.     

YULE-WALKER ESTIMATES FOR CONTINUOUS-TIME AUTOREGRESSIVE MODELS

Abstract. I consider continuous-time autoregressive processes of order p and develop estimators of the model parameters based on Yule-Walker type equations. For continuously recorded data, it is shown that these estimators are least squares estimators and have the same asymptotic distribution as maximum likelihood estimators.
In practice, though, data can only be observed discretely. For discrete data, I consider approximations to the continuous-time estimators. It is shown that some of these discrete-time estimators are asymptotically biased. Alternative estimators based on the autocovariance function are suggested. These are asymptotically unbiased and are a fast alternative to the maximum likelihood estimators described by Jones. They may also be used as starting values for maximum likelihood estimation.     

ESTIMATION OF COEFFICIENTS OF TIME SERIES REGRESSION WITH A NONSTATIONARY ERROR PROCESS

Abstract. We treat a problem of estimating unknown coefficients of a time series regression when the variance of the error changes with time, i.e. when a process which the error term obeys is nonstationary. First, we show the weak consistency of the ordinary least squares estimator for the coefficients of a polynomial regression under some assumptions on the covariance structure of the error process. Next, we propose a nonparametric method for estimating the variance of the error process and a weighted least squares estimator of the regression coefficients, which is constructed by using the estimator of the variance. We investigate statistical properties of our proposed estimator in the following way. We consider the prediction of a future value of a linear trend by using our proposed estimator and evaluate its prediction error. By simulation studies, we compare the prediction error of the predictor constructed by using our proposed estimator with the prediction errors obtained for other estimators including the ordinary least squares estimator when the variance of the error process increases with time and the sample sizes are small. As a result, our proposed estimator seems to be reasonable.     

BIAS-CORRECTED NONPARAMETRIC SPECTRAL ESTIMATION

Abstract. The theory of nonparametric spectral density estimation based on an observed stretch X₁,…, X_N from a stationary time series has been studied extensively in recent years. However, the most popular spectral estimators, such as the ones proposed by Bartlett, Daniell, Parzen, Priestley and Tukey, are plagued by the problem of bias, which effectively prohibits ?N-convergence of the estimator. This is true even in the case where the data are known to be m-dependent, in which case ?N-consistent estimation is possible by a simple plug-in method. In this report, an intuitive method for the reduction in the bias of a nonparametric spectral estimator is presented. In fact, applying the proposed methodology to Bartlett's estimator results in bias-corrected estimators that are related to kernel estimators with lag-windows of trapezoidal shape. The asymptotic performance (bias, variance, rate of convergence) of the proposed estimators is investigated; in particular, it is found that the trapezoidal lag-window spectral estimator is ?N-consistent in the case of moving-average processes, and ?(N/log/N)-consistent in the case of autoregressive moving-average processes. The finite-sample performance of the trapezoidal lag-window estimator is also assessed by means of a numerical simulation.     

REDUCTION OF THE ASYMPTOTIC BIAS OF AUTOREGRESSIVE AND SPECTRAL ESTIMATORS BY TAPERING

Abstract. The asymptotic bias to terms of order T ^-1, where T is the observed series length, is studied for estimators of the coefficients and disturbance variance in an AR( p ) model. Reduction of the asymptotic bias by tapering is established and, if the tapering function is defined appropriately to depend on T , not only is the asymptotic bias reduced, but the asymptotic distribution of the estimators is not altered. In addition, the asymptotic biases of other time series parameter estimators constructed from the sample covariance function, such as several types of spectral estimators, can also be reduced by tapering.     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.     

On Optimal Adaptive Prediction of Multivariate Autoregression

Abstract

The problem of asymptotic efficiency of adaptive one-step predictors for a stable multivariate first-order autoregressive process (AR(1)) with unknown parameters is considered. The predictors are based on the truncated estimators of the dynamic matrix parameter. The truncated estimation method is a modification of the truncated sequential estimation method that makes it possible to obtain estimators of ratio-type functionals with a given accuracy by samples of fixed size. The criterion of optimality is based on the loss function, defined as a sum of sample size and squared prediction error's sample mean. The cases of known and unknown variance of the noise model are studied. In the latter case the optimal sample size is a special stopping time. The simulation results are given.     

Statistical tests for a single change in mean against long‐range dependence

Statistical tests are introduced for distinguishing between short‐range dependent time series with a single change in mean, and long‐range dependent time series, with the former making the null hypothesis. The tests are based on estimation of the self‐similarity parameter after removing the change in mean from the series. The focus is on the GPH (Geweke and Porter‐Hudak, 1983) and local Whittle estimation methods in the spectral domain. Theoretical properties of the resulting estimators are established when testing for a single change in mean, and small sample properties of the tests are examined in simulations. The introduced tests improve on the BHKS ( Berkes et al., 2006 ) test which is the only other available test for the considered problem. It is argued that the BHKS test has a low power against long‐range dependence alternatives and that this happens because the BHKS test statistic involves estimation of the long‐run variance. The BHKS test could be improved readily by considering its R/S‐like regression version which estimates the self‐similarity parameter and which does not involve the long‐run variance. Yet better alternatives are to use more powerful estimation methods (such as GPH or local Whittle) and lead to the tests introduced here.     

CONSISTENT ESTIMATION OF THE FOURTH-ORDER CUMULANT SPECTRAL DENSITY

Abstract. In this paper we consider the estimation of the fourth-order cumulant spectral density. Indeed this is the first case where the cumulant depends on lower-order product moments for a mean-zero stationary process. The proposed estimator of the fourth-order cumulant spectral density is constructed by replacing product moments with appropriately weighted estimates of product moments according to the definition of the fourth-order cumulant spectral density. Asymptotic unbiasedness and consistency are shown to hold for these estimators under stationarity and absolute summability of cumulants up to various orders with no restrictions on the frequencies. An expression for the asymptotic variance is also obtained.     

DESIGN OF A COMPOSITION ESTIMATOR FOR INFERENTIAL CONTROL OF DISTILLATION COLUMNS

In distillation column control, secondary measurements such as temperatures and flows are widely used in order to infer product composition. This paper addresses the design of the linear static estimators using the secondary measurements for estimating product compositions of distillation columns. Based on the unified framework for the estimator design, the relationships among various static estimators are discussed in terms of the estimator structure. Il is shown that the projection estimator is equivalent to the regression estimators in the special cases. Since the projection estimator heavily depends on the measured inputs such as reflux flow and heat input to the reboiler due to its structural characteristic, the estimation performance is far more sensitive to measurement noise and nonlinearity of them, compared wiih the regression estimators based on the PCR or PLS method. It is also found that the use of the measured inputs leads to performance deterioration of both the projection and regression estimators because of their nonlinear effects on the product compositions especially in high-purity columns. Design guidelines for the PCR and PLS estimators are presented by analyzing the results of the simulation studies on a high-purity column example. The estimator based on the guidelines is robust to sensor noise and has a good predictive power     

Postmodel selection estimators of variance function for nonlinear autoregression

We consider a problem of estimating a conditional variance function of an autoregressive process. A finite collection of parametric models for conditional density is studied when both regression and variance are modelled by parametric functions. The proposed estimators are defined as the maximum likelihood estimators in the models chosen by penalized selection criteria. Consistency properties of the resulting estimator of the variance when the conditional density belongs to one of the parametric models are studied as well as its behaviour under mis‐specification. The autoregressive process does not need to be stationary but only existence of a stationary distribution and ergodicity is required. Analogous results for the pseudolikelihood method are also discussed. A simulation study shows promising behaviour of the proposed estimator in the case of heavy‐tailed errors in comparison with local linear smoothers.     

Statistical Inference for Unified Garch–Itô Models with High‐Frequency Financial Data

The existing estimation methods for the model parameters of the unified GARCH–Itô model (Kim and Wang, 2014 ) require long period observations to obtain the consistency. However, in practice, it is hard to believe that the structure of a stock price is stable during such a long period. In this article, we introduce an estimation method for the model parameters based on the high‐frequency financial data with a finite observation period. In particular, we establish a quasi‐likelihood function for daily integrated volatilities, and realized volatility estimators are adopted to estimate the integrated volatilities. The model parameters are estimated by maximizing the quasi‐likelihood function. We establish asymptotic theories for the proposed estimator. A simulation study is conducted to check the finite sample performance of the proposed estimator. We apply the proposed estimation approach to the Bank of America stock price data.     

MAXIMUM LIKELIHOOD ESTIMATION FOR AUTOREGRESSIVE PROCESSES DISTURBED BY A MOVING AVERAGE

Abstract. Maximum likelihood estimation for stationary autoregressive processes when the signal is subject to a moving-average sampling error is discussed. A modified maximum likelihood estimator is proposed. An algorithm for computing derivatives of the modified likelihood is suggested. Maximum likelihood estimators of the parameter vector are shown to be strongly consistent and to have a multivariate normal limiting distribution. A Monte Carlo simulation shows that the modified maximum likelihood estimator performs better than other available estimators. US current labour force data are analysed as an example.     

Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

BIAS CALCULATIONS FOR ADAPTIVE URN DESIGNS

A clinical trial model is considered in which two treatments with immediate binary responses are to be compared. An adaptive urn design is used to assign patients to the treatments. The bias and variance of the maximum likelihood estimators of the probabilities of success are derived by differentiating the fundamental identity of sequential analysis. By embedding the design in a continuous-time process, probability generating functions are then calculated to obtain approximations for the bias and variance. Simulation is used to assess the accuracy of the approximations. It is shown that the bias cannot be ignored, and that the adaptive rules which are subcritical in nature have the most mathematically tractable bias and are the least variable. Methods for correcting for the bias are also addressed.     

ON M‐Estimation Under Long‐Range Dependence in Volatility

Abstract. We consider M‐estimation of a location parameter for processes with zero autocorrelations but long‐range dependence in volatility. The observed process is the product of i.i.d. Gaussian observations and a long‐memory Gaussian process. For nonlinear estimators, the rate of convergence depends on the type of the ψ‐function. For skew‐symmetric ψ‐functions, a central limit theorem with ‐rate of convergence holds, under suitable regularity assumptions. This is not true in general for M‐estimators where the ψ‐function is not skewsymmetric.     

ESTIMATION OF THE NON-STATIONARY FACTOR IN ARUMA MODELS

Abstract. Two methods for the estimation of the non-stationary factor in ARUMA models are given. Both methods yield strongly consistent estimators and the roots of the corresponding filters lie on the unit circle.