Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

Practical improvements to autocovariance least‐squares

Identifying disturbance covariances from data is a critical step in estimator design and controller performance monitoring. Here, the autocovariance least‐squares (ALS) method for this identification is examined. For large industrial models with poorly observable states, the process noise covariance is high dimensional and the optimization problem is poorly conditioned. Also, weighting the least‐squares problem with the identity matrix does not provide minimum variance estimates. Here, ALS method to resolve these two challenges is modified. Poorly observable states using the singular value decomposition (SVD) of the observability matrix is identified and removed, thus decreasing the computational time. Using a new feasible‐generalized least‐squares estimator that approximates the optimal weighting from data, the variance of the estimates is significantly reduced. The new approach on industrial data sets provided by Praxair is successfully demonstrated. The disturbance model identified by the ALS method produces an estimator that performs optimally over a year‐long period. © 2015 American Institute of Chemical Engineers AIChE J, 61: 1840–1855, 2015     

A Note on Mean-squared Prediction Errors of the Least Squares Predictors in Random Walk Models

An asymptotic expression for the mean-squared prediction error (MSPE) of the least squares predictor is obtained in the random walk model. It is shown that the term of order 1/ n in this error, where n is the sample size, is twice as large as the one obtained from the first-order autoregressive (AR(1)) model satisfying the stationary assumption. Moreover, while the correlation between the squares of the (normalized) regressor variable and normalized least squares estimator is asymptotically negligible in the stationary AR(1) model, we have found that the correlation has significantly negative value in the random walk model. To obtain these results, a new methodology, which is found to be useful in dealing with the moment properties of a strongly dependent process, is introduced.     

NON‐STATIONARITY AND QUASI‐MAXIMUM LIKELIHOOD ESTIMATION ON A DOUBLE AUTOREGRESSIVE MODEL

This article first studies the non‐stationarity of the first‐order double AR model, which is defined by the random recurrence equation , where γ₀ > 0, α₀ ≥ 0, and {η_t}is a sequence of i.i.d. symmetric random variables. It is shown that the double AR(1) model is explosive under the condition . Based on this, it is shown that the quasi‐maximum likelihood estimator of (φ₀,α₀) is consistent and asymptotically normal so that the unit root problem does not exist in the double AR(1) model. Simulation studies are carried out to assess the performance of the quasi‐maximum likelihood estimator in finite samples.     

Non‐stationary autoregressive processes with infinite variance

Consider an AR(p) process , where {?_t} is a sequence of i.i.d. random variables lying in the domain of attraction of a stable law with index 0<α<2. This time series {Y_t} is said to be a non‐stationary AR(p) process if at least one of its characteristic roots lies on the unit circle. The limit distribution of the least squares estimator (LSE) of for {Y_t} with infinite variance innovation {?_t} is established in this paper. In particular, by virtue of the result of Kurtz and Protter (1991) of stochastic integrals, it is shown that the limit distribution of the LSE is a functional of integrated stable process. Simulations for the estimator of β and its limit distribution are also given.     

Polynomial Trend Regression With Long‐memory Errors

Abstract. For a time series generated by polynomial trend with stationary long‐memory errors, the ordinary least squares estimator (OLSE) of the trend coefficients is asymptotically normal, provided the error process is linear. The asymptotic distribution may no longer be normal, if the error is in the form of a long‐memory linear process passing through certain nonlinear transformations. However, one hardly has sufficient information about the transformation to determine which type of limiting distribution the OLSE converges to and to apply the correct distribution so as to construct valid confidence intervals for the coefficients based on the OLSE. The present paper proposes a modified least squares estimator to bypass this drawback. It is shown that the asymptotic normality can be assured for the modified estimator with mild trade‐off of efficiency even when the error is nonlinear and the original limit for the OLSE is non‐normal. The estimator performs fairly well when applied to various simulated series and two temperature data sets concerning global warming.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

Computer Algebra Derivation of the Bias of Linear Estimators of Autoregressive Models

Abstract. A symbolic method which can be used to obtain the asymptotic bias and variance coefficients to order O(1/n) for estimators in stationary time series is discussed. Using this method, the large‐sample bias of the Burg estimator in the AR(p) for p = 1, 2, 3 is shown to be equal to that of the least squares estimators in both the known and unknown mean cases. Previous researchers have only been able to obtain simulation results for the Burg estimator's bias because this problem is too intractable without using computer algebra. The asymptotic bias coefficient to O(1/n) of Yule–Walker as well as least squares estimates is also derived in AR(3) models. Our asymptotic results show that for the AR(3), just as in the AR(2), the Yule–Walker estimates have a large bias when the parameters are near the nonstationary boundary. The least squares and Burg estimates are much better in this situation. Simulation results confirm our findings.     

Semiparametric Estimation in Time‐Series Regression with Long‐Range Dependence

Abstract. We consider semiparametric estimation in time‐series regression in the presence of long‐range dependence in both the errors and the stochastic regressors. A central limit theorem is established for a class of semiparametric frequency domain‐weighted least squares estimates, which includes both narrow‐band ordinary least squares and narrow‐band generalized least squares as special cases. The estimates are semiparametric in the sense that focus is on the neighbourhood of the origin, and only periodogram ordinates in a degenerating band around the origin are used. This setting differs from earlier studies on time‐series regression with long‐range dependence, where a fully parametric approach has been employed. The generalized least squares estimate is infeasible when the degree of long‐range dependence is unknown and must be estimated in an initial step. In that case, we show that a feasible estimate which has the same asymptotic properties as the infeasible estimate, exists. By Monte Carlo simulation, we evaluate the finite‐sample performance of the generalized least squares estimate and the feasible estimate.     

Asymptotics for the Conditional‐Sum‐of‐Squares Estimator in Multivariate Fractional Time‐Series Models

This article proves consistency and asymptotic normality for the conditional‐sum‐of‐squares estimator, which is equivalent to the conditional maximum likelihood estimator, in multivariate fractional time‐series models. The model is parametric and quite general and, in particular, encompasses the multivariate non‐cointegrated fractional autoregressive integrated moving average (ARIMA) model. The novelty of the consistency result, in particular, is that it applies to a multivariate model and to an arbitrarily large set of admissible parameter values, for which the objective function does not converge uniformly in probability, thus making the proof much more challenging than usual. The neighbourhood around the critical point where uniform convergence fails is handled using a truncation argument.     

Identification of Persistent Cycles in Non‐Gaussian Long‐Memory Time Series

Abstract. Asymptotic distribution is derived for the least squares estimates (LSE) in the unstable AR(p) process driven by a non‐Gaussian long‐memory disturbance. The characteristic polynomial of the autoregressive process is assumed to have pairs of complex roots on the unit circle. In order to describe the limiting distribution of the LSE, two limit theorems involving long‐memory processes are established in this article. The first theorem gives the limiting distribution of the weighted sum, is a non‐Gaussian long‐memory moving‐average process and (c_n,k,1 ≤ k ≤ n) is a given sequence of weights; the second theorem is a functional central limit theorem for the sine and cosine Fourier transforms      

Multivariate Portmanteau Test For Autoregressive Models with Uncorrelated but Nonindependent Errors

Abstract. We study the asymptotic behaviour of the least squares estimator, of the residual autocorrelations and of the Ljung–Box (or Box–Pierce) portmanteau test statistic for multiple autoregressive time series models with nonindependent innovations. Under mild assumptions, it is shown that the asymptotic distribution of the portmanteau tests is that of a weighted sum of independent chi‐squared random variables. When the innovations exhibit conditional heteroscedasticity or other forms of dependence, this asymptotic distribution can be quite different from that of models with independent and identically distributed innovations. Consequently, the usual chi‐squared distribution does not provide an adequate approximation to the distribution of the Box–Pierce goodness‐of‐fit portmanteau test in the presence of nonindependent innovations. Hence we propose a method to adjust the critical values of the portmanteau tests. Monte carlo experiments illustrate the finite sample performance of the modified portmanteau test.     

A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.     

A superharmonic prior for the autoregressive process of the second‐order

Abstract. The Bayesian estimation of the spectral density of the AR(2) process is considered. We propose a superharmonic prior on the model as a non‐informative prior rather than the Jeffreys prior. Theoretically, the Bayesian spectral density estimator based on it dominates asymptotically the one based on the Jeffreys prior under the Kullback–Leibler divergence. In the present article, an explicit form of a superharmonic prior for the AR(2) process is presented and compared with the Jeffreys prior in computer simulation.     

Minimum α‐divergence estimation for arch models

Abstract. This paper considers a minimum α‐divergence estimation for a class of ARCH(p) models. For these models with unknown volatility parameters, the exact form of the innovation density is supposed to be unknown in detail but is thought to be close to members of some parametric family. To approximate such a density, we first construct an estimator for the unknown volatility parameters using the conditional least squares estimator given by Tjøstheim [Stochastic processes and their applications (1986) Vol. 21, pp. 251–273]. Then, a nonparametric kernel density estimator is constructed for the innovation density based on the estimated residuals. Using techniques of the minimum Hellinger distance estimation for stochastic models and residual empirical process from an ARCH(p) model given by Beran [Annals of Statistics (1977) Vol. 5, pp. 445–463] and Lee and Taniguchi [Statistica Sinica (2005) Vol. 15, pp. 215–234] respectively, it is shown that the proposed estimator is consistent and asymptotically normal. Moreover, a robustness measure for the score of the estimator is introduced. The asymptotic efficiency and robustness of the estimator are illustrated by simulations. The proposed estimator is also applied to daily stock returns of Dell Corporation.     

Ensemble locally weighted partial least squares as a just‐in‐time modeling method

The predictive ability of soft sensors, which estimate values of an objective variable y online, decreases due to process changes in chemical plants. To reduce the decrease of predictive ability, adaptive soft sensors have been developed. We focused on just‐in‐time soft sensors, especially locally weighted partial least squares (LWPLS) regression. Since a set of hyperparameters in an LWPLS model has to be set beforehand and there is only onedataset, a traditional LWPLS model is difficult to accurately predict y‐values in multiple process states. In this study, we propose to combine LWPLS and ensemble learning, and predict y‐values with multiple LWPLS models, whose datasets and sets of hyperparameters are different. The weights of LWPLS models are determined based on Bayes’ theorem, considering their predictive ability. We confirmed that the proposed model has higher predictive accuracy than traditional models through numerical simulation data and two industrial data analyses. © 2015 American Institute of Chemical Engineers AIChE J, 62: 717–725, 2016     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.     

Small‐b and Fixed‐b Asymptotics for Weighted Covariance Estimation in Fractional Cointegration

In a standard cointegrating framework, Phillips (1991) introduced the weighted covariance (WC) estimator of cointegrating parameters. Later, Marinucci (2000) applied this estimator to fractional circumstances and, like Phillips (1991), analysed the so‐called small‐b asymptotic approximation to its sampling distribution. Recently, an alternative limiting theory (fixed‐b asymptotics) has been successfully employed to approximate sampling distributions. With the purpose of comparing both approaches, we derive here the fixed‐b limit of WC estimators in a fractional setting, filling also some gaps in the traditional (small‐b) theory. We also provide some Monte Carlo evidence that suggests that the fixed‐b limit is more accurate.     

Guaranteed parameter estimation in a first order autoregressive progress with infinite variance

Let be a first order autoregressive process where the ε_t's are independent identically distributed random variables. The distribution of ε₁ is assumed to be symmetric and asymptotically Pareto of index p for some p ε (1,2). The variance in this case is infinite but for any r < p the absolute moment of order r is finite. At first, for r ε (1, p], we introduce an appropriate weighted least squares estimator of the autoregression parameter λ. It is shown that it is strongly consistent whatever is the true value of λ. Then a sequential procedure based on a modification of this estimator is proposed. It enables to achieve a prescribed precision measured by the r th order absolute moment of the estimation error     

NORMING RATES AND LIMIT THEORY FOR SOME TIME‐VARYING COEFFICIENT AUTOREGRESSIONS

A time‐varying autoregression is considered with a similarity‐based coefficient and possible drift. It is shown that the random‐walk model has a natural interpretation as the leading term in a small‐sigma expansion of a similarity model with an exponential similarity function as its AR coefficient. Consistency of the quasi‐maximum likelihood estimator of the parameters in this model is established, the behaviours of the score and Hessian functions are analysed and test statistics are suggested. A complete list is provided of the normalization rates required for the consistency proof and for the score and Hessian function standardization. A large family of unit root models with stationary and explosive alternatives is characterized within the similarity class through the asymptotic negligibility of a certain quadratic form that appears in the score function. A variant of the stochastic unit root model within the class is studied, and a large‐sample limit theory provided, which leads to a new nonlinear diffusion process limit showing the form of the drift and conditional volatility induced by sustained stochastic departures from unity. The findings provide a composite case for time‐varying coefficient dynamic modelling. Some simulations and a brief empirical application to data on international Exchange Traded Funds are included. Copyright © 2014 Wiley Publishing Ltd