A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.     

Bootstrapping the Local Periodogram of Locally Stationary Processes

Abstract. Locally stationary processes are non‐stationary stochastic processes the second‐order structure of which varies smoothly over time. In this paper, we develop a method to bootstrap the local periodogram of a locally stationary process. Our method generates pseudo local periodogram ordinates by combining a parametric time and non‐parametric frequency domain bootstrap approach. We first fit locally a time varying autoregressive model so as to capture the essential characteristics of the underlying process. A locally calculated non‐parametric correction in the frequency domain is then used so as to improve upon the locally parametric autoregressive fit. As an application, we investigate theoretically the asymptotic properties of the bootstrap method proposed applied to the class of local spectral means, local ratio statistics and local spectral density estimators. Some simulations demonstrate the ability of our method to give accurate estimates of the quantities of interest in finite sample situations and an application to a real‐life data‐set is presented.     

Local Likelihood for non‐parametric ARCH(1) models

Abstract. We propose a non‐parametric local likelihood estimator for the log‐transformed autoregressive conditional heteroscedastic (ARCH) (1) model. Our non‐parametric estimator is constructed within the likelihood framework for non‐Gaussian observations: it is different from standard kernel regression smoothing, where the innovations are assumed to be normally distributed. We derive consistency and asymptotic normality for our estimators and show, by a simulation experiment and some real‐data examples, that the local likelihood estimator has better predictive potential than classical local regression. A possible extension of the estimation procedure to more general multiplicative ARCH(p) models with p > 1 predictor variables is also described.     

Consistent estimation of the memory parameter for nonlinear time series

Abstract. For linear processes, semiparametric estimation of the memory parameter, based on the log‐periodogram and local Whittle estimators, has been exhaustively examined and their properties well established. However, except for some specific cases, little is known about the estimation of the memory parameter for nonlinear processes. The purpose of this paper is to provide the general conditions under which the local Whittle estimator of the memory parameter of a stationary process is consistent and to examine its rate of convergence. We show that these conditions are satisfied for linear processes and a wide class of nonlinear models, among others, signal plus noise processes, nonlinear transforms of a Gaussian process ξ_t and exponential generalized autoregressive, conditionally heteroscedastic (EGARCH) models. Special cases where the estimator satisfies the central limit theorem are discussed. The finite‐sample performance of the estimator is investigated in a small Monte Carlo study.     

SIGNIFICANT VARIABLE SELECTION AND AUTOREGRESSIVE ORDER DETERMINATION FOR TIME‐SERIES PARTIALLY LINEAR MODELS

This paper is concerned with the regression coefficient and autoregressive order shrinkage and selection via the smoothly clipped absolute deviation (SCAD) penalty for a partially linear model with time‐series errors. By combining the profile semi‐parametric least squares method and SCAD penalty technique, a new penalized estimation for the regression and autoregressive parameters in the model is proposed. We show that the asymptotic property of the resultant estimator is the same as if the order of autoregressive error structure and non‐zero regression coefficients are known in advance, thus achieving the oracle property in the sense of Fan and Li (2001). In addition, based on a prewhitening technique, we construct a two‐stage local linear estimator (TSLLE) for the non‐parametric component. It is shown that the TSLLE is more asymtotically efficient than the one that ignores the autoregressive time‐series error structure. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on electricity usage data is also illustrated. Copyright © 2014 Wiley Publishing Ltd     

Akaike’s information criterion correction for the least‐squares autoregressive spectral estimator

In this article we propose a new correction for the penalty term of the Akaike’s information criterion (AIC), when it is used in the context of order selection for an autoregressive fit of the spectral density of a stationary time series. The classical AIC penalty term may be viewed as an approximation of an appropriate target quantity. Simulations show that the quality of this approximation strongly depends on the type of autoregressive estimator used, as well as on the discrepancy used. Therefore here we consider the least squares autoregressive estimator and the Whittle discrepancy only. In this context we propose a closed formula correction of the AIC penalty term. We also develop asymptotic theory which justifies this proposal: an asymptotically valid second‐order expansion of a stochastic approximation of the target quantity. This expansion assumes a non‐parametric framework: it does not assume gaussianity of the process and only requires its spectral density to be smooth enough. Simulations show that, as compared to previously introduced corrections, this new correction performs similarly to finite sample information criterion, while they both outperform AIC corrected and AIC.     

Properties of the nonparametric autoregressive bootstrap

For nonparametric autoregression, we investigate a model based bootstrap procedure (`autoregressive bootstrap') that mimics the complete dependence structure of the original time series. We give consistency results for uniform bootstrap confidence bands of the autoregression function based on kernel estimates of the autoregression function. This result is achieved by global strong approximations of the kernel estimates for the resample and for the original sample. Furthermore, it is obtained that the autoregressive bootstrap also yields asymptotically correct approximations for distributions of parametric statistics, for which regression-type bootstrap-techniques like the wild bootstrap do not work. For this purpose, we prove geometric ergodicity and absolute regularity of the nonparametric autoregressive bootstrap process. We propose some particular estimators of the autoregression function and of the density of the innovations such that the mixing coefficients of the autoregressive bootstrap process can be bounded uniformly by some exponentially decaying sequence. This is achieved by using well-established coupling techniques. Moreover, by using some `decoupling' argument, we show that the stationary density of the bootstrap process converges to that of the original process. The paper may serve as a template for proving similar consistency results for other bootstrap techniques such as the Markov bootstrap.     

Density estimation with locally identically distributed data and with locally stationary data

Abstract. We consider multivariate density estimation when the assumptions of identically distributed data or stationary data are relaxed to the assumptions of locally identically distributed data or locally stationary data. We assume that the distribution of the data is changing continuously as function of time. To estimate densities non‐parametrically with these local regularity conditions, we need time localization in addition to the usual space localization. We define a time‐localized kernel estimator that estimates the density non‐parametrically at any given point of time. The consistency of the time‐localized kernel estimator is proved and the rates of convergence of the estimator are derived under conditions on the β‐and α‐mixing coefficients. Both the time‐series setting and spatial setting are covered.     

On the Spectral Density of the Wavelet Coefficients of Long‐Memory Time Series with Application to the Log‐Regression Estimation of the Memory Parameter

Abstract. In recent years, methods to estimate the memory parameter using wavelet analysis have gained popularity in many areas of science. Despite its widespread use, a rigorous semi‐parametric asymptotic theory, comparable with the one developed for Fourier methods, is still lacking. In this article, we adapt to the wavelet setting, the classical semi‐parametric framework introduced by Robinson and his co‐authors for estimating the memory parameter of a (possibly) non‐stationary process. Our results apply to a class of wavelets with bounded supports, which include but are not limited to Daubechies wavelets. We derive an explicit expression of the spectral density of the wavelet coefficients and show that it can be approximated, at large scales, by the spectral density of the continuous‐time wavelet coefficients of fractional Brownian motion. We derive an explicit bound for the difference between the spectral densities. As an application, we obtain minimax upper bounds for the log‐scale regression estimator of the memory parameter for a Gaussian process and we derive an explicit expression of its asymptotic variance.     

A Goodness‐of‐Fit Test for Integer‐Valued Autoregressive Processes

For autoregressive count data time series, a goodness‐of‐fit test based on the empirical joint probability generating function is considered. The underlying process is contained in a general class of Markovian models satisfying a drift condition. Asymptotic theory for the test statistic is provided, including a functional central limit theorem for the non‐parametric estimation of the stationary distribution and a parametric bootstrap method. Connections between the new approach and existing tests for count data time series based on moment estimators appear in limiting scenarios. Finally, the test is applied to a real data set.     

Postmodel selection estimators of variance function for nonlinear autoregression

We consider a problem of estimating a conditional variance function of an autoregressive process. A finite collection of parametric models for conditional density is studied when both regression and variance are modelled by parametric functions. The proposed estimators are defined as the maximum likelihood estimators in the models chosen by penalized selection criteria. Consistency properties of the resulting estimator of the variance when the conditional density belongs to one of the parametric models are studied as well as its behaviour under mis‐specification. The autoregressive process does not need to be stationary but only existence of a stationary distribution and ergodicity is required. Analogous results for the pseudolikelihood method are also discussed. A simulation study shows promising behaviour of the proposed estimator in the case of heavy‐tailed errors in comparison with local linear smoothers.     

Local Whittle estimation of multi‐variate fractionally integrated processes

This article derives a semi‐parametric estimator of multi‐variate fractionally integrated processes covering both stationary and non‐stationary values of d. We utilize the notion of the extended discrete Fourier transform and periodogram to extend the multi‐variate local Whittle estimator of Shimotsu (2007) to cover non‐stationary values of d. Consistency and asymptotic normality is shown for d ∈ (?1/2,∞). A simulation study illustrates the performance of the proposed estimator for relevant sample sizes. Empirical justification of the proposed estimator is shown through an empirical analysis of log spot exchange rates. We find that the log spot exchange rates of Germany, United Kingdom, Japan, Canada, France, Italy and Switzerland against the US Dollar for the period January 1974 until December 2001 are well decribed as I(1) processes.     

Oracle M‐Estimation for Time Series Models

We propose a thresholding M‐estimator for multivariate time series. Our proposed estimator has the oracle property that its large‐sample properties are the same as of the classical M‐estimator obtained under the a priori information that the zero parameters were known. We study the consistency of the standard block bootstrap, the centred block bootstrap and the empirical likelihood block bootstrap distributions of the proposed M‐estimator. We develop automatic selection procedures for the thresholding parameter and for the block length of the bootstrap methods. We present the results of a simulation study of the proposed methods for a sparse vector autoregressive VAR(2) time series model. The analysis of two real‐world data sets illustrate applications of the methods in practice.     

NON‐PARAMETRIC ESTIMATION OF HIGH‐FREQUENCY SPOT VOLATILITY FOR BROWNIAN SEMIMARTINGALE WITH JUMPS

The availability of high‐frequency financial data has led to substantial improvements in our understanding of financial volatility. Most existing literature focuses on estimating the integrated volatility over a fixed period. This article proposes a non‐parametric threshold kernel method to estimate the time‐dependent spot volatility and jumps when the underlying price process is governed by Brownian semimartingale with finite activity jumps. The threshold kernel estimator combines the threshold estimation for integrated volatility and the kernel filtering approach for spot volatility when the price process is driven only by diffusions without jumps. The estimator proposed is consistent and asymptotically normal and has the same rate of convergence as the estimator studied by Kristensen (2010) in a setting without jumps. The Monte Carlo simulation study shows that the proposed estimator exhibits excellent performance over a wide range of jump sizes and for different sampling frequencies. An empirical example is given to illustrate the potential applications of the proposed method.     

A Gaussian Mixture Autoregressive Model for Univariate Time Series

The Gaussian mixture autoregressive model studied in this article belongs to the family of mixture autoregressive models, but it differs from its previous alternatives in several advantageous ways. A major theoretical advantage is that, by the definition of the model, conditions for stationarity and ergodicity are always met and these properties are much more straightforward to establish than is common in nonlinear autoregressive models. Another major advantage is that, for a pth‐order model, explicit expressions of the stationary distributions of dimension p + 1 or smaller are known and given by mixtures of Gaussian distributions with constant mixing weights. In contrast, the conditional distribution given the past observations is a Gaussian mixture with time‐varying mixing weights that depend on p lagged values of the series in a natural and parsimonious way. Because of the known stationary distribution, exact maximum likelihood estimation is feasible and one can assess the applicability of the model in advance by using a non‐parametric estimate of the stationary density. An empirical example with interest rate series illustrates the practical usefulness and flexibility of the model, particularly in allowing for level shifts and temporary changes in variance. Copyright © 2014 Wiley Publishing Ltd     

UNIFIED INTERVAL ESTIMATION FOR RANDOM COEFFICIENT AUTOREGRESSIVE MODELS

The consistency of the quasi‐maximum likelihood estimator for random coefficient autoregressive models requires that the coefficient be a non‐degenerate random variable. In this article, we propose empirical likelihood methods based on weighted‐score equations to construct a confidence interval for the coefficient. We do not need to distinguish whether the coefficient is random or deterministic and whether the process is stationary or non‐stationary, and we present two classes of equations depending on whether a constant trend is included in the model. A simulation study confirms the good finite‐sample behaviour of our resulting empirical likelihood‐based confidence intervals. We also apply our methods to study US macroeconomic data.     

Nonlinear ARMA models with functional MA coefficients

Abstract. In the present article, we propose and study a new class of nonlinear autoregressive moving‐average (ARMA) models, in which each moving‐average (MA) coefficient is enlarged to an arbitrary univariate function. We first provide a sufficient condition for the existence of the stationary solution and further discuss the moment structure. We investigate the estimation method to the proposed models. The global estimates of parameters and local linear estimates of functional coefficients are obtained by using a back‐fitting algorithm. For testing whether the functional coefficients are some specified parametric forms, a bootstrap test approach is provided. The proposed models are illustrated by both simulated and real data examples.     

Random Walks with Drift – A Sequential Approach

Abstract. In this paper, sequential monitoring schemes to detect nonparametric drifts are studied for the random walk case. The procedure is based on a kernel smoother. As a by‐product we obtain the asymptotics of the Nadaraya–Watson estimator and its associated sequential partial sum process under non‐standard sampling. The asymptotic behaviour differs substantially from the stationary situation, if there is a unit root (random walk component). To obtain meaningful asymptotic results, we consider local nonparametric alternatives for the drift component. It turns out that the rate of convergence at which the drift vanishes determines whether the asymptotic properties of the monitoring procedure are determined by a deterministic or random function. Furthermore, we provide a theoretical result about the optimal kernel for a given alternative.     

Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

Stability of nonlinear AR‐GARCH models

Abstract. This article studies the stability of nonlinear autoregressive models with conditionally heteroskedastic errors. We consider a nonlinear autoregression of order p [AR(p)] with the conditional variance specified as a nonlinear first‐order generalized autoregressive conditional heteroskedasticity [GARCH(1,1)] model. Conditions under which the model is stable in the sense that its Markov chain representation is geometrically ergodic are provided. This implies the existence of an initial distribution such that the process is strictly stationary and β‐mixing. Conditions under which the stationary distribution has finite moments are also given. The results cover several nonlinear specifications recently proposed for both the conditional mean and conditional variance, and only require mild moment conditions.