Conditional variance estimation in regression models with long memory

In this article we study asymptotic properties of a non‐parametric kernel estimator of the conditional variance in a random design model with parametric mean and heteroscedastic errors, for a class of long‐memory errors and predictors. We establish small and large bandwidths asymptotics, which show a different behaviour compared with that of kernel estimators of the conditional mean. We distinguish between an oracle case (i.e. where the errors are directly observed) and a non‐oracle case (where the errors are replaced with residuals) and show non‐equivalence between the oracle and non‐oracle case. We also discuss a practical problem of bandwidth choice. Theoretical results are justified by simulation studies. We apply our theory to DJA and FTSE indices.     

Nonparametric Sequential Minimax Estimation of the Drift Coefficient in Diffusion Processes

Abstract

A sequential procedure for estimating the drift function of a diffusion process is constructed. The asymptotic properties are established, such as the optimal covergence rate for a global risk and the asymptotic efficiency for a local risk of the procedure as well as the asymptotic normality of observation time.

     

Local Likelihood for non‐parametric ARCH(1) models

Abstract. We propose a non‐parametric local likelihood estimator for the log‐transformed autoregressive conditional heteroscedastic (ARCH) (1) model. Our non‐parametric estimator is constructed within the likelihood framework for non‐Gaussian observations: it is different from standard kernel regression smoothing, where the innovations are assumed to be normally distributed. We derive consistency and asymptotic normality for our estimators and show, by a simulation experiment and some real‐data examples, that the local likelihood estimator has better predictive potential than classical local regression. A possible extension of the estimation procedure to more general multiplicative ARCH(p) models with p > 1 predictor variables is also described.     

NORMING RATES AND LIMIT THEORY FOR SOME TIME‐VARYING COEFFICIENT AUTOREGRESSIONS

A time‐varying autoregression is considered with a similarity‐based coefficient and possible drift. It is shown that the random‐walk model has a natural interpretation as the leading term in a small‐sigma expansion of a similarity model with an exponential similarity function as its AR coefficient. Consistency of the quasi‐maximum likelihood estimator of the parameters in this model is established, the behaviours of the score and Hessian functions are analysed and test statistics are suggested. A complete list is provided of the normalization rates required for the consistency proof and for the score and Hessian function standardization. A large family of unit root models with stationary and explosive alternatives is characterized within the similarity class through the asymptotic negligibility of a certain quadratic form that appears in the score function. A variant of the stochastic unit root model within the class is studied, and a large‐sample limit theory provided, which leads to a new nonlinear diffusion process limit showing the form of the drift and conditional volatility induced by sustained stochastic departures from unity. The findings provide a composite case for time‐varying coefficient dynamic modelling. Some simulations and a brief empirical application to data on international Exchange Traded Funds are included. Copyright © 2014 Wiley Publishing Ltd     

JOINT HYPOTHESIS TESTS FOR A RANDOM WALK BASED ON INSTRUMENTAL VARIABLE ESTIMATORS

Abstract. In this paper we propose test statistics for the null hypothesis of a random walk or a random walk with drift for the case in which the innovations to the series are a moving-average process. The statistics are based on the instrumental variable estimators proposed by Hall and by Pantula and Hall and are shown to have the limiting distributions tabulated by Dickey and Fuller.     

Fault Detection of Non‐Linear Processes Using Kernel Independent Component Analysis

In this paper, a new non‐linear process monitoring method based on kernel independent component analysis (KICA) is developed. Its basic idea is to use KICA to extract some dominant independent components capturing non‐linearity from normal operating process data and to combine them with statistical process monitoring techniques. The proposed method is applied to the fault detection in the Tennessee Eastman process and is compared with PCA, modified ICA, and KPCA. The proposed approach effectively captures the non‐linear relationship in the process variables and showed superior fault detectability compared to other methods while attaining comparable false alarm rates.     

Residual Empirical Processes and Weighted Sums for Time‐Varying Processes with Applications to Testing for Homoscedasticity

In the context of heteroscedastic time‐varying autoregressive (AR)‐process we study the estimation of the error/innovation distributions. Our study reveals that the non‐parametric estimation of the AR parameter functions has a negligible asymptotic effect on the estimation of the empirical distribution of the residuals even though the AR parameter functions are estimated non‐parametrically. The derivation of these results involves the study of both function‐indexed sequential residual empirical processes and weighted sum processes. Exponential inequalities and weak convergence results are derived. As an application of our results we discuss testing for the constancy of the variance function, which in special cases corresponds to testing for stationarity.     

Efficient recursive kernel canonical variate analysis for monitoring nonlinear time‐varying processes

Kernel canonical variate analysis (KCVA) cannot be adopted for monitoring nonlinear time‐varying processes because of changes in variance, mean, and correlation between variables. Efficient recursive kernel canonical variate analysis (ERKCVA) is thus proposed to monitor the nonlinear time‐varying processes. In a high‐dimensional feature space, the covariance matrix can be updated recursively by the exponentially weighted moving average approach. The first‐order perturbation theory is introduced to obtain the recursive singular value decomposition of the Hankel matrix, which can significantly reduce the computational cost of the proposed method. Prediction errors and state variables are non‐Gaussian; thus, upper control limits can be derived from the estimated probability density function by kernel density estimation. The proposed method is demonstrated by simulating a continuous stirred tank reactor. Simulation results indicate that ERKCVA could efficiently capture the predefined normal and natural changes in nonlinear time‐varying processes. In addition, ERKCVA can also identify 4 types of sensor faults.     

Sequential model selection method for nonparametric autoregression

Abstract

In this article, the nonparametric autoregression estimation problem for quadratic risks is considered. To this end, we develop a new adaptive sequential model selection method based on the efficient sequential kernel estimators proposed by Arkoun and Pergamenshchikov (2016). Moreover, we develop a new analytical tool for general regression models to obtain the non-asymptotic sharp oracle inequalities for both usual quadratic and robust quadratic risks. Then, we show that the constructed sequential model selection procedure is optimal in the sense of oracle inequalities.     

A Note on Non‐Negative Arma Processes

Abstract. Recently, there has been much research on developing models suitable for analysing the volatility of a discrete‐time process. Since the volatility process, like many others, is necessarily non‐negative, there is a need to construct models for stationary processes which are non‐negative with probability one. Such models can be obtained by driving autoregressive moving average (ARMA) processes with non‐negative kernel by non‐negative white noise. This raises the problem of finding simple conditions under which an ARMA process with given coefficients has a non‐negative kernel. In this article, we derive a necessary and sufficient condition. This condition is in terms of the generating function of the ARMA kernel which has a simple form. Moreover, we derive some readily verifiable necessary and sufficient conditions for some ARMA processes to be non‐negative almost surely.     

sequential estimation of the hgarginal density function for a strongly mixing process

Suppose that {X_n} is a strongly mixing process with unknow marginal density f(x) and that we estimate f(x) by a kernel estimator [fcirc]_n(x|h_n)and want to achive the MISE no larger than some preassigned postive number w. However,the appropriate sample size n^*depends on a functional of the unknow density function. Therefore some sequential procedure is required and we adopt a fully sequential procedure. In this paper we investigate the asymptotic properties of the procedure and show that the producure is asymptotically efficient in a certain sense as w→0. The results are almost the same in the i.i.d. setting. our result extend a class of models to which the methodology can be applied. For example economic variable,experiments on a single subject in which obervation are not indepent, and so on.     

On detecting the optimal structure of a neural network under strong statistical features in errors

The purpose of this article is to investigate the empirical performance of various statistical techniques for detecting the optimal structure of a neural network (NN) regression model. We are particularly concerned with the specification of the NN architecture when the error component is characterized by special statistical properties, such as heteroskedasticity and non‐normality. We consider the sequential testing procedure based on standard Lagrange multiplier (LM) tests for neglected nonlinearity and also examine three modifications of this test that are robust to heteroskedasticity. By means of Monte Carlo simulations, we investigate the ability of these procedures to detect the right structure of the NN under different types of heteroskedasticity and noise distributions. Simulation results show that robustified LM tests allow the researcher to control the complexity of the NN without having to explicitly model all statistical aspects of the data‐generating process, something which is not generally feasible with the standard LM test. The combination of robust regression‐based testing with bootstrapping and generalized autoregressive conditional heteroskedasticity modelling techniques increases the efficiency of the statistical sequential procedure in eliciting the optimal NN architecture.     

Sequentially Updated Residuals and Detection of Stationary Errors in Polynomial Regression Models

Abstract

The question whether a time series behaves as a random walk or as a stationary process is an important and delicate problem, particularly arising in financial statistics, econometrics, and engineering. This article studies the problem to detect sequentially that the error terms in a polynomial regression model no longer behave as a random walk but as a stationary process. We provide the asymptotic distribution theory for a Monitoring procedure given by a control chart; i.e., a stopping time, which is related to a well-known unit root test statistic calculated from sequentially updated residuals. We provide a functional central limit theorem for the corresponding stochastic process that implies a central limit theorem for the control chart. The finite sample properties are investigated by a simulation study.     

Learning a data-dependent kernel function for KPCA-based nonlinear process monitoring

Kernel principal component analysis (KPCA)-based process monitoring methods have recently shown to be very effective for monitoring nonlinear processes. However, their performances largely depend on the kernel function and currently there is no general rule for kernel selection. Existing methods simply choose the kernel function empirically or experimentally from a given set of candidates. This paper proposes a kernel function learning method for KPCA to learn a kernel function tailored to specific data and explores its potential for KPCA-based process monitoring. Motivated by the manifold learning method maximum variance unfolding (MVU), we obtain the kernel function by optimizing over a family of data-dependent kernels such that the nonlinear structure in input data is unfolded in the kernel feature space and gets more likely to be linear there. Using the optimized kernel, the nonlinear principal components of KPCA which are linear principal components in the kernel feature space can effectively capture the variation in data, and thus the data under normal operating conditions can be more precisely modeled by KPCA for process monitoring. Simulation results on an simple nonlinear system and the benchmark Tennessee Eastman (TE) demonstrate that the optimized kernel functions lead to significant improvement in the performance over the popular Gaussian kernels when used in the KPCA-based process monitoring.     

Inference on Multivariate Heteroscedastic Time Varying Random Coefficient Models

In this article, we introduce the general setting of a multivariate time series autoregressive model with stochastic time‐varying coefficients and time‐varying conditional variance of the error process. This allows modelling VAR dynamics for non‐stationary time series and estimation of time‐varying parameter processes by the well‐known rolling regression estimation techniques. We establish consistency, convergence rates, and asymptotic normality for kernel estimators of the paths of coefficient processes and provide pointwise valid standard errors. The method is applied to a popular seven‐variable dataset to analyse evidence of time variation in empirical objects of interest for the DSGE (dynamic stochastic general equilibrium) literature.     

A Note on Mean-squared Prediction Errors of the Least Squares Predictors in Random Walk Models

An asymptotic expression for the mean-squared prediction error (MSPE) of the least squares predictor is obtained in the random walk model. It is shown that the term of order 1/ n in this error, where n is the sample size, is twice as large as the one obtained from the first-order autoregressive (AR(1)) model satisfying the stationary assumption. Moreover, while the correlation between the squares of the (normalized) regressor variable and normalized least squares estimator is asymptotically negligible in the stationary AR(1) model, we have found that the correlation has significantly negative value in the random walk model. To obtain these results, a new methodology, which is found to be useful in dealing with the moment properties of a strongly dependent process, is introduced.     

SIGNIFICANT VARIABLE SELECTION AND AUTOREGRESSIVE ORDER DETERMINATION FOR TIME‐SERIES PARTIALLY LINEAR MODELS

This paper is concerned with the regression coefficient and autoregressive order shrinkage and selection via the smoothly clipped absolute deviation (SCAD) penalty for a partially linear model with time‐series errors. By combining the profile semi‐parametric least squares method and SCAD penalty technique, a new penalized estimation for the regression and autoregressive parameters in the model is proposed. We show that the asymptotic property of the resultant estimator is the same as if the order of autoregressive error structure and non‐zero regression coefficients are known in advance, thus achieving the oracle property in the sense of Fan and Li (2001). In addition, based on a prewhitening technique, we construct a two‐stage local linear estimator (TSLLE) for the non‐parametric component. It is shown that the TSLLE is more asymtotically efficient than the one that ignores the autoregressive time‐series error structure. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on electricity usage data is also illustrated. Copyright © 2014 Wiley Publishing Ltd     

A FIXED‐ b TEST FOR A BREAK IN LEVEL AT AN UNKNOWN TIME UNDER FRACTIONAL INTEGRATION

In this paper, we propose a test for a break in the level of a fractionally integrated process when the timing of the putative break is not known. This testing problem has received considerable attention in the literature in the case where the time series is weakly autocorrelated. Less attention has been given to the case where the underlying time series is allowed to be fractionally integrated. Here, valid testing can only be performed if the limiting null distribution of the level break test statistic is well defined for all values of the fractional integration exponent considered. However, conventional sup‐Wald type tests diverge when the data are strongly autocorrelated. We show that a sup‐Wald statistic, which is standardized using a non‐parametric kernel‐based long‐run variance estimator, does possess a well‐defined limit distribution, depending only on the fractional integration parameter, provided the recently developed fixed‐b asymptotic framework is applied. We give the appropriate asymptotic critical values for this sup‐Wald statistic and show that it has good finite sample size and power properties.     

Consistent Monitoring of Cointegrating Relationships: The US Housing Market and the Subprime Crisis

We propose a consistent monitoring procedure for structural change in a cointegrating relationship. The procedure is inspired by Chu et al. (1996) by being based on parameter estimation on a prebreak ‘calibration’ period. We use three modified least squares estimators to obtain nuisance parameter‐free limiting distributions. We study the asymptotic and finite sample properties of the procedures and finally apply the approach to monitor two‐fundamentals‐driven US housing prices cointegrating relationships over the period 1976:Q1–2010:Q4 using the data of Anundsen (2015). Depending on the relationship considered and the estimation method used, a break point is detected as early as 2003:Q2, that is, well before US housing prices started to fall in 2007.     

A STATE SPACE TIME SERIES MODELLING METHOD WITHOUT INDIVIDUAL DETRENDING

Abstract. A state space method for building time series models without detrending each component of data vectors individually is presented. The method uses the recent algorithm based on the singular-value decomposition of the Hankel matrix and a two-step sequential procedure suggested by the notion of dynamic aggregation. Some asymptotic properties of the estimators of the model parameter and error estimates are also presented.