Mixtures-of-experts of autoregressive time series: asymptotic normality and model specification

We consider a class of nonlinear models based on mixtures of local autoregressive time series. At any given time point, we have a certain number of linear models, denoted as experts, where the vector of covariates may include lags of the dependent variable. Additionally, we assume the existence of a latent multinomial variable, whose distribution depends on the same covariates as the experts, that determines which linear process is observed. This structure, denoted as mixture-of-experts (ME), is considerably flexible in modeling the conditional mean function, as shown by Jiang and Tanner. We present a formal treatment of conditions to guarantee the asymptotic normality of the maximum likelihood estimator (MLE), under stationarity and nonstationarity, and under correct model specification and model misspecification. The performance of common model selection criteria in selecting the number of experts is explored via Monte Carlo simulations. Finally, we present applications to simulated and real data sets, to illustrate the ability of the proposed structure to model not only the conditional mean, but also the whole conditional density.     

Detection in sea clutter based on nonlinear ARCH model

In this paper, a complex nonlinear autoregressive conditional heteroscedasticity (CNARCH) model is proposed to model sea clutter. For heteroscedastic model, since the likelihood function is not obtained from explicit probability density function (PDF) expression, it is typically referred to as a quasi-likelihood function. The corresponding quasi-maximum likelihood estimation (QMLE) of the model parameters is derived. Furthermore, the corresponding detection algorithm is derived based on this model. We also conduct the simulations of both synthetic and practical data, demonstrate that the proposed model offers higher accuracy in detection, than the linear ARCH model, when used in the sea clutter.     

Order and delay selection for neural network modelling by identification of linearized models

An approach to selecting the order and delay for neural network modelling of nonlinear dynamic systems is proposed by identifying local linear models at points spanning the system operating range. The method is based on relationships between linear and nonlinear models and is developed for three popular nonlinear model structures; nonlinear autoregressive with exogenous inputs (NARX), NARX with a linear noise model and nonlinear autoregressive moving-average with exogenous inputs (NARMAX). Simulation results illustrate the application of the method, and the suitability of the orders and delays selected are demonstrated by nonlinear system identification using radial basis function neural networks. The method is also shown to indicate the suitability of a particular nonlinear model structure for representing a system.     

A Long Memory Model with Normal Mixture GARCH

We present an exploratory analysis of a class of long memory models with a normal mixture generalized autoregressive conditional heteroskedasticity innovation process. Monte Carlo results are used to infer the performance of the maximum likelihood estimator. The estimation biases are associated with, amongst others, the mixing parameter, and these biases are usually insignificant. As an illustration, we fit the proposed model to four countries inflation data. It is found that the performance of the long memory model with normal mixture generalized autoregressive conditional heteroskedasticity is better than, say, both autoregressive moving average and long memory models with a standard generalized autoregressive conditional heteroskedasticity specification in terms of the flexibility to describe both the time-varying conditional skewness and kurtosis.     

Experiments on the application of IOHMMs to model financial returnsseries

Input-output hidden Markov models (IOHMM) are conditional hidden Markov models in which the emission (and possibly the transition) probabilities can be conditioned on an input sequence. For example, these conditional distributions can be linear, logistic, or nonlinear (using for example multilayer neural networks). We compare the generalization performance of several models which are special cases of input-output hidden Markov models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden Markov model, and various IOHMMs. The experiments compare these models on predicting the conditional density of returns of market and sector indices. Note that the unconditional Gaussian estimates the first moment with the historical average. The results show that, although for the first moment the historical average gives the best results, for the higher moments, the IOHMMs yielded significantly better performance, as estimated by the out-of-sample likelihood.     

Modeling threshold conditional heteroscedasticity with regime-dependent skewness and kurtosis

Construction of nonlinear time series models with a flexible probabilistic structure is an important challenge for statisticians. Applications of such a time series model include ecology, economics and finance. In this paper we consider a threshold model for all the first four conditional moments of a time series. The nonlinear structure in the conditional mean is specified by a threshold autoregression and that of the conditional variance by a threshold generalized autoregressive conditional heteroscedastic (GARCH) model. There are many options for the conditional innovation density in the modeling of the skewness and kurtosis such as the Gram-Charlier (GC) density and the skewed-t density. The Gram-Charlier (GC) density allows explicit modeling of the skewness and kurtosis parameters and therefore is the main focus of this paper. However, its performance is compared with that of Hansen’s skewed-t distribution in the data analysis section of the paper. The regime-dependent feature for the first four conditional moments allows more flexibility in modeling and provides better insights into the structure of a time series. A Lagrange multiplier (LM) test is developed for testing for the presence of threshold structure. The test statistic is similar to the classical tests for the presence of a threshold structure but allowing for a more general regime-dependent structure. The new model and the LM test are illustrated using the Dow Jones Industrial Average, the Hong Kong Hang Seng Index and the Yen/US exchange rate.     

Comparison of alternative ACD models via density and interval forecasts: Evidence from the Australian stock market

In this paper a number of alternative autoregressive conditional duration (ACD) models are compared using a sample of data for three major companies traded on the Australian Stock Exchange. The comparison is performed by employing the methodology for evaluating density and interval forecasts, developed by Diebold et al. [F. Diebold, A. Gunther, S. Tay, Evaluating density forecasts with applications to financial risk management, International Economic Review 39 (1998) 863–883] and Christoffersen [P. Christoffersen, Evaluating interval forecasts, International Economic Review 39 (1998) 841–862], respectively. Our main finding is that the generalized gamma and log-normal distributions for the error terms have similar performance and perform better that the exponential and Weibull distributions. Additionally, there seems to be no substantial difference between the standard ACD specification of Engle and Russel [R. Engle, J. Russell, Autoregressive conditional duration: a new model for irregularly-spaced transaction data, Econometrica 66 (1998) 1127–1162] and the log-ACD specification of Bauwens and Giot [L. Bauwens, P. Giot, The logarithmic ACD model: an application to the bid-ask quote process of three NYSE stocks, Annales d’Economie et de Statistique 60 (2000) 117–150].     

A parameter optimization method for radial basis function type models

This paper considers the nonlinear systems modeling problem for control. A structured nonlinear parameter optimization method (SNPOM) adapted to radial basis function (RBF) networks and an RBF network-style coefficients autoregressive model with exogenous variable model parameter estimation is presented. This is an off-line nonlinear model parameter optimization method, depending partly on the Levenberg-Marquardt method for nonlinear parameter optimization and partly on the least-squares method using singular value decomposition for linear parameter estimation. When compared with some other algorithms, the SNPOM accelerates the computational convergence of the parameter optimization search process of RBF-type models. The usefulness of this approach is illustrated by means of several examples.     

On pseudo maximum likelihood estimation for multivariate time series models with conditional heteroskedasticity

We consider a general multivariate conditional heteroskedastic model under a conditional distribution that is not necessarily normal. This model contains autoregressive conditional heteroskedastic (ARCH) models as a special class. We use the pseudo maximum likelihood estimation method and derive a new estimator of the asymptotic variance matrix for the pseudo maximum likelihood estimator. We also study four special cases in this class, which are conditional heteroskedastic autoregressive moving-average models, regression models with ARCH errors, models with constant conditional correlations, and ARCH in mean models.     

A self-organising mixture autoregressive network for FX time series modelling and prediction

Nowadays a great deal of effort has been made in order to gain advantages in foreign exchange (FX) rates predictions. However, most existing techniques seldom excel the simple random walk model in practical applications. This paper describes a self-organising network formed on the basis of a mixture of adaptive autoregressive models. The proposed network, termed self-organising mixture autoregressive (SOMAR) model, can be used to describe and model nonstationary, nonlinear time series by means of a number of underlying local regressive models. An autocorrelation coefficient-based measure is proposed as the similarity measure for assigning input samples to the underlying local models. Experiments on both benchmark time series and several FX rates have been conducted. The results show that the proposed method consistently outperforms other local time series modelling techniques on a range of performance measures including the mean-square-error, correct trend predication percentage, accumulated profit and model variance.     

Boosting nonlinear additive autoregressive time series

Several methods for the analysis of nonlinear time series models have been proposed. As in linear autoregressive models the main problems are model identification, estimation and prediction. A boosting method is proposed that performs model identification and estimation simultaneously within the framework of nonlinear autoregressive time series. The method allows one to select influential terms from a large number of potential lags and exogenous variables. The influence of the selected terms is modeled by an expansion in basis function allowing for a flexible additive form of the predictor. The approach is very competitive in particular in high dimensional settings where alternative fitting methods fail. This is demonstrated by means of simulations and two applications to real world data.     

Improving prediction of exchange rates using Differential EMD

Volatility is a key parameter when measuring the size of errors made in modelling returns and other financial variables such as exchanged rates. The autoregressive moving-average (ARMA) model is a linear process in time series; whilst in the nonlinear system, the generalised autoregressive conditional heteroskedasticity (GARCH) and Markov switching GARCH (MS-GARCH) have been widely applied. In statistical learning theory, support vector regression (SVR) plays an important role in predicting nonlinear and nonstationary time series variables. In this paper, we propose a new algorithm, differential Empirical Mode Decomposition (EMD) for improving prediction of exchange rates under support vector regression (SVR). The new algorithm of Differential EMD has the capability of smoothing and reducing the noise, whereas the SVR model with the filtered dataset improves predicting the exchange rates. Simulations results consisting of the Differential EMD and SVR model show that our model outperforms simulations by a state-of-the-art MS-GARCH and Markov switching regression (MSR) models.     

On a constrained mixture vector autoregressive model

A mixture vector autoregressive model has recently been introduced to the literature. Although this model is a promising candidate for nonlinear multiple time series modeling, high dimensionality of the parameters and lack of method for computing the standard errors of estimates limit its application to real data. The contribution of this paper is threefold. First, a form of parameter constraints is introduced with an efficient EM algorithm for estimation. Second, an accurate method for computing standard errors is presented for the model with and without parameter constraints. Lastly, a hypothesis-testing approach based on likelihood ratio tests is proposed, which aids in the selection of unnecessary parameters and leads to the greater efficiency at the estimation. A case study employing U.S. Treasury constant maturity rates illustrates the applicability of the mixture vector autoregressive model with parameter constraints, and the importance of using a reliable method to compute standard errors.     

Limited memory optimal filtering

Linear and nonlinear optimal filters with limited memory length are developed. The filter output is the conditional probability density function and, in the linear Gaussian case, is the conditional mean and covariance matrix where the conditioning is only on a fixed amount of most recent data. This is related to maximum-likelihood least-squares estimation. These filters have application in problems where standard filters diverge due to dynamical model errors. This is demonstrated via numerical simulations.     

A post nonlinear geometric algorithm for independent component analysis

Simple linear independent component analysis (ICA) algorithms work efficiently only in linear mixing environments. Whereas, a nonlinear ICA model, which is more complicated, would be more practical for general applications as it can work with both linear and nonlinear mixtures. In this paper, we introduce a novel method for nonlinear ICA problem. The proposed method follows the post nonlinear approach to model the mixtures, and exploits the difference between a linear mixture and a nonlinear one from their nature of distributions in a multidimensional space to develop a separation scheme. The nonlinear mixture is represented by a nonlinear surface while the linear mixture is represented by a plane. A geometric learning algorithm named as post nonlinear geometric ICA (pnGICA) is developed by geometrically transforming the nonlinear surface to a plane, i.e., to a linear mixture. Computer simulations of the algorithm provide promising performance on different data sets.     

A hybrid linear-neural model for time series forecasting

This paper considers a linear model with time varying parameters controlled by a neural network to analyze and forecast nonlinear time series. We show that this formulation, called neural coefficient smooth transition autoregressive model, is in close relation to the threshold autoregressive model and the smooth transition autoregressive model with the advantage of naturally incorporating linear multivariate thresholds and smooth transitions between regimes. In our proposal, the neural-network output is used to induce a partition of the input space, with smooth and multivariate thresholds. This also allows the choice of good initial values for the training algorithm.     

基于小波神经网络的非线性误差校正模型及其预测

针对非线性系统的预测问题,在线性和非线性协整理论涵义的基础上,提出利用小波神经网络进行非线性协整系统的非线性误差校正模型的研究,并给出该模型的建模方法．对沪深股市进行实证研究,与线性向量自回归模型进行比较．研究证明,小波神经网络所建立的非线性误差校正模型有较好的预测效果,能够有效地预测非线性经济系统．     

Bayesian estimation of the Gaussian mixture GARCH model

Bayesian inference and prediction for a generalized autoregressive conditional heteroskedastic (GARCH) model where the innovations are assumed to follow a mixture of two Gaussian distributions is performed. The mixture GARCH model can capture the patterns usually exhibited by many financial time series such as volatility clustering, large kurtosis and extreme observations. A Griddy-Gibbs sampler implementation is proposed for parameter estimation and volatility prediction. Bayesian prediction of the Value at Risk is also addressed providing point estimates and predictive intervals. The method is illustrated using the Swiss Market Index.     

Exchange rate prediction using hybrid neural networks and trading indicators

This paper describes a hybrid model formed by a mixture of various regressive neural network models, such as temporal self-organising maps and support vector regressions, for modelling and prediction of foreign exchange rate time series. A selected set of influential trading indicators, including the moving average convergence/divergence and relative strength index, are also utilised in the proposed method. A genetic algorithm is applied to fuse all the information from the mixture regression models and the economical indicators. Experimental results and comparisons show that the proposed method outperforms the global modelling techniques such as generalised autoregressive conditional heteroscedasticity in terms of profit returns. A virtual trading system is built to examine the performance of the methods under study.