Forecasting daily time series using periodic unobserved components time series models

A periodic time series analysis is explored in the context of unobserved components time series models that include stochastic time functions for trend, seasonal and irregular effects. Periodic time series models allow dynamic characteristics (autocovariances) to depend on the period of the year, month, week or day. In the standard multivariate approach one can interpret a periodic time series analysis as a simultaneous treatment of typically yearly time series where each series is related to a particular season. Here, the periodic analysis applies to a vector of monthly time series related to each day of the month. Particular focus is on the forecasting performance and therefore on the underlying periodic forecast function, defined by the in-sample observation weights for producing (multi-step) forecasts. These weight patterns facilitate the interpretation of periodic model extensions. A statistical state space approach is used to estimate the model and allows for irregularly spaced observations in daily time series. Recent algorithms are adopted for the computation of observation weights for forecasting based on state space models with regressor variables. The methodology is illustrated for daily Dutch tax revenues that appear to have periodic dynamic properties. The dimension of our periodic unobserved components model is relatively large as we allow each element (day) of the vector of monthly time series to have a changing seasonal pattern. Nevertheless, even with only five years of data we find that the increased periodic flexibility can help in out-of-sample forecasting for two extra years of data.     

On pseudo maximum likelihood estimation for multivariate time series models with conditional heteroskedasticity

We consider a general multivariate conditional heteroskedastic model under a conditional distribution that is not necessarily normal. This model contains autoregressive conditional heteroskedastic (ARCH) models as a special class. We use the pseudo maximum likelihood estimation method and derive a new estimator of the asymptotic variance matrix for the pseudo maximum likelihood estimator. We also study four special cases in this class, which are conditional heteroskedastic autoregressive moving-average models, regression models with ARCH errors, models with constant conditional correlations, and ARCH in mean models.     

Method of maximum likelihood for stationary time series models

Whittle [1] proposed a method of obtaining the likelihood function for a linear dynamic model (with rational pulse transfer function and excited by Gaussian signal). In this note a simple derivation of his result is given.     

Exact maximum likelihood estimation of partially nonstationary vector ARMA models

A useful class of partially nonstationary vector autoregressive moving average (VARMA) models is considered with regard to parameter estimation. An exact maximum likelihood (EML) approach is developed on the basis of a simple transformation applied to the error-correction representation of the models considered. The employed transformation is shown to provide a standard VARMA model with the important property that it is stationary. Parameter estimation can thus be carried out by applying standard EML methods to the stationary VARMA model obtained from the error-correction representation. This approach resolves at least two problems related to the current limited availability of EML estimation methods for partially nonstationary VARMA models. Firstly, it resolves the apparent impossibility of computing the exact log-likelihood for such models using currently available methods. And secondly, it resolves the inadequacy of considering lagged endogenous variables as exogenous variables in the error-correction representation. Theoretical discussion is followed by an example using a popular data set. The example illustrates the feasibility of the EML estimation approach as well as some of its potential benefits in cases of practical interest which are easy to come across. As in the case of stationary models, the proposed EML method provides estimated model structures that are more reliable and accurate than results produced by conditional methods.     

Recursive maximum likelihood parameter estimation for state space systems using polynomial chaos theory

This paper combines polynomial chaos theory with maximum likelihood estimation for a novel approach to recursive parameter estimation in state-space systems. A simulation study compares the proposed approach with the extended Kalman filter to estimate the value of an unknown damping coefficient of a nonlinear Van der Pol oscillator. The results of the simulation study suggest that the proposed polynomial chaos estimator gives comparable results to the filtering method but may be less sensitive to user-defined tuning parameters. Because this recursive estimator is applicable to linear and nonlinear dynamic systems, the authors portend that this novel formulation will be useful for a broad range of estimation problems.     

Traveling waves for epidemic models with nonlocal dispersal in time and space periodic habitats

This paper deals with the spatial spreading speed and traveling wave solutions of a general epidemic model with nonlocal dispersal in time and space periodic habitats. It should be mentioned that the existence of spreading speed and traveling wave solutions of nonlocal dispersal cooperative system in space–time periodic habitats have been established previously. In this paper, we further show that the epidemic system has a spreading speed $c^1\left(\xi \right)$ and for any $c>c^1\left(\xi \right)$, there exist a unique, continuous space–time periodic traveling wave solution $({\Phi }_1(x?ct\xi ,t,ct\xi ),{\Phi }_2(x?ct\xi ,t,ct\xi ))$ of epidemic model in the direction of $\xi$ with speed $c$, and there is no such solution for $c<c^1\left(\xi \right)$.     

A linearization procedure and a VDM/ECM algorithm for penalized and constrained nonparametric maximum likelihood estimation for mixture models

Suppose independent observations X_i, i=1,…,n are observed from a mixture model , where λ is a scalar and Q(λ) is a nondegenerate distribution with an unspecified form. We consider to estimate Q(λ) by nonparametric maximum likelihood (NPML) method under two scenarios: (1) the likelihood is penalized by a functional g(Q); and (2) Q is under a constraint g(Q)=g₀. We propose a simple and reliable algorithm termed VDM/ECM for Q-estimation when the likelihood is penalized by a linear functional. We show this algorithm can be applied to a more general situation where the penalty is not linear, but a function of linear functionals by a linearization procedure. The constrained NPMLE can be found by penalizing the quadratic distance |g(Q)-g₀|² under a large penalty factor γ>0 using this algorithm. The algorithm is illustrated with two real data sets.     

Attention based hybrid parametric and neural network models for non-stationary time series prediction

This paper investigates non-stationary time series analysis and forecasting techniques for financial datasets. We focus on the use of a popular non-stationary parametric model namely GARCH and neural network model LSTM, with an attention mechanism to capture the complex temporal dynamics and dependencies in the data. We propose a hybrid GARCH-ATT-LSTM model where the GARCH model is employed for volatility forecasting, attention mechanism is applied to capture the more important parts of the data sequence and enhance the interpretability of the model, and the LSTM model is used for price forecasting. Our experiments are conducted on real-world financial datasets, that is, Apple stock price, Dow Jones index, and gold futures price. We compare the performance of GARCH-ATT-LSTM against the sole LSTM model, ATT-LSTM model, and LSTM-GARCH model. Our results show that GARCH-ATT-LSTM outperforms the baseline methods and achieves high accuracy in price forecasting. It implies the effectiveness of the attention mechanism in improving the interpretability and stability of the model and the success of combining parametric models with neural network models. The findings suggest that GARCH-ATT-LSTM can be a valuable tool for non-stationary time series analysis and forecasting in financial applications.     

A data cloning algorithm for computing maximum likelihood estimates in spatial generalized linear mixed models

Non-Gaussian spatial data are common in many sciences such as environmental sciences, biology and epidemiology. Spatial generalized linear mixed models (SGLMMs) are flexible models for modeling these types of data. Maximum likelihood estimation in SGLMMs is usually made cumbersome due to the high-dimensional intractable integrals involved in the likelihood function and therefore the most commonly used approach for estimating SGLMMs is based on the Bayesian approach. This paper proposes a computationally efficient strategy to fit SGLMMs based on the data cloning (DC) method suggested by Lele et al. (2007). This method uses Markov chain Monte Carlo simulations from an artificially constructed distribution to calculate the maximum likelihood estimates and their standard errors. In this paper, the DC method is adapted and generalized to estimate SGLMMs and some of its asymptotic properties are explored. Performance of the method is illustrated by a set of simulated binary and Poisson count data and also data about car accidents in Mashhad, Iran. The focus is inference in SGLMMs for small and medium data sets.     

Robust maximum likelihood estimation for stochastic state space model with observation outliers

The objective of this paper is to develop a robust maximum likelihood estimation (MLE) for the stochastic state space model via the expectation maximisation algorithm to cope with observation outliers. Two types of outliers and their influence are studied in this paper: namely,the additive outlier (AO) and innovative outlier (IO). Due to the sensitivity of the MLE to AO and IO, we propose two techniques for robustifying the MLE: the weighted maximum likelihood estimation (WMLE) and the trimmed maximum likelihood estimation (TMLE). The WMLE is easy to implement with weights estimated from the data; however, it is still sensitive to IO and a patch of AO outliers. On the other hand, the TMLE is reduced to a combinatorial optimisation problem and hard to implement but it is efficient to both types of outliers presented here. To overcome the difficulty, we apply the parallel randomised algorithm that has a low computational cost. A Monte Carlo simulation result shows the efficiency of the proposed algorithms.     

Maximum likelihood estimation of an extended latent Markov model for clustered binary panel data

Computational aspects concerning a model for clustered binary panel data are analyzed. The model is based on the representation of the behavior of a subject (individual panel member) in a given cluster by means of a latent process. This latent process is decomposed into a cluster-specific component and an individual-specific component. The first component follows a first-order Markov chain, whereas the second is time-invariant and is represented by a discrete random variable. An algorithm for computing the joint distribution of the response variables is introduced. The algorithm may be used even in the presence of a large number of subjects in the same cluster. An Expectation-Maximization (EM) scheme for the maximum likelihood estimation of the model is also described together with the estimation of the Fisher information matrix on the basis of the numerical derivative of the score vector. The estimate of this matrix is used to obtain standard errors for the parameter estimates and to check the identifiability of the model and the convergence of the EM algorithm. The approach is illustrated by means of an application to a data set concerning Italian employees’ illness benefits.     

Performance of state space and ARIMA models for consumer retail sales forecasting

Forecasting future sales is one of the most important issues that is beyond all strategic and planning decisions in effective operations of retail businesses. For profitable retail businesses, accurate demand forecasting is crucial in organizing and planning production, purchasing, transportation and labor force. Retail sales series belong to a special type of time series that typically contain trend and seasonal patterns, presenting challenges in developing effective forecasting models. This work compares the forecasting performance of state space models and ARIMA models. The forecasting performance is demonstrated through a case study of retail sales of five different categories of women footwear: Boots, Booties, Flats, Sandals and Shoes. On both methodologies the model with the minimum value of Akaike's Information Criteria for the in-sample period was selected from all admissible models for further evaluation in the out-of-sample. Both one-step and multiple-step forecasts were produced. The results show that when an automatic algorithm the overall out-of-sample forecasting performance of state space and ARIMA models evaluated via RMSE, MAE and MAPE is quite similar on both one-step and multi-step forecasts. We also conclude that state space and ARIMA produce coverage probabilities that are close to the nominal rates for both one-step and multi-step forecasts.     

混沌时间序列预测模型的比较研究

针对目前混沌时间序列预测模型预测结果差异较大的问题,归纳了4种混沌时间序列预测模型：BRF神经网络模型、最大Lyapunov指数模型、局域线性模型和Volterra滤波器自适应预测模型,并对这4种预测模型进行了比较研究。应用4种预测模型对几个典型的非线性系统进行预测仿真。结果表明,这4种预测模型对典型混沌时间序列预测都具有很好的预测效果;在预测精度上BRF模型和Volterra模型明显优于最大Lyapunov指数模型和局域线性模型。     

An analysis of separable least squares data driven local coordinates for maximum likelihood estimation of linear systems

In this paper we study a novel parametrization for state-space systems, namely separable least squares data driven local coordinates (slsDDLC). The parametrization by slsDDLC has recently been successfully applied to maximum likelihood estimation of linear dynamic systems. In a simulation study, the use of slsDDLC has led to numerical advantages in comparison to the use of more conventional parametrizations, including data driven local coordinates (DDLC). However, an analysis of properties of slsDDLC, which are relevant to identification, has not been performed up to now. In this paper, we provide insights into the geometry and topology of the slsDDLC construction and show a number of results which are important for actual identification, in particular for maximum likelihood estimation. We also prove that the separable least squares methodology is indeed guaranteed to be applicable to maximum likelihood estimation of linear dynamic systems in typical situations.     

Prediction method of autoregressive moving average models for uncertain time series

ABSTRACT

Time series analysis is based on the continuous regularity of the development of objective things to predict the next value depending on observed values. Based on time series analysis, we present autoregressive moving average models to predict the next future value for an uncertain time series. In this paper, imprecise observations and disturbance terms are regarded as uncertain variables and assume that the latter are satisfied uncertain normal distribution. The prediction models of uncertain time series are established combining the knowledge of autoregressive model and uncertainty theory. Therefore, the interval range of the next future value is predicted based on the reliability constraint. As an illustration to compare with the numerical examples of the existing prediction method, the innovations and effectiveness of the work are further demonstrated by the computational results.     

A new class of hybrid models for time series forecasting

Applying quantitative models for forecasting and assisting investment decision making has become more indispensable in business practices than ever before. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to overcome the deficiencies of single models and yield hybrid models that are more accurate. In this paper, in contrast of the traditional hybrid models, a new methodology is proposed in order to construct a new class of hybrid models using a time series model as basis model and a classifier. As classifiers cannot be lonely applied as forecasting model for continuous problems, in the first stage of the proposed model, a forecasting model is used as basis model. Then, the estimated values of the basis model are modified in the second stage, based on the distinguished trend of the residuals of the basis model and the optimum step length, which are respectively calculated by a classifier model and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than its basis time series model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Bayesian estimation of unrestricted and order-restricted association models for a two-way contingency table

In two-way contingency tables analysis, a popular class of models for describing the structure of the association between the two categorical variables are the so-called “association” models. Such models assign scores to the classification variables which can be either fixed and prespecified or unknown parameters to be estimated. Under the row-column (RC) association model, both row and column scores are unknown parameters without any restriction concerning their ordinality. It is natural to impose order restrictions on the scores when the classification variables are ordinal. The Bayesian approach for the RC (unrestricted and restricted) model is adopted. MCMC methods are facilitated in order the parameters to be estimated. Furthermore, an alternative parametrization of the association models is proposed. This new parametrization simplifies computation in the MCMC procedure and leads to a natural parameter space for the order constrained model. The proposed methodology is illustrated via a popular dataset.     

Novel approaches for online playout delay prediction in VoIP applications using time series models

Voice over IP (VoIP) applications requires a buffer at the receiver to minimize the packet loss due to late arrival. Several algorithms are available in the literature to estimate the playout buffer delay. Classic estimation algorithms are non-adaptive, i.e. they differ from more recent approaches basically due to the absence of learning mechanisms. This paper introduces two new formulations of adaptive algorithms for online learning and prediction of the playout buffer delay, the first one being based on the standard Box-Jenkins autoregressive model, while the second one being based on the feedforward and recurrent neural networks. The obtained results indicate that the proposed algorithms present better overall performance than the classic ones.     

State estimation using model order reduction for unstable systems

The problem of state estimation occurs in many applications of fluid flow. For example, to produce a reliable weather forecast it is essential to find the best possible estimate of the true state of the atmosphere. To find this best estimate a nonlinear least squares problem has to be solved subject to dynamical system constraints. Usually this is solved iteratively by an approximate Gauss–Newton method where the underlying discrete linear system is in general unstable. In this paper we propose a new method for deriving low order approximations to the problem based on a recently developed model reduction method for unstable systems. To illustrate the theoretical results, numerical experiments are performed using a two-dimensional Eady model – a simple model of baroclinic instability, which is the dominant mechanism for the growth of storms at mid-latitudes. It is a suitable test model to show the benefit that may be obtained by using model reduction techniques to approximate unstable systems within the state estimation problem.