Forecasting daily time series using periodic unobserved components time series models

A periodic time series analysis is explored in the context of unobserved components time series models that include stochastic time functions for trend, seasonal and irregular effects. Periodic time series models allow dynamic characteristics (autocovariances) to depend on the period of the year, month, week or day. In the standard multivariate approach one can interpret a periodic time series analysis as a simultaneous treatment of typically yearly time series where each series is related to a particular season. Here, the periodic analysis applies to a vector of monthly time series related to each day of the month. Particular focus is on the forecasting performance and therefore on the underlying periodic forecast function, defined by the in-sample observation weights for producing (multi-step) forecasts. These weight patterns facilitate the interpretation of periodic model extensions. A statistical state space approach is used to estimate the model and allows for irregularly spaced observations in daily time series. Recent algorithms are adopted for the computation of observation weights for forecasting based on state space models with regressor variables. The methodology is illustrated for daily Dutch tax revenues that appear to have periodic dynamic properties. The dimension of our periodic unobserved components model is relatively large as we allow each element (day) of the vector of monthly time series to have a changing seasonal pattern. Nevertheless, even with only five years of data we find that the increased periodic flexibility can help in out-of-sample forecasting for two extra years of data.     

The application of a general time series model to floodplain fisheries in the Amazon

Time series analysis is a common tool in environmental and ecological studies to construct models to explain and forecast serially correlated data. There are several statistical techniques that are used to deal with univariate and multivariate (more than one series) chronological patterns of fisheries data. In this paper, an additive stochastic model is proposed with explicative and predictive features to capture the main seasonal patterns and trends of a fisheries system in the Amazon. The model is constructed on the assumption that the multivariate response variable – vector containing fishery yield of eight periodic species and the total fishery yield – can be decomposed into three terms: an autoregression of the response vector, an exogenous environmental variable (river level), and a seasonal component (significant frequencies obtained by using spectral analysis and the periodogram indicating the regularity of periodic cycles in the natural and fisheries system). The estimation procedure is carried out via maximum likelihood estimation. The model explained, on average, 78% of the variability in yield of the study species. The model represents the optimal solution (minimum mean square mean error) among the class of all multivariate autoregressive processes with exogenous and seasonal variables. Predictions for one period ahead are provided to illustrate how the model works in practice.     

Exact maximum likelihood estimation of partially nonstationary vector ARMA models

A useful class of partially nonstationary vector autoregressive moving average (VARMA) models is considered with regard to parameter estimation. An exact maximum likelihood (EML) approach is developed on the basis of a simple transformation applied to the error-correction representation of the models considered. The employed transformation is shown to provide a standard VARMA model with the important property that it is stationary. Parameter estimation can thus be carried out by applying standard EML methods to the stationary VARMA model obtained from the error-correction representation. This approach resolves at least two problems related to the current limited availability of EML estimation methods for partially nonstationary VARMA models. Firstly, it resolves the apparent impossibility of computing the exact log-likelihood for such models using currently available methods. And secondly, it resolves the inadequacy of considering lagged endogenous variables as exogenous variables in the error-correction representation. Theoretical discussion is followed by an example using a popular data set. The example illustrates the feasibility of the EML estimation approach as well as some of its potential benefits in cases of practical interest which are easy to come across. As in the case of stationary models, the proposed EML method provides estimated model structures that are more reliable and accurate than results produced by conditional methods.     

Exact maximum likelihood estimation of partially nonstationary vector ARMA models

A useful class of partially nonstationary vector autoregressive moving average (VARMA) models is considered with regard to parameter estimation. An exact maximum likelihood (EML) approach is developed on the basis of a simple transformation applied to the error-correction representation of the models considered. The employed transformation is shown to provide a standard VARMA model with the important property that it is stationary. Parameter estimation can thus be carried out by applying standard EML methods to the stationary VARMA model obtained from the error-correction representation. This approach resolves at least two problems related to the current limited availability of EML estimation methods for partially nonstationary VARMA models. Firstly, it resolves the apparent impossibility of computing the exact log-likelihood for such models using currently available methods. And secondly, it resolves the inadequacy of considering lagged endogenous variables as exogenous variables in the error-correction representation. Theoretical discussion is followed by an example using a popular data set. The example illustrates the feasibility of the EML estimation approach as well as some of its potential benefits in cases of practical interest which are easy to come across. As in the case of stationary models, the proposed EML method provides estimated model structures that are more reliable and accurate than results produced by conditional methods.     

Linear dynamic harmonic regression

Among the alternative unobserved components formulations within the stochastic state space setting, the dynamic harmonic regression (DHR) model has proven to be particularly useful for adaptive seasonal adjustment, signal extraction, forecasting and back-casting of time series. First, it is shown how to obtain AutoRegressive moving average (ARMA) representations for the DHR components under a generalized random walk setting for the associated stochastic parameters; a setting that includes several well-known random walk models as special cases. Later, these theoretical results are used to derive an alternative algorithm, based on optimization in the frequency domain, for the identification and estimation of DHR models. The main advantages of this algorithm are linearity, fast computational speed, avoidance of some numerical issues, and automatic identification of the DHR model. The signal extraction performance of the algorithm is evaluated using empirical applications and comprehensive Monte Carlo simulation analysis.     

One-step approximations for detecting regime changes in the state space model with application to the influenza data

Kim and Nelson [1999. State Space Models with Regime Switching. MIT Press, Cambridge, MA] and others extended the framework of state space models involving independent regime changes to the Markov dependent case. The cost of dealing with state space models with Markov switching is high in computational effort because of the number of the possible paths through the chain. Thus it is necessary to make some approximations in order to obtain a computationally feasible algorithm for estimation. The approximations depend on modified smoothing and filtering recursions that can be easily incorporated into an EM algorithm for maximum likelihood estimation. To investigate the accuracy of approximations, we develop a new method to obtain more exact solutions, and then compare the two methods. We apply both methods to a simulated series. The result shows that employing the approximation-based algorithm not only provides accurate results but also leads to a significant reduction in the computational costs. We also apply the methods to an influenza mortality series, in which we develop a model that is general enough to include most structural models useful in monitoring changes of regime. The model proposed has the flexibility to deal with a wide range of problems involving possible regime shifts in pattern that may be seen to occur in many biological, medical and epidemiological studies.     

Factorial Hidden Markov Models

Hidden Markov models (HMMs) have proven to be one of the most widely used tools for learning probabilistic models of time series data. In an HMM, information about the past is conveyed through a single discrete variable—the hidden state. We discuss a generalization of HMMs in which this state is factored into multiple state variables and is therefore represented in a distributed manner. We describe an exact algorithm for inferring the posterior probabilities of the hidden state variables given the observations, and relate it to the forward–backward algorithm for HMMs and to algorithms for more general graphical models. Due to the combinatorial nature of the hidden state representation, this exact algorithm is intractable. As in other intractable systems, approximate inference can be carried out using Gibbs sampling or variational methods. Within the variational framework, we present a structured approximation in which the the state variables are decoupled, yielding a tractable algorithm for learning the parameters of the model. Empirical comparisons suggest that these approximations are efficient and provide accurate alternatives to the exact methods. Finally, we use the structured approximation to model Bach's chorales and show that factorial HMMs can capture statistical structure in this data set which an unconstrained HMM cannot.     

Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method for errors-in-variables and output error identification

For identifying errors-in-variables models, the time domain maximum likelihood (TML) method and the sample maximum likelihood (SML) method are two approaches. Both methods give optimal estimation accuracy but under different assumptions. In the TML method, an important assumption is that the noise-free input signal is modelled as a stationary process with rational spectrum. For SML, the noise-free input needs to be periodic. It is interesting to know which of these assumptions contain more information to boost the estimation performance. In this paper, the estimation accuracy of the two methods is analyzed statistically for both errors-in-variables (EIV) and output error models (OEM). Numerical comparisons between these two estimates are also done under different signal-to-noise ratios (SNRs). The results suggest that TML and SML have similar estimation accuracy at moderate or high SNR for EIV. For OEM identification, these two methods have the same accuracy at any SNR.     

Efficient Bayesian estimation of multivariate state space models

A Bayesian Markov chain Monte Carlo methodology is developed for the estimation of multivariate linear Gaussian state space models. In particular, an efficient simulation smoothing algorithm is proposed that makes use of the univariate representation of the state space model. Substantial gains over existing algorithms in computational efficiency are achieved using the new simulation smoother for the analysis of high dimensional multivariate time series.The methodology is used to analyse a multivariate time series dataset of the Normalised Difference Vegetation Index (NDVI), which is a proxy for the level of live vegetation, for a particular grazing property located in Queensland, Australia.     

Pairwise likelihood inference for ordinal categorical time series

Ordinal categorical time series may be analyzed as censored observations from a suitable latent stochastic process, which describes the underlying evolution of the system. This approach may be considered as an alternative to Markov chain models or to regression methods for categorical time series data. The problem of parameter estimation is solved through a simple pseudolikelihood, called pairwise likelihood. This inferential methodology is successfully applied to the class of autoregressive ordered probit models. Potential usefulness for inference and model selection within more general classes of models are also emphasized. Illustrations include simulation studies and two simple real data applications.     

Performance of state space and ARIMA models for consumer retail sales forecasting

Forecasting future sales is one of the most important issues that is beyond all strategic and planning decisions in effective operations of retail businesses. For profitable retail businesses, accurate demand forecasting is crucial in organizing and planning production, purchasing, transportation and labor force. Retail sales series belong to a special type of time series that typically contain trend and seasonal patterns, presenting challenges in developing effective forecasting models. This work compares the forecasting performance of state space models and ARIMA models. The forecasting performance is demonstrated through a case study of retail sales of five different categories of women footwear: Boots, Booties, Flats, Sandals and Shoes. On both methodologies the model with the minimum value of Akaike's Information Criteria for the in-sample period was selected from all admissible models for further evaluation in the out-of-sample. Both one-step and multiple-step forecasts were produced. The results show that when an automatic algorithm the overall out-of-sample forecasting performance of state space and ARIMA models evaluated via RMSE, MAE and MAPE is quite similar on both one-step and multi-step forecasts. We also conclude that state space and ARIMA produce coverage probabilities that are close to the nominal rates for both one-step and multi-step forecasts.     

Hybrid visual servoing: A combination of nonlinear control and linear vision

In this paper, the visual servoing problem is addressed by coupling nonlinear control theory with a convenient representation of the visual information used by the robot. The visual representation, which is based on a linear camera model, is extremely compact to comply with active vision requirements. The devised control law is proven to ensure global asymptotic stability in the Lyapunov sense, assuming exact model and state measurements. It is also shown that, in the presence of bounded uncertainties, the closed-loop behavior is characterized by a global attractor. The well known pose ambiguity arising from the use of linear camera models is solved at the control level by choosing a hybrid visual state vector including both image space (2D) information and 3D object parameters. A method is expounded for on-line visual state estimation that avoids camera calibration. Simulation and real-time experiments validate the theoretical framework in terms of both system convergence and control robustness.     

Bayesian estimation of motion vector fields

A stochastic approach to the estimation of 2D motion vector fields from time-varying images is presented. The formulation involves the specification of a deterministic structural model along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vector-binary Markov random fields. Two estimation criteria are studied. In the maximum a posteriori probability (MAP) estimation, the a posteriori probability of motion given data is maximized, whereas in the minimum expected cost (MEC) estimation, the expectation of a certain cost function is minimized. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs sampler. Two versions are developed: one for a discrete state space and the other for a continuous state space. The MAP estimation is incorporated into a hierarchical environment to deal efficiently with large displacements     

Parameter estimation in multivariate stochastic difference equations

We will review the principal methods of estimation of parameters in multivariate autoregressive moving average equations which have additional observable input terms in them and present some new methods of estimation as well. We begin with the conditions for the estimability of the parameters. In addition to the usual method of system representation, the canonical form I, we will present two new representations of the system equation, the so-called canonical forms II and III which are convenient for parameter estimation. We will mention, in some detail, the various methods of estimation like the various least-squares methods, the maximum likelihood methods, etc., and discuss them regarding their relative accuracy of the estimate and the corresponding computational complexity. We will introduce a new class of estimates, the so-called limited information estimates which utilizes the canonical forms II and III. The accuracy of these estimates is close to that of maximum likelihood, but their computation time is only a fraction of the computation time for the usual maximum likelihood estimates. We will present a few numerical examples to illustrate the various methods.     

Analytical solution of the linear fractional system of commensurate order

A useful representation of fractional order systems is the state space representation. For the linear fractional systems of commensurate order, the state space representation is defined as for regular integer state space representation with the state vector differentiated to a real order. This paper presents a solution of the linear fractional order systems of commensurate order in the state space. The solution is obtained using a technique based on functions of square matrices and the Cayley-Hamilton theorem. The technique developed for linear systems of integer order is extended to derive analytical solutions of linear fractional systems of commensurate order. The basic ideas and the derived formulations of the technique are presented. Both, homogeneous and inhomogeneous cases with usual input functions are solved. The solution is calculated in the form of a linear combination of suitable fundamental functions. The presented results are illustrated by analyzing some examples to demonstrate the effectiveness of the presented analytical approach.     

含时延和丢失数据不确定系统的FIR滤波器

针对状态空间模型中存在服从伯努利分布的时延和随机观测丢失的情况,基于极大似然法则,分别设计有限脉冲响应(finite impulse response, FIR)滤波器的慢速率批处理形式和快速率迭代形式.首先,将时延和数据丢失情况下的模型表述为服从伯努利分布的概率线性函数;然后,通过极大似然处理从而得到所提出极大似然FIR算法;最后,将在相同条件下的极大似然FIR估计、改进型卡尔曼滤波以及无偏FIR估计3种滤波方法进行对比,从估计误差、均方根误差和不确定性影响等角度进行比较分析.实验部分通过3-DOF直升机模型仿真,可发现所提出极大似然FIR估计方法在处理时延和数据丢失问题时更加有效,鲁棒性更高.     

Algorithms for deterministic balanced subspace identification

New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses directly from data. The proposed algorithms are more efficient than the existing alternatives that compute the whole Hankel matrix of Markov parameters. It is shown that the computations can be performed on Hankel matrices of the input-output data of various dimensions. By choosing wider matrices, we need persistency of excitation of smaller order. Moreover, this leads to computational savings and improved statistical accuracy when the data is noisy. Using a finite amount of input-output data, the existing algorithms compute finite time balanced representation and the identified models have a lower bound on the distance to an exact balanced representation. The proposed algorithm can approximate arbitrarily closely an exact balanced representation. Moreover, the finite time balancing parameter can be selected automatically by monitoring the decay of the impulse response. We show what is the optimal in terms of minimal identifiability condition partition of the data into “past” and “future”.     

A new look at the statistical model identification

The history of the development of statistical hypothesis testing in time series analysis is reviewed briefly and it is pointed out that the hypothesis testing procedure is not adequately defined as the procedure for statistical model identification. The classical maximum likelihood estimation procedure is reviewed and a new estimate minimum information theoretical criterion (AIC) estimate (MAICE) which is designed for the purpose of statistical identification is introduced. When there are several competing models the MAICE is defined by the model and the maximum likelihood estimates of the parameters which give the minimum of AIC defined by AIC = (-2)log-(maximum likelihood) + 2(number of independently adjusted parameters within the model). MAICE provides a versatile procedure for statistical model identification which is free from the ambiguities inherent in the application of conventional hypothesis testing procedure. The practical utility of MAICE in time series analysis is demonstrated with some numerical examples.     

Analytical Derivatives for Markov Switching Models

This paper presents analytical gradients for a broad class of regime-switching models with Markovian state-transition probabilities. Such models are usually estimated by maximum likelihood methods, which require the derivatives of the likelihood function with respect to the parameter vector. These gradients are usually calculated by means of numerical techniques. The paper shows that analytical gradients considerably speed up maximum-likelihood estimation.