Clustering Time Series with Clipped Data

Clustering time series is a problem that has applications in a wide variety of fields, and has recently attracted a large amount of research. Time series data are often large and may contain outliers. We show that the simple procedure of clipping the time series (discretising to above or below the median) reduces memory requirements and significantly speeds up clustering without decreasing clustering accuracy. We also demonstrate that clipping increases clustering accuracy when there are outliers in the data, thus serving as a means of outlier detection and a method of identifying model misspecification. We consider simulated data from polynomial, autoregressive moving average and hidden Markov models and show that the estimated parameters of the clipped data used in clustering tend, asymptotically, to those of the unclipped data. We also demonstrate experimentally that, if the series are long enough, the accuracy on clipped data is not significantly less than the accuracy on unclipped data, and if the series contain outliers then clipping results in significantly better clusterings. We then illustrate how using clipped series can be of practical benefit in detecting model misspecification and outliers on two real world data sets: an electricity generation bid data set and an ECG data set.     

非线性时间序列建模的混合自回归滑动平均模型

提出了一类用于非线性时间序列建模的混合自回归滑动平均模型(MARMA).该模型是由K个平稳或非平稳的ARMA分量经过混合得到的.讨论了MARMA模型的平稳性条件和自相关函数.给出了MARMA模型参数估计的期望极大化(expectation maximization)算法.运用贝叶斯信息准则(Bayes information criterion)来选择该模型.MARMA模型分布形式富于变化的特征使得它能够对具有多峰分布以及条件异方差的序列进行建模.通过两个实例验证了该模型,并和其他模型进行比较,结果表明MARMA模型能够更好地描述这些数据的特征.     

Autoregressive forecast of monthly total ozone concentration: A neurocomputing approach

The present study endeavors to generate autoregressive neural network (AR-NN) models to forecast the monthly total ozone concentration over Kolkata (22°34′, 88°22′), India. The issues associated with the applicability of neural network to geophysical processes are discussed. The autocorrelation structure of the monthly total ozone time series is investigated, and stationarity of the time series is established through the periodogram. From various autoregressive moving average (ARMA) and autoregressive models fit to the time series, the autoregressive model of order 10 is identified as the best. Subsequently, 10 autoregressive neural network (AR-NN) models are generated; the 10th order autoregressive neural network model with extensive input variable selection performs the best among all the competitive models in forecasting the monthly total ozone concentration over the study zone.     

Autoregressive spectral analysis when observations are missing

A new missing data algorithm ARFIL gives good results in spectral estimation. The log likelihood of a multivariate Gaussian random variable can always be written as a sum of conditional log likelihoods. For a complete set of autoregressive AR(p) data the best predictor in the likelihood requires only p previous observations. If observations are missing, the best AR predictor in the likelihood will in general include all previous observations. Using only those observations that fall within a finite time interval will approximate this likelihood. The resulting non-linear estimation algorithm requires no user provided starting values. In various simulations, the spectral accuracy of robust maximum likelihood methods was much better than the accuracy of other spectral estimates for randomly missing data.     

Exact maximum likelihood estimation for non-stationary periodic time series models

Time series models with parameter values that depend on the seasonal index are commonly referred to as periodic models. Periodic formulations for two classes of time series models are considered: seasonal autoregressive integrated moving average and unobserved components models. Convenient state space representations of the periodic models are proposed to facilitate model identification, specification and exact maximum likelihood estimation of the periodic parameters. These formulations do not require a priori (seasonal) differencing of the time series. The time-varying state space representation is an attractive alternative to the time-invariant vector representation of periodic models which typically leads to a high dimensional state vector in monthly periodic time series models. A key development is our method for computing the variance-covariance matrix of the initial set of observations which is required for exact maximum likelihood estimation. The two classes of periodic models are illustrated for a monthly postwar US unemployment time series.     

Input-output modeling of nonlinear systems with time-varying linearmodels

Time varying ARMA (autoregressive moving average) and ARMAX (autoregressive moving average with exogenous inputs) models are proposed for input-output modeling of nonlinear deterministic and stochastic systems. The coefficients of these models are estimated by a random walk Kalman filter (RWKF). This method requires no prior assumption on the nature of the model coefficients, and is suitable for real-time implementation since no off-line training is needed. A simulation example illustrates the method. Goodness of performance is judged by the quality of the residuals, histograms, autocorrelation functions and the Kolmogorov-Smirnoff test     

Hybridization of autoregressive integrated moving average (ARIMA) with probabilistic neural networks (PNNs)

Autoregressive integrated moving average (ARIMA) models are one of the most important time series models applied in financial market forecasting over the past three decades. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to yield results that are more accurate. In this paper, a new hybrid model of the autoregressive integrated moving average (ARIMA) and probabilistic neural network (PNN), is proposed in order to yield more accurate results than traditional ARIMA models. In proposed model, the estimated values of the ARIMA model are modified based on the distinguished trend of the ARIMA residuals and optimum step length, which are respectively obtained from a probabilistic neural network and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than ARIMA model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Combining seasonal ARIMA models with computational intelligence techniques for time series forecasting

Seasonal autoregressive integrated moving average (SARIMA) models form one of the most popular and widely used seasonal time series models over the past three decades. However, in several researches it has been argued that they have two basic limitations that detract from their popularity for seasonal time series forecasting tasks. SARIMA models assume that future values of a time series have a linear relationship with current and past values as well as with white noise; therefore, approximations by SARIMA models may not be adequate for complex nonlinear problems. In addition, SARIMA models require a large amount of historical data to produce desired results. However, in real situations, due to uncertainty resulting from the integral environment and rapid development of new technology, future situations must be forecasted using small data sets over a short span of time. Using hybrid models or combining several models has become a common practice to overcome the limitations of single models and improve forecasting accuracy. In this paper, a new hybrid model, which combines the seasonal autoregressive integrated moving average (SARIMA) and computational intelligence techniques such as artificial neural networks and fuzzy models for seasonal time series forecasting is proposed. In the proposed model, these two techniques are applied to simultaneously overcome the linear and data limitations of SARIMA models and yield more accurate results. Empirical results of forecasting two well-known seasonal time series data sets indicate that the proposed model exhibits effectively improved forecasting accuracy, so that it can be used as an appropriate seasonal time series model.     

智慧农业中时序数据组合预测模型

智慧农业是实现农业精准化的技术解决方案,智慧农业系统可以实时监测植物生长的各类环境参数,并可以应用相应的预测模型来模拟农作物生长环境的变化趋势,为科学决策提供依据。近年来有很多学者提出了时间序列的预测模型算法,在预测稳定性方面取得了不错的效果。为了进一步提升时间序列的预测精度,提出一种基于差分整合移动平均自回归模型和小波神经网络的组合预测模型。该组合模型结合2个单项模型优点,用差分整合移动平均自回归模型来拟合序列的线性部分,用小波神经网络来校正其残差,使其拟合曲线更接近于实际值,采用温室内的历史温度数据来验证该组合模型的精确度,最后将组合模型与传统预测模型的预测结果进行对比。结果表明,该组合模型用于温室温度预测的精确度更高,拟合效果更好,相比于传统模型预测算法计算效能提高了20%左右。     

Input–output data filtering based recursive least squares identification for CARARMA systems

This paper uses an estimated noise transfer function to filter the input–output data and presents filtering based recursive least squares algorithms (F-RLS) for controlled autoregressive autoregressive moving average (CARARMA) systems. Through the data filtering, we obtain two identification models, one including the parameters of the system model, and the other including the parameters of the noise model. Thus, the recursive least squares method can be used to estimate the parameters of these two identification models, respectively, by replacing the unmeasurable variables in the information vectors with their estimates. The proposed F-RLS algorithm has a high computational efficiency because the dimensions of its covariance matrices become small and can generate more accurate parameter estimation compared with other existing algorithms.     

非线性时间序列预报的隐多分辨ARMA模型

研究一类用于非线性时间序列预报的隐多分辨自回归滑动平均(ARMA)模型,该模型以ARMA模型为初始细水平模型(即隐多分辨模型的基本块)．证明了模型的建模精度由水平问的方差决定．研究了新模型的自相关函数结构,给出了参数估计的Bayes方法和Metropolis-Hasting算法．进一步提出了一种可以直接用于不同基本块的隐多分辨模型的非线性时间序列预报方法,证明了其比其他的线性预报方法和隐多分辨模型预报方法降低了预报误差．最后通过数值模拟和实例验证了模型和预报方法,并和其他模型进行比较,结果表明新提出模型和预报方法能够更好地描述数据的特征,提高预报的精度．     

Parameter estimation in multivariate stochastic difference equations

We will review the principal methods of estimation of parameters in multivariate autoregressive moving average equations which have additional observable input terms in them and present some new methods of estimation as well. We begin with the conditions for the estimability of the parameters. In addition to the usual method of system representation, the canonical form I, we will present two new representations of the system equation, the so-called canonical forms II and III which are convenient for parameter estimation. We will mention, in some detail, the various methods of estimation like the various least-squares methods, the maximum likelihood methods, etc., and discuss them regarding their relative accuracy of the estimate and the corresponding computational complexity. We will introduce a new class of estimates, the so-called limited information estimates which utilizes the canonical forms II and III. The accuracy of these estimates is close to that of maximum likelihood, but their computation time is only a fraction of the computation time for the usual maximum likelihood estimates. We will present a few numerical examples to illustrate the various methods.     

A moving-average filter based hybrid ARIMA–ANN model for forecasting time series data

A suitable combination of linear and nonlinear models provides a more accurate prediction model than an individual linear or nonlinear model for forecasting time series data originating from various applications. The linear autoregressive integrated moving average (ARIMA) and nonlinear artificial neural network (ANN) models are explored in this paper to devise a new hybrid ARIMA–ANN model for the prediction of time series data. Many of the hybrid ARIMA–ANN models which exist in the literature apply an ARIMA model to given time series data, consider the error between the original and the ARIMA-predicted data as a nonlinear component, and model it using an ANN in different ways. Though these models give predictions with higher accuracy than the individual models, there is scope for further improvement in the accuracy if the nature of the given time series is taken into account before applying the models. In the work described in this paper, the nature of volatility was explored using a moving-average filter, and then an ARIMA and an ANN model were suitably applied. Using a simulated data set and experimental data sets such as sunspot data, electricity price data, and stock market data, the proposed hybrid ARIMA–ANN model was applied along with individual ARIMA and ANN models and some existing hybrid ARIMA–ANN models. The results obtained from all of these data sets show that for both one-step-ahead and multistep-ahead forecasts, the proposed hybrid model has higher prediction accuracy.     

Sensitivity of vegetation phenology detection to the temporal resolution of satellite data

Vegetation phenology derived from satellite data has increasingly received attention for applications in environmental monitoring and modelling. The accuracy of phenological estimates, however, is unknown at the regional and global level because field validation data are insufficient. To assess the accuracy of satellite‐derived phenology, this study investigates the sensitivity of phenology detection to both the temporal resolution of sampling and the number of consecutive missing values (usually representing cloud cover) in the time series of satellite data. To do this, time series of daily vegetation index data for various ecosystems are modelled and simulated using data from Moderate‐Resolution Imaging Spectroradiometer (MODIS) data. The annual temporal data are then fitted using piecewise logistic functions, which are employed to calculate curvature change rate for detecting phenological transition dates. The results show that vegetation phenology can be estimated with a high precision from time series with temporal resolutions of 6–16 days even if daily data contains some uncertainties. If the temporal resolution is no coarser than 16 days for time series sampled using an average composite, the absolute errors are less than 3 days. On the other hand, the phase shift of temporal sampling is shown to have limited impacts on phenology detection. However, the accuracy of phenology detection may be reduced greatly if missing values in the time series of 16‐day MODIS data occur around the onsets of phenological transition dates. Even so, the probability that the absolute error in phenological estimates is greater than 5 days is less than 4% when only one period is missing in the time series of 16‐day data during vegetation growing seasons; this probability increases to 20% if there are two consecutive missing values.     

Forecasting financial time series using a methodology based on autoregressive integrated moving average and Taylor expansion

Financial time series prediction is regarded as one of the most challenging job because of its inherent complexity, and the hybrid forecasting model incorporating autoregressive integrated moving average and support vector machine (SVM) has been implemented widely to deal with the both linear and nonlinear patterns in time series data. However, the SVM model does not take into consideration the time correlation knowledge between different data points in time series, which impacts the learning efficiency of the SVM in real application. To overcome this restriction, this paper proposes the Taylor Expansion Forecasting model as an alternative to the SVM and develops a novel hybrid methodology via combining autoregressive integrated moving average and Taylor Expansion Forecasting to exploit the comprehensive forecasting capacity to the financial time series data with noise. Both theoretical proof and empirical results obtained on several commodity future prices demonstrate that the proposed hybrid model improves greatly the forecasting accuracy.     

Time Series Prediction With Genetic-Algorithm Designed Neural Networks: An Empirical Comparison With Modern Statistical Models

Neural networks whose architecture is determined by genetic algorithms outperform autoregressive integrated moving average forecasting models in six different time series examples. Refinements to the autoregressive integrated moving average model improve forecasting performance over standard ordinary least squares estimation by 8% to 13%. In contrast, neural networks achieve dramatic improvements of 10% to 40%. Additionally, neural networks give evidence of detecting patterns in data which remain hidden to the autoregression and moving average models. The consequent forecasting potential of neural networks makes them a very promising addition to the variety of techniques and methodologies used to anticipate future movements in time series.
     

ECM-based maximum likelihood inference for multivariate linear mixed models with autoregressive errors

For the analysis of longitudinal data with multiple characteristics, we are devoted to providing additional tools for multivariate linear mixed models in which the errors are assumed to be serially correlated according to an autoregressive process. We present a computationally flexible ECM procedure for obtaining the maximum likelihood estimates of model parameters. A score test statistic for testing the existence of autocorrelation among within-subject errors of each characteristic is derived. The techniques for the estimation of random effects and the prediction of further responses given past repeated measures are also investigated. The methodology is illustrated through an application to a set of AIDS data and two small simulation studies.     

A hybrid multi-order fuzzy time series for forecasting stock markets

This paper proposes a hybrid model based on multi-order fuzzy time series, which employs rough sets theory to mine fuzzy logical relationship from time series and an adaptive expectation model to adjust forecasting results, to improve forecasting accuracy. Two empirical stock markets (TAIEX and NASDAQ) are used as empirical databases to verify the forecasting performance of the proposed model, and two other methodologies, proposed earlier by Chen and Yu, are employed as comparison models. Besides, to compare with conventional statistic method, the partial autocorrelation function and autoregressive models are utilized to estimate the time lags periods within the databases. Based on comparison results, the proposed model can effectively improve the forecasting performance and outperforms the listing models. From the empirical study, the conventional statistic method and the proposed model both have revealed that the estimated time lags for the two empirical databases are one lagged period.     

The Kalman filter as an adaptive forecasting procedure for use with Box-Jenkins arima models

The Box-Jenkins (BJ) methodology of time series analysis is currently one of the most accurate of the historical approaches to forecasting. It involves the formation of an autoregressive integrated moving average model of the time series. However, there are some important limitations to the procedure: (1) It requires an extensive amount of past observations in order to develop an acceptable model. (2) The model identification process requires a great deal of time and expertise. (3) The model selected is a constant model; there is no convenient way to modify the coefficients with new observations. The research reported here uses the Kalman filter (KF) algorithm to develop a method using an autoregressive moving average model, called “KARMA,” to overcome the three problems mentioned above. The KF estimates the states for dynamic systems in state-variable formulation. The conclusion reached is that the general KARMA method developed in this research can be used to resolve the three main problems associated with the BJ methodology. First, the KARMA model can give good forecasts with fewer data than are required with the BJ. Second, the general model eliminates the need for the model identification process; the only requirement to obtain the forecasts is the time series. Third, the KARMA method is adaptive in the parameters and model.     

Recursive least squares parameter identification algorithms for systems with colored noise using the filtering technique and the auxilary model

This paper focuses on the parameter estimation problems of output error autoregressive systems and output error autoregressive moving average systems (i.e., the Box–Jenkins systems). Two recursive least squares parameter estimation algorithms are proposed by using the data filtering technique and the auxiliary model identification idea. The key is to use a linear filter to filter the input–output data. The proposed algorithms can identify the parameters of the system models and the noise models interactively and can generate more accurate parameter estimates than the auxiliary model based recursive least squares algorithms. Two examples are given to test the proposed algorithms.