Multivariate exponential smoothing: A Bayesian forecast approach based on simulation

This paper deals with the prediction of time series with correlated errors at each time point using a Bayesian forecast approach based on the multivariate Holt–Winters model. Assuming that each of the univariate time series comes from the univariate Holt–Winters model, all of them sharing a common structure, the multivariate Holt–Winters model can be formulated as a traditional multivariate regression model. This formulation facilitates obtaining the posterior distribution of the model parameters, which is not analytically tractable: simulation is needed. An acceptance sampling procedure is used in order to obtain a sample from this posterior distribution. Using Monte Carlo integration the predictive distribution is then approached. The forecasting performance of this procedure is illustrated using the hotel occupancy time series data from three provinces in Spain.     

多元混沌时间序列的相关状态机预测模型研究

针对多元混沌时间序列预测存在的过拟合问题及高维输入变量冗余问题,提出一种新型的多变量稀疏化预测模型——多元相关状态机.该模型采用主成分分析方法对相空间重构后的高维输入变量进行低维表示,将动态储备池作为相关向量机的核函数,充分映射多元混沌时间序列的动力学特性,使得模型具有丰富的动态机制和良好的稀疏性能,有效避免过拟合问题,提高预测精度.基于两组多元混沌时间序列的仿真实验验证了模型的有效性.     

Discriminant analysis of multivariate time series: Application to diagnosis based on ECG signals

In analysing ECG data, the main aim is to differentiate between the signal patterns of healthy subjects and those of individuals with specific heart conditions. We propose an approach for classifying multivariate ECG signals based on discriminant and wavelet analyses. For this purpose we use multiple-scale wavelet variances and wavelet correlations to distinguish between the patterns of multivariate ECG signals based on the variability of the individual components of each ECG signal and on the relationships between every pair of these components. Using the results of other ECG classification studies in the literature as references, we demonstrate that our approach applied to 12-lead ECG signals from a particular database compares favourably. We also demonstrate with real and synthetic ECG data that our approach to classifying multivariate time series out-performs other well-known approaches for classifying multivariate time series.     

Exploiting multi-channels deep convolutional neural networks for multivariate time series classification

Time series classification is related to many different domains, such as health informatics, finance, and bioinformatics. Due to its broad applications, researchers have developed many algorithms for this kind of tasks, e.g., multivariate time series classification. Among the classification algorithms, k-nearest neighbor (k-NN) classification (particularly 1-NN) combined with dynamic time warping (DTW) achieves the state of the art performance. The deficiency is that when the data set grows large, the time consumption of 1-NN with DTWwill be very expensive. In contrast to 1-NN with DTW, it is more efficient but less effective for feature-based classification methods since their performance usually depends on the quality of hand-crafted features. In this paper, we aim to improve the performance of traditional feature-based approaches through the feature learning techniques. Specifically, we propose a novel deep learning framework, multi-channels deep convolutional neural networks (MC-DCNN), for multivariate time series classification. This model first learns features from individual univariate time series in each channel, and combines information from all channels as feature representation at the final layer. Then, the learnt features are applied into a multilayer perceptron (MLP) for classification. Finally, the extensive experiments on real-world data sets show that our model is not only more efficient than the state of the art but also competitive in accuracy. This study implies that feature learning is worth to be investigated for the problem of time series classification.     

基于储备池主成分分析的多元时间序列预测研究

提出一种基于回声状态网络储备池的非线性ＰＣＡ 方法,并将其应用于多元时间序列的预测中．由于多维输入变量间的相关性会影响建模效果,通过储备池将输入在原空间的非线性特征转化成高维空间的线性特征．在其中运用线性ＰＣＡ 技术寻找输入在储备池空间的最大方差方向,提取有效的多元变量综合信息．经储备池主成分分析处理后的输入与预测点呈动态线性映射,可使用线性方法建模．仿真结果表明了该方法的有效性．     

Blast Furnace System Modeling by Multivariate Phase Space Reconstruction and Neural Networks

Time series data collected from a medium‐size blast furnace (BF) is analyzed using the phase space reconstruction. To achieve better reconstruction, multivariate correlation analysis is first applied to screen out correlated variables, which shows that three important variables, i.e., silicon content in hot metal ([Si]), permeability index (FF), and coal injection (PM), are most appropriate for multivariate reconstruction. The time delay and embedding dimension are determined via the autocorrelation function and false nearest neighbor method. With the reconstructed time series, the neural networks model is applied to construct the predictive model for silicon content in hot metal. The simulation shows that the models based on multivariate reconstruction give better predictions than those obtained by univariate reconstruction. Moreover, it reveals that multivariate reconstruction can greatly mitigate the drawbacks caused by insufficiency of data.     

复杂高炉炼铁过程的数据驱动建模及预测算法

高炉炼铁过程的控制意味着控制高炉铁水温度及成份在指定的范围. 本文以高炉炉内热状态的重要指示剂---高炉铁水硅含量为研究对象, 针对机理建模难以准确预测、控制高炉铁水硅含量的发展变化, 利用数据驱动建模的思想, 建立了基于多元时间序列的高炉铁水硅含量数据驱动预测模型. 实例分析表明, 建立的数据驱动预测模型能够很好地预测高炉铁水硅含量, 连续预测167炉高炉铁水硅含量, 命中率高达83.23%, 预测均方根误差为0.07260. 这些指标均优于基于单一硅时间序列所建立的数据驱动模型, 对实际生产具有很好的指导作用.     

Time series representation and similarity based on local autopatterns

Time series data mining has received much greater interest along with the increase in temporal data sets from different domains such as medicine, finance, multimedia, etc. Representations are important to reduce dimensionality and generate useful similarity measures. High-level representations such as Fourier transforms, wavelets, piecewise polynomial models, etc., were considered previously. Recently, autoregressive kernels were introduced to reflect the similarity of the time series. We introduce a novel approach to model the dependency structure in time series that generalizes the concept of autoregression to local autopatterns. Our approach generates a pattern-based representation along with a similarity measure called learned pattern similarity (LPS). A tree-based ensemble-learning strategy that is fast and insensitive to parameter settings is the basis for the approach. Then, a robust similarity measure based on the learned patterns is presented. This unsupervised approach to represent and measure the similarity between time series generally applies to a number of data mining tasks (e.g., clustering, anomaly detection, classification). Furthermore, an embedded learning of the representation avoids pre-defined features and an extraction step which is common in some feature-based approaches. The method generalizes in a straightforward manner to multivariate time series. The effectiveness of LPS is evaluated on time series classification problems from various domains. We compare LPS to eleven well-known similarity measures. Our experimental results show that LPS provides fast and competitive results on benchmark datasets from several domains. Furthermore, LPS provides a research direction and template approach that breaks from the linear dependency models to potentially foster other promising nonlinear approaches.     

Subspace based feature selection for pattern recognition

Feature selection is an essential topic in the field of pattern recognition. The feature selection strategy has a direct influence on the accuracy and processing time of pattern recognition applications. Features can be evaluated with either univariate approaches, which examine features individually, or multivariate approaches, which consider possible feature correlations and examine features as a group. Although univariate approaches do not take the correlation among features into consideration, they can provide the individual discriminatory power of the features, and they are also much faster than multivariate approaches. Since it is crucial to know which features are more or less informative in certain pattern recognition applications, univariate approaches are more useful in these cases. This paper therefore proposes subspace based separability measures to determine the individual discriminatory power of the features. These measures are then employed to sort and select features in a multi-class manner. The feature selection performances of the proposed measures are evaluated and compared with the univariate forms of classic separability measures (Divergence, Bhattacharyya, Transformed Divergence, and Jeffries-Matusita) on several datasets. The experimental results clearly indicate that the new measures yield comparable or even better performance than the classic ones in terms of classification accuracy and dimension reduction rate.     

Combining univariate approaches for ensemble change detection in multivariate data

Detecting change in multivariate data is a challenging problem, especially when class labels are not available. There is a large body of research on univariate change detection, notably in control charts developed originally for engineering applications. We evaluate univariate change detection approaches —including those in the MOA framework — built into ensembles where each member observes a feature in the input space of an unsupervised change detection problem. We present a comparison between the ensemble combinations and three established ‘pure’ multivariate approaches over 96 data sets, and a case study on the KDD Cup 1999 network intrusion detection dataset. We found that ensemble combination of univariate methods consistently outperformed multivariate methods on the four experimental metrics.     

A novel classifier for multivariate instance using graph class signatures

Applications like identifying different customers from their unique buying behaviours, determining ratingsof a product given by users based on different sets of features, etc. require classification using class-specific subsets of features. Most of the existing state-of-the-art classifiers for multivariate data use complete feature set for classification regardless of the different class labels. Decision tree classifier can produce class-wise subsets of features. However, none of these classifiers model the relationship between features which may enhance classification accuracy. We call the class-specific subsets of features and the features’ interrelationships as class signatures. In this work, we propose to map the original input space of multivariate data to the feature space characterized by connected graphs as graphs can easily model entities, their attributes, and relationships among attributes. Mostly, entities are modeled using graphs, where graphs occur naturally, for example, chemical compounds. However, graphs do not occur naturally in multivariate data. Thus, extracting class signatures from multivariate data is a challenging task. We propose some feature selection heuristics to obtain class-specific prominent subgraph signatures. We also propose two variants of class signatures based classifier namely: 1) maximum matching signature (gMM), and 2) score and size of matched signatures (gSM). The effectiveness of the proposed approach on real-world and synthetic datasets has been studied and compared with other established classifiers. Experimental results confirm the ascendancy of the proposed class signatures based classifier on most of the datasets.     

Generalized random shapelet forests

Shapelets are discriminative subsequences of time series, usually embedded in shapelet-based decision trees. The enumeration of time series shapelets is, however, computationally costly, which in addition to the inherent difficulty of the decision tree learning algorithm to effectively handle high-dimensional data, severely limits the applicability of shapelet-based decision tree learning from large (multivariate) time series databases. This paper introduces a novel tree-based ensemble method for univariate and multivariate time series classification using shapelets, called the generalized random shapelet forest algorithm. The algorithm generates a set of shapelet-based decision trees, where both the choice of instances used for building a tree and the choice of shapelets are randomized. For univariate time series, it is demonstrated through an extensive empirical investigation that the proposed algorithm yields predictive performance comparable to the current state-of-the-art and significantly outperforms several alternative algorithms, while being at least an order of magnitude faster. Similarly for multivariate time series, it is shown that the algorithm is significantly less computationally costly and more accurate than the current state-of-the-art.     

基于多尺度卷积和注意力机制的LSTM时间序列分类

时间序列的多尺度特征包含丰富的类别信息,且这些信息对分类具有不同的重要程度,然而现有的单变量时间序列分类模型通常以固定大小的卷积核提取序列特征,导致不能有效地获取并聚焦重要的多尺度特征。针对上述问题,提出一种基于多尺度卷积和注意力机制（MCA）的长短时记忆（LSTM）模型（MCA-LSTM）,它能够关注并融合重要的多尺度特征,从而实现更准确的分类。其中,LSTM使用记忆细胞和门机制控制序列信息的传递,并充分提取时间序列的相关性信息;多尺度卷积模块（MCM）使用具有不同卷积核的卷积神经网络（CNN）提取序列的多尺度特征;注意力模块（AM）融合通道信息获取特征的重要性并分配注意力权重,从而使网络关注重要的时间序列特征。在UCR档案的65个单变量时间序列数据集上的实验结果表明,对比当前最先进的基于深度学习的时间序列分类模型：USRL-FordA（Unsupervised Scalable Representation Learning-FordA）、USRL-Combined （1-NN） （Unsupervised Scalable Representation Learning-Combined （1-Nearest Neighbor）） OS-CNN（Omni-Scale Convolutional Neural Network）、Inception-Time和RTFN（Robust Temporal Feature Network for time series classification）,MCA-LSTM在平均错误率（ME）上分别降低了7.48、9.92、2.43、2.09和0.82个百分点,并取得了最高的算术平均排名（AMR）和几何平均排名（GMR）,分别为2.14和3.23,这些充分体现了MCA-LSTM模型在单变量时间序列分类中的有效性。     

Change point detection in time series data with random forests

A large class of monitoring problems can be cast as the detection of a change in the parameters of a static or dynamic system, based on the effects of these changes on one or more observed variables. In this paper, the use of random forest models to detect change points in dynamic systems is considered. The approach is based on the embedding of multivariate time series data associated with normal process conditions, followed by the extraction of features from the resulting lagged trajectory matrix. The features are extracted by recasting the data into a binary classification problem, which can be solved with a random forest model. A proximity matrix can be calculated from the model and from this matrix features can be extracted that represent the trajectory of the system in phase space. The results of the study suggest that the random forest approach may afford distinct advantages over a previously proposed linear equivalent, particularly when complex nonlinear systems need to be monitored.     

Frameworks for multivariate m-mediods based modeling and classification in Euclidean and general feature spaces

This paper presents an extension of m-mediods based modeling technique to cater for multimodal distributions of sample within a pattern. The classification of new samples and anomaly detection is performed using a novel classification algorithm which can handle patterns with underlying multivariate probability distributions. We have proposed two frameworks, namely MMC-ES and MMC-GFS, to enable our proposed multivarite m-mediods based modeling and classification approach workable for any feature space with a computable distance metric. MMC-ES framework is specialized for finite dimensional features in Euclidean space whereas MMC-GFS works on any feature space with a computable distance metric. Experimental results using simulated and complex real life dataset show that multivariate m-mediods based frameworks are effective and give superior performance than competitive modeling and classification techniques especially when the patterns exhibit multivariate probability density functions.     

Prediction of multivariate chaotic time series with local polynomial fitting

To improve the prediction accuracy of complex multivariate chaotic time series, a novel scheme formed on the basis of multivariate local polynomial fitting with the optimal kernel function is proposed. According to Takens Theorem, a chaotic time series is reconstructed into vector data, multivariate local polynomial regression is used to fit the predicted complex chaotic system, then the regression model parameters with the least squares method based on embedding dimensions are estimated,and the prediction value is calculated. To evaluate the results, the proposed multivariate chaotic time series predictor based on multivariate local polynomial model is compared with a univariate predictor with the same numerical data. The simulation results obtained by the Lorenz system show that the prediction mean squares error of the multivariate predictor is much smaller than the univariate one, and is much better than the existing three methods. Even if the last half of the training data are used in the multivariate predictor, the prediction mean squares error is smaller than that of the univariate predictor.     

Optimal error predictors for economic models

A class of predictors based on concepts and results of the general theory of optimal algorithms is proposed for time series analysis as a possible alternative approach to classical statistical techniques. In econometric contexts, it frequently happens that time series are relatively short (less than one or two hundred data); in these cases, whenever statistical methods do not provide reliable forecast results, the optimal error predictors can be used effectively. Optimal error predictors are derived both for external and internal univariate and multivariate time series models. In particular, optimal algorithm predictors are presented for two classical economic models: the multiplier accelerator model and the dynamic multivariate Leontief model.     

Decision trees for hierarchical multi-label classification

Hierarchical multi-label classification (HMC) is a variant of classification where instances may belong to multiple classes at the same time and these classes are organized in a hierarchy. This article presents several approaches to the induction of decision trees for HMC, as well as an empirical study of their use in functional genomics. We compare learning a single HMC tree (which makes predictions for all classes together) to two approaches that learn a set of regular classification trees (one for each class). The first approach defines an independent single-label classification task for each class (SC). Obviously, the hierarchy introduces dependencies between the classes. While they are ignored by the first approach, they are exploited by the second approach, named hierarchical single-label classification (HSC). Depending on the application at hand, the hierarchy of classes can be such that each class has at most one parent (tree structure) or such that classes may have multiple parents (DAG structure). The latter case has not been considered before and we show how the HMC and HSC approaches can be modified to support this setting. We compare the three approaches on 24 yeast data sets using as classification schemes MIPS’s FunCat (tree structure) and the Gene Ontology (DAG structure). We show that HMC trees outperform HSC and SC trees along three dimensions: predictive accuracy, model size, and induction time. We conclude that HMC trees should definitely be considered in HMC tasks where interpretable models are desired.     

Univariate modeling and forecasting of monthly energy demand time series using abductive and neural networks

Neural networks have been widely used for short-term, and to a lesser degree medium and long-term, demand forecasting. In the majority of cases for the latter two applications, multivariate modeling was adopted, where the demand time series is related to other weather, socio-economic and demographic time series. Disadvantages of this approach include the fact that influential exogenous factors are difficult to determine, and accurate data for them may not be readily available. This paper uses univariate modeling of the monthly demand time series based only on data for 6 years to forecast the demand for the seventh year. Both neural and abductive networks were used for modeling, and their performance was compared. A simple technique is described for removing the upward growth trend prior to modeling the demand time series to avoid problems associated with extrapolating beyond the data range used for training. Two modeling approaches were investigated and compared: iteratively using a single next-month forecaster, and employing 12 dedicated models to forecast the 12 individual months directly. Results indicate better performance by the first approach, with mean percentage error (MAPE) of the order of 3% for abductive networks. Performance is superior to naı¨ve forecasts based on persistence and seasonality, and is better than results quoted in the literature for several similar applications using multivariate abductive modeling, multiple regression, and univariate ARIMA analysis. Automatic selection of only the most relevant model inputs by the abductive learning algorithm provides better insight into the modeled process and allows constructing simpler neural network models with reduced data dimensionality and improved forecasting performance.