Time series classification with feature covariance matrices

In this work, a novel approach utilizing feature covariance matrices is proposed for time series classification. In order to adapt the feature covariance matrices into time series classification problem, a feature vector is defined for each point in a time series. The feature vector comprises local and global information such as value, derivative, rank, deviation from the mean, the time index of the point and cumulative sum up to the point. Extracted feature vectors for the time instances are concatenated to construct feature matrices for the overlapping subsequences. Covariances of the feature matrices are used to describe the subsequences. Our main purpose in this work is to introduce and evaluate the feature covariance representation for time series classification. Therefore, in classification stage, firstly, 1-NN classifier is utilized. After showing the effectiveness of the representation with 1-NN classifier, the experiments are repeated with SVM classifier. The other novelty in this work is that a novel distance measure is introduced for time series by feature covariance matrix representation. Conducted experiments on UCR time series datasets show that the proposed method mostly outperforms the well-known methods such as DTW, shapelet transform and other state-of-the-art techniques.     

Analytical evaluation of modal covariance matrices

Dynamic excitations in the form of stationary random processes with normal distribution are completely defined by their power spectral and cross spectral density functions. The stationary response of a linear structure to such excitations will also consist of random processes with normal distribution. In a modal formulation the statistical quantities of all output processes are obtained from modal covariance matrices. The elements of these matrices represent integrals which are usually evaluated numerically. In lightly damped structures, however, the integrand shows pronounced peaks. Thus small integration steps may be necessary for accurate results. In the applications the spectral density functions are conveniently described by discrete values and piecewise polynomial interpolation. The elements of the modal covariance matrices can then be evaluated analytically. For lightly damped structures this method is much more effective than numerical integration and maintains full accuracy in the modal properties of the structural model. The accuracy and efficiency of the method is illustrated by a numerical example.     

Robust estimation of covariance matrices

Data manipulations which increase the robustness and accuracy of estimators of covariance parameters by using the innovations correlation approach are considered. The procedures are especially useful for improving estimates of process-noise covariance parameters for slowly varying systems when measurement noise is large. The innovations correlate covariance estimation technique developed by P.R. Belanger (1974) is extended to the case where process noise is weak in magnitude compared to measurement noise. Belanger's method exploits the linear relationship between the desired noise covariance parameters and the correlations of the innovation sequence of a suboptimal Kalman filter to formulate a least-squares algorithm. The estimates of the process-noise covariance parameters are improved by low-pass prefiltering and downsampling the data before applying the least-squares innovations correlation algorithm. Results for a single-output, linear time-invariant system are stated, and the subsequent analysis treats only this case     

Finite series solutions for the transition matrix and the covariance of a time-invariant system

The transition matrixvarphicorresponding to then-dimensional matrixAcan be represented byvarphi(t) = g_{1}(t)I + g_{2}(t)A + ... + g_{n}(t)A^{n-1}, where the vectorg^{T} = (g_{1}, ... , g_{n})is generated fromdot{g}^{T} = g^{T}A_{c}, g^{T}(0) = (1, 0, ... , 0)and Acis the companion matrix toA. The result is applied to the covariance differential equationdot{C} = AC + CA^{T} + Qand its solution is written as a finite series. The equations are presented in a form amenable for implementation on a digital computer.     

On the evaluation of transition matrices in finite fields

A technique for evaluating the transition matrix A^k of a linear sequential machine has been given by Seherba and Roesser by transforming matrix A into a matrix A_R in the real field, finding the powers A_R ^k of.A_R In in the real field and then transforming A_R ^k back to finite field. A technique for the same problem is suggested by carrying out all the computations in the finite field only, thus avoiding the cumbersomeness of the earlier technique.     

Fusion of covariance matrices of PCA and FLD

In this paper, we propose a novel approach for fusing two classifiers, specifically classifiers based on subspace analysis, during feature extraction. A method of combining the covariance matrices of the Principal Component Analysis (PCA) and Fisher Linear Discriminant (FLD) is presented. Unlike other existing fusion strategies which fuse classifiers either at data level, or at feature level or at decision level, the proposed work combines two classifiers while extracting features introducing a new unexplored area for further research. The covariance matrices of PCA and FLD are combined using a product rule to preserve the natures of both covariance matrices with an expectation to have an increased performance. In order to show the effectiveness of the proposed fusion method, we have conducted a visual simulation on iris data. The proposed model has also been tested by performing clustering on standard datasets such as Zoo, Wine, and Iris. To study the versatility of the proposed method we have carried out an experimentation on sports video shot retrieval problem. The experimental results signify that the proposed fusing approach has an improved performance over individual classifiers.     

Structural equation modeling with near singular covariance matrices

Conventional structural equation modeling involves fitting a structural model to the sample covariance matrix . Due to collinearity or small samples with practical data, nonconvergences often occur in the estimation process. For a small constant a, this paper proposes to fit the structural model to the covariance matrix . When treating as the sample covariance matrix in the maximum likelihood (ML) procedure, consistent parameter estimates are still obtained. The asymptotic distributions of the parameter estimates and the corresponding likelihood ratio statistic are studied and compared to those by the conventional ML. Two rescaled statistics for the overall model evaluation with modeling are constructed. Empirical results imply that the estimates from modeling are more efficient than those of fitting the structural model to even when data are normally distributed. Simulations and real data examples indicate that modeling allows us to evaluate the overall model structure even when is literally singular. Implications of modeling in a broader context are discussed.     

Testing equality of covariance matrices when data are incomplete

In the statistics literature, a number of procedures have been proposed for testing equality of several groups’ covariance matrices when data are complete, but this problem has not been considered for incomplete data in a general setting. This paper proposes statistical tests for equality of covariance matrices when data are missing. A Wald test (denoted by T₁), a likelihood ratio test (LRT) (denoted by R), based on the assumption of normal populations are developed. It is well-known that for the complete data case the classic LRT and the Wald test constructed under the normality assumption perform poorly in instances when data are not from multivariate normal distributions. As expected, this is also the case for the incomplete data case and therefore has led us to construct a robust Wald test (denoted by T₂) that performs well for both normal and non-normal data. A re-scaled LRT (denoted by R^*) is also proposed. A simulation study is carried out to assess the performance of T₁, T₂, R, and R^* in terms of closeness of their observed significance level to the nominal significance level as well as the power of these tests. It is found that T₂ performs very well for both normal and non-normal data in both small and large samples. In addition to its usual applications, we have discussed the application of the proposed tests in testing whether a set of data are missing completely at random (MCAR).     

Properties of separable covariance matrices and their associated gaussian random processes

A number of properties of separable covariance matrices are summarized. Expressions for the divergence of the corresponding two-dimensional Gaussian random processes are given in terms of row and column covariance matrices, and in terms of linear prediction parameters and maximum likelihood spectral estimates. Such time and frequency domain expressions are not widely known, even for one-dimensional random processes.     

Complex-valued ICA based on a pair of generalized covariance matrices

It is shown that any pair of scatter and spatial scatter matrices yields an estimator of the separating matrix for complex-valued independent component analysis (ICA). Scatter (resp. spatial scatter) matrix is a generalized covariance matrix in the sense that it is a positive definite hermitian matrix functional that satisfies the same affine (resp. unitary) equivariance property as does the covariance matrix and possesses an additional IC-property, namely, it reduces to a diagonal matrix at distributions with independent marginals. Scatter matrix is used to decorrelate the data and the eigenvalue decomposition of the spatial scatter matrix is used to find the unitary mixing matrix of the uncorrelated data. The method is a generalization of the FOBI algorithm, where a conventional covariance matrix and a certain fourth-order moment matrix take the place of the scatter and spatial scatter matrices, respectively. Emphasis is put on estimators employing robust scatter and spatial scatter matrices. The proposed approach is one among the computationally most attractive ones, and a new efficient algorithm that avoids decorrelation of the data is also proposed. Moreover, the method does not rely upon the commonly made assumption of complex circularity of the sources. Simulations and examples are used to confirm the reliable performance of our method.     

Robust estimation of constrained covariance matrices for confirmatory factor analysis

Confirmatory factor analysis (CFA) is a data analysis procedure that is widely used in social and behavioral sciences in general and other applied sciences that deal with large quantities of data (variables). The classical estimator (and inference) procedures are based either on the maximum likelihood (ML) or generalized least squares (GLS) approaches which are known to be nonrobust to departures from the multivariate normal assumption underlying CFA. A natural robust estimator is obtained by first estimating the (mean and) covariance matrix of the manifest variables and then “plug-in” this statistic into the ML or GLS estimating equations. This two-stage method however does not fully take into account the covariance structure implied by the CFA model. An S-estimator for the parameters of the CFA model that is computed directly from the data is proposed instead and the corresponding estimating equations and an iterative procedure are derived. It is also shown that the two estimators have different asymptotic properties. A simulation study compares the finite sample properties of both estimators showing that the proposed direct estimator is more stable (smaller MSE) than the two-stage estimator.     

多尺度积的协方差矩阵行列式的角点检测方法

研究平面轮廓局部支撑域上的协方差矩阵,通过对图像协方差矩阵的特征值和特征向量的分析,以V角点模型为例,证明了协方差矩阵行列式在角点位置有唯一的极值响应.同时,为了有效地融合各个尺度信息,采用多尺度乘积方法来增强角点响应的幅度,抑制非角点或噪声的幅度.基于此,提出以多尺度乘积的协方差矩阵行列式作为角点响应函数的角点检测算...     

Modeling inverse covariance matrices by expansion of tied basis matrices for online handwritten Chinese character recognition

The state-of-the-art modified quadratic discriminant function (MQDF) based approach for online handwritten Chinese character recognition (HCCR) assumes that the feature vectors of each character class can be modeled by a Gaussian distribution with a mean vector and a full covariance matrix. In order to achieve a high recognition accuracy, enough number of leading eigenvectors of the covariance matrix have to be retained in MQDF. This paper presents a new approach to modeling each inverse covariance matrix by basis expansion, where expansion coefficients are character-dependent while a common set of basis matrices are shared by all the character classes. Consequently, our approach can achieve a much better accuracy–memory tradeoff. The usefulness of the proposed approach to designing compact HCCR systems has been confirmed and demonstrated by comparative experiments on popular Nakayosi and Kuchibue Japanese character databases.     

Estimation of time series noise covariance using correlation technology

Covariance of clean signal and observed noise is necessary for extracting clean signal from a time series.This is transferred to calculate the covariance of observed noise and clean signal’s MA process,when the clean signal is described by an autoregressive moving average (ARMA) model.Using the correlations of the innovations data from observed time series to form a least-squares problem,a concisely autocovariance least-square (CALS) method has been proposed to estimate the covariance.We also extended our work to the case of unknown MA process coefficients.Comparisons between Odelson’s autocovariance least-square (ALS) estimation algorithm and the proposed CALS method show that the CALS method could get a much more exact and compact estimation of the covariance than ALS and its extended form.     

Estimation of noise covariance matrices for a linear time-varying stochastic process

An algorithm is given to estimate the noise covariance matrices for a linear, discrete, time-varying stochastic system. If these matrices are linear with respect to a set of aparameters, it is found that the correlation products of the innovations sequence is also linear in these parameters. The fact is used to derive a least-squares algorithm, which takes a particularly simple form in the stationary case. Two examples are given.     

Nonnegative companion matrices and star-height of N-rational series

We prove a new result on N-rational series in one variable. This result gives, under an appropriate hypothesis, a necessary and sufficient condition for an N-rational series to be of star-height 1. The proof uses a theorem of Handelman on integral companion matrices.     

Three algorithms to compute covariance matrices: Comparison of their computational complexity

Three numerical algorithms for computing the solution of the covariance matrix differential equations of states of a linear time-invariant dynamical system forced by white Gaussian noise are analyzed. Estimates of errors due to truncation and roundoff are derived for each algorithm. The error analyses are based on the assumption that computation is performed in floating point mode and that it is not numerically ill-conditioned. Computational complexity of each algorithm is also discussed. Two numerical examples are presented to evaluate the performance of each algorithm.     

The nonlinearity structure of point feature SLAM problems with spherical covariance matrices

This paper proves that the optimization problem of one-step point feature Simultaneous Localization and Mapping (SLAM) is equivalent to a nonlinear optimization problem of a single variable when the associated uncertainties can be described using spherical covariance matrices. Furthermore, it is proven that this optimization problem has at most two minima. The necessary and sufficient conditions for the existence of one or two minima are derived in a form that can be easily evaluated using observation and odometry data. It is demonstrated that more than one minimum exists only when the observation and odometry data are extremely inconsistent with each other. A numerical algorithm based on bisection is proposed for solving the one-dimensional nonlinear optimization problem. It is shown that the approach extends to joining of two maps, thus can be used to obtain an approximate solution to the complete SLAM problem through map joining.