Maximum likelihood parameter estimation algorithm for controlled autoregressive autoregressive models

A maximum likelihood parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) models based on the maximum likelihood principle. In this derivation, we use an estimated noise transfer function to filter the input–output data. The simulation results show that the proposed estimation algorithm can effectively estimate the parameters of such class of CARAR systems and give more accurate parameter estimates than the recursive generalized least-squares algorithm.     

Recursive identification for multivariate autoregressive equation-error systems with autoregressive noise

This paper considers the recursive identification problems for a class of multivariate autoregressive equation-error systems with autoregressive noise. By decomposing the system into several regressive identification subsystems, a maximum likelihood recursive generalised least squares identification algorithm is proposed to identify the parameter vectors in each subsystem. In addition, a multivariate recursive generalised least squares algorithm is derived as a comparison. The numerical simulation results indicate that the maximum likelihood recursive generalised least squares algorithm can effectively estimate the parameters of the multivariate autoregressive equation-error autoregressive systems and get more accurate parameter estimates than the multivariate recursive generalised least squares algorithm.     

An expectation-maximization method for spatio-temporal blind source separation using an AR-MOG source model.

In this paper, we develop a maximum-likelihood (ML) spatio-temporal blind source separation (BSS) algorithm, where the temporal dependencies are explained by assuming that each source is an autoregressive (AR) process and the distribution of the associated independent identically distributed (i.i.d.) innovations process is described using a mixture of Gaussians. Unlike most ML methods, the proposed algorithm takes into account both spatial and temporal information, optimization is performed using the expectation-maximization (EM) method, the source model is adapted to maximize the likelihood, and the update equations have a simple, analytical form. The proposed method, which we refer to as autoregressive mixture of Gaussians (AR-MOG), outperforms nine other methods for artificial mixtures of real audio. We also show results for using AR-MOG to extract the fetal cardiac signal from real magnetocardiographic (MCG) data.     

An Expectation–Maximization Method for Spatio–Temporal Blind Source Separation Using an AR-MOG Source Model

In this paper, we develop a maximum-likelihood (ML) spatio-temporal blind source separation (BSS) algorithm, where the temporal dependencies are explained by assuming that each source is an autoregressive (AR) process and the distribution of the associated independent identically distributed (i.i.d.) innovations process is described using a mixture of Gaussians. Unlike most ML methods, the proposed algorithm takes into account both spatial and temporal information, optimization is performed using the expectation-maximization (EM) method, the source model is adapted to maximize the likelihood, and the update equations have a simple, analytical form. The proposed method, which we refer to as autoregressive mixture of Gaussians (AR-MOG), outperforms nine other methods for artificial mixtures of real audio. We also show results for using AR-MOG to extract the fetal cardiac signal from real magnetocardiographic (MCG) data.     

Motion prediction of moving objects based on autoregressive model

In this paper, we describe a framework for predicting future positions and orientation of moving obstacles in a time-varying environment using autoregressive model (ARM) with conditional maximum likelihood estimate of the model parameters. No constraints are placed on the obstacles motion. The proposed algorithm can be used in a variety of applications, one of which is robot motion planning in time varying environments     

Model selection for convolutive ICA with an application to spatiotemporal analysis of EEG

We present a new algorithm for maximum likelihood convolutive independent component analysis (ICA) in which components are unmixed using stable autoregressive filters determined implicitly by estimating a convolutive model of the mixing process. By introducing a convolutive mixing model for the components, we show how the order of the filters in the model can be correctly detected using Bayesian model selection. We demonstrate a framework for deconvolving a subspace of independent components in electroencephalography (EEG). Initial results suggest that in some cases, convolutive mixing may be a more realistic model for EEG signals than the instantaneous ICA model.     

On a constrained mixture vector autoregressive model

A mixture vector autoregressive model has recently been introduced to the literature. Although this model is a promising candidate for nonlinear multiple time series modeling, high dimensionality of the parameters and lack of method for computing the standard errors of estimates limit its application to real data. The contribution of this paper is threefold. First, a form of parameter constraints is introduced with an efficient EM algorithm for estimation. Second, an accurate method for computing standard errors is presented for the model with and without parameter constraints. Lastly, a hypothesis-testing approach based on likelihood ratio tests is proposed, which aids in the selection of unnecessary parameters and leads to the greater efficiency at the estimation. A case study employing U.S. Treasury constant maturity rates illustrates the applicability of the mixture vector autoregressive model with parameter constraints, and the importance of using a reliable method to compute standard errors.     

Unsupervised learning algorithm for time series using bivariate AR(1) model

Currently, there is an increased interest in time series clustering research, particularly for finding useful similar time series in various applied areas such as speech recognition, environmental research, finance and medical imaging. Clustering and classification of time series has the potential to analyze large volumes of data. Most of the traditional time series clustering and classification algorithms deal only with univariate time series data. In this paper, we develop an unsupervised learning algorithm for bivariate time series. The initial clusters are found using K-means algorithm and the model parameters are estimated using the EM algorithm. The learning algorithm is developed by utilizing component maximum likelihood and Bayesian Information Criteria (BIC). The performance of the developed algorithm is evaluated using real time data collected from a pollution centre. A comparative study of the proposed algorithm is made with the existing data mining algorithm that uses univariate autoregressive process of order 1 (AR(1)) model. It is observed that the proposed algorithm out performs the existing algorithms.     

A Long Memory Model with Normal Mixture GARCH

We present an exploratory analysis of a class of long memory models with a normal mixture generalized autoregressive conditional heteroskedasticity innovation process. Monte Carlo results are used to infer the performance of the maximum likelihood estimator. The estimation biases are associated with, amongst others, the mixing parameter, and these biases are usually insignificant. As an illustration, we fit the proposed model to four countries inflation data. It is found that the performance of the long memory model with normal mixture generalized autoregressive conditional heteroskedasticity is better than, say, both autoregressive moving average and long memory models with a standard generalized autoregressive conditional heteroskedasticity specification in terms of the flexibility to describe both the time-varying conditional skewness and kurtosis.     

Automatic spectral analysis with missing data

The current computational power and some recently developed algorithms allow a new automatic spectral analysis method for randomly missing data. Accurate spectra and autocorrelation functions are computed from the estimated parameters of time series models, without user interaction. If only a few data are missing, the accuracy is almost the same as when all observations were available. For larger missing fractions, low-order time series models can still be estimated with a good accuracy if the total observation time is long enough. Autoregressive models are best estimated with the maximum likelihood method if data are missing. Maximum likelihood estimates of moving average and of autoregressive moving average models are not very useful with missing data. Those models are found most accurately if they are derived from the estimated parameters of an intermediate autoregressive model. With statistical criteria for the selection of model order and model type, a completely automatic and numerically reliable algorithm is developed that estimates the spectrum and the autocorrelation function in randomly missing data problems. The accuracy was better than what can be obtained with other methods, including the famous expectation–maximization (EM) algorithm.     

Detection in sea clutter based on nonlinear ARCH model

In this paper, a complex nonlinear autoregressive conditional heteroscedasticity (CNARCH) model is proposed to model sea clutter. For heteroscedastic model, since the likelihood function is not obtained from explicit probability density function (PDF) expression, it is typically referred to as a quasi-likelihood function. The corresponding quasi-maximum likelihood estimation (QMLE) of the model parameters is derived. Furthermore, the corresponding detection algorithm is derived based on this model. We also conduct the simulations of both synthetic and practical data, demonstrate that the proposed model offers higher accuracy in detection, than the linear ARCH model, when used in the sea clutter.     

Estimating a state-space model from point process observations

A widely used signal processing paradigm is the state-space model. The state-space model is defined by two equations: an observation equation that describes how the hidden state or latent process is observed and a state equation that defines the evolution of the process through time. Inspired by neurophysiology experiments in which neural spiking activity is induced by an implicit (latent) stimulus, we develop an algorithm to estimate a state-space model observed through point process measurements. We represent the latent process modulating the neural spiking activity as a gaussian autoregressive model driven by an external stimulus. Given the latent process, neural spiking activity is characterized as a general point process defined by its conditional intensity function. We develop an approximate expectation-maximization (EM) algorithm to estimate the unobservable state-space process, its parameters, and the parameters of the point process. The EM algorithm combines a point process recursive nonlinear filter algorithm, the fixed interval smoothing algorithm, and the state-space covariance algorithm to compute the complete data log likelihood efficiently. We use a Kolmogorov-Smirnov test based on the time-rescaling theorem to evaluate agreement between the model and point process data. We illustrate the model with two simulated data examples: an ensemble of Poisson neurons driven by a common stimulus and a single neuron whose conditional intensity function is approximated as a local Bernoulli process.     

Sea clutter modeling using an autoregressive generalized nonlinear-asymmetric GARCH model

The sea clutter modeling is critical to the radar design and assessment of relevant detection algorithms. In this paper, we investigate a family of generalized autoregressive conditional heteroscedastic (GARCH) processes to model the sea clutter as a time series, in which the current variance is dependent on historical information. The most general model (so-called the ALLGARCH model) provides more flexible variance structures to model non-Gaussian, asymmetry, and nonlinear properties of the clutter. However, after going through the usage of the ALLGARCH model, we find that it is not very suitable because the coefficients of the model, which are numerous, would be difficult to estimate in a real-time operating environment. Meanwhile, we find that some of the coefficients are negligible under almost all kinds of sea environments and weather conditions. Motivated by these observations, we propose a novel GARCH model for sea clutter modeling, which is a generalization of the nonlinear-asymmetric GARCH (NAGARCH) model. Considering the correlation between adjacent clutter returns, autoregressive terms are also introduced. By systematically analyzing practical sea clutter data under different sea environments, we demonstrate that the proposed model achieves comparable fitting effect to some commonly used statistical models. Also, we develop the corresponding generalized likelihood ratio test (GLRT) algorithm for the new model. Numerical simulations exhibit that the proposed detector achieves higher probability of detection, comparing with the AR-GARCH detector.     

Seasonal and trend time series forecasting based on a quasi-linear autoregressive model

Modeling and forecasting seasonal and trend time series is an important research topic in many areas of industrial and economic activity. In this study, we forecast the seasonal and trend time series using a quasi-linear autoregressive model. This quasi-linear autoregressive model belongs to a class of varying coefficient models in which its autoregressive coefficients are constructed by radial basis function networks. A combined genetic optimization and gradient-based optimization algorithm is applied for automatic selection of proper input variables and model-dependent variables, and optimizing the model parameters simultaneously. The model is tested by five monthly time series. We compare the results with those of other various methods, which show the effectiveness of the proposed approach for the seasonal time series.     

Maximum likelihood estimation in vector long memory processes via EM algorithm

We present an approach for exact maximum likelihood estimation of parameters from univariate and multivariate autoregressive fractionally integrated moving average models with Gaussian errors using the Expectation Maximization (EM) algorithm. The method takes advantage of the relation between the VARFIMA(0,d,0) process and the corresponding VARFIMA(p,d,q) process in the computation of the likelihood.     

On pseudo maximum likelihood estimation for multivariate time series models with conditional heteroskedasticity

We consider a general multivariate conditional heteroskedastic model under a conditional distribution that is not necessarily normal. This model contains autoregressive conditional heteroskedastic (ARCH) models as a special class. We use the pseudo maximum likelihood estimation method and derive a new estimator of the asymptotic variance matrix for the pseudo maximum likelihood estimator. We also study four special cases in this class, which are conditional heteroskedastic autoregressive moving-average models, regression models with ARCH errors, models with constant conditional correlations, and ARCH in mean models.     

Change Point Detection in GARCH Models for Voice Activity Detection

This paper presents a robust algorithm for voice activity detection (VAD) based on change point detection in a generalized autoregressive conditional heteroscedasticity (GARCH) process. GARCH models are new statistical methods that are used especially in economic time series and are a popular choice to model speech signals and their changing variances. Change point detection is also important in economic sciences. In this paper, no distinct probability functions are assumed for speech and noise distributions. Also, to detect speech/nonspeech intervals, no likelihood ratio test (LRT) is employed. For testing parameter constancy in GARCH models, the algorithm of the Cramer-von Mises (CVM) test is described. This test is a nonparametric test and is based on the empirical quantiles. We show that VAD is related to the parameter constancy test in GARCH process, and we illustrate several examples.     

Information hiding with maximum likelihood detector for correlated signals

In this paper, a new scaling based information hiding approach with high robustness against noise and gain attack is presented. The host signal is assumed to be stationary Gaussian with first-order autoregressive model. For data embedding, the host signal is divided into two parts, and just one patch is manipulated while the other one is kept unchanged for parameter estimation. A maximum likelihood (ML) decoder is proposed which uses the ratio of samples for decoding the watermarked data. Due to the decorrelating property of the proposed decoder, it is very efficient for watermarking highly correlated signals for which the decoding process is not straightforward. By calculating the distribution of the decision variable, the performance of the decoder is analytically studied. To verify the validity of the proposed algorithm, it is applied to artificial Gaussian autoregressive signals. Simulation results for highly correlated host signals confirm the robustness of our decoder.     

Optimal input for discrimination of autoregressive models under output amplitude constraints

Optimal input design is considered for discriminating effectively between two rival autoregressive models when the amplitude of the system output has to be regulated within a certain tolerance limit with high certainty. This constraint is more appropriate than a power constraint when an extremely large system output may cause hazardous conditions in the system. First, conditions for the optimal input are derived based on the Ds-criterion, which corresponds to the power of the likelihood ratio test. Then two approaches are presented to construct the optimal input satisfying the conditions: one is based on the idea of a Chebyshev system, and the other is an autoregressive recursion approach. Numerical simulations illustrate the applicability of the proposed optimal input for autoregressive model discrimination.