Recursive identification for multivariate autoregressive equation-error systems with autoregressive noise

This paper considers the recursive identification problems for a class of multivariate autoregressive equation-error systems with autoregressive noise. By decomposing the system into several regressive identification subsystems, a maximum likelihood recursive generalised least squares identification algorithm is proposed to identify the parameter vectors in each subsystem. In addition, a multivariate recursive generalised least squares algorithm is derived as a comparison. The numerical simulation results indicate that the maximum likelihood recursive generalised least squares algorithm can effectively estimate the parameters of the multivariate autoregressive equation-error autoregressive systems and get more accurate parameter estimates than the multivariate recursive generalised least squares algorithm.     

Maximum likelihood estimation for continuous-time autoregressive models by relaxation on residual variances ratio parameters

In this paper we derive an explicit expression for the log likelihood function of a continuous-time autoregressive model. Then, using earlier results relating the autoregressive coefficients to the set of positive parameters called residual variances ratios, we develop an iterative algorithm for computing the maximum likelihood estimator of the model, similar to one in the discrete-time case. A simple noniterative estimation method, which can be used to produce an initial estimate for the algorithm, is also proposed.     

On a constrained mixture vector autoregressive model

A mixture vector autoregressive model has recently been introduced to the literature. Although this model is a promising candidate for nonlinear multiple time series modeling, high dimensionality of the parameters and lack of method for computing the standard errors of estimates limit its application to real data. The contribution of this paper is threefold. First, a form of parameter constraints is introduced with an efficient EM algorithm for estimation. Second, an accurate method for computing standard errors is presented for the model with and without parameter constraints. Lastly, a hypothesis-testing approach based on likelihood ratio tests is proposed, which aids in the selection of unnecessary parameters and leads to the greater efficiency at the estimation. A case study employing U.S. Treasury constant maturity rates illustrates the applicability of the mixture vector autoregressive model with parameter constraints, and the importance of using a reliable method to compute standard errors.     

Multi-innovation stochastic gradient parameter estimation for input nonlinear controlled autoregressive models

This paper proposes a multi-innovation stochastic gradient (MISG) parameter estimation algorithm for an input nonlinear controlled autoregressive (IN-CAR) model, i.e., a Hammerstein nonlinear CAR system, by expanding the innovation length. The analysis and simulation results indicate that the proposed MISG algorithm can generate more accurate parameter estimates for IN-CAR systems compared with the stochastic gradient algorithm.     

受控自回归系统的带惩罚项梯度辨识算法

基于带惩罚项准则函数,研究受控自回归系统的辨识问题.首先,通过负梯度搜索,极小化带惩罚项的准则函数,得到计算参数估计的递推关系,并利用一维线搜索确定最佳步长,推导带惩罚项投影梯度辨识算法和带惩罚项随机梯度辨识算法;然后,为了提高带惩罚项随机梯度算法的收敛速度,使用多新息辨识理论,推导带惩罚项多新息随机梯度辨识算法;最后,通过仿真实例验证所提出算法的有效性.     

Motion prediction of moving objects based on autoregressive model

In this paper, we describe a framework for predicting future positions and orientation of moving obstacles in a time-varying environment using autoregressive model (ARM) with conditional maximum likelihood estimate of the model parameters. No constraints are placed on the obstacles motion. The proposed algorithm can be used in a variety of applications, one of which is robot motion planning in time varying environments     

Generalized method of moments estimation for cointegrated vector autoregressive models

In this study, a generalized method of moments (GMM) for the estimation of nonstationary vector autoregressive models with cointegration is considered. Two iterative methods are considered: a simultaneous estimation method and a switching estimation method. The asymptotic properties of the GMM estimators of these methods are found to be the same as those of the Gaussian reduced-rank estimator. Through Monte Carlo simulation, the small-sample properties of the GMM estimators are studied and compared with those of the Gaussian reduced-rank estimator and the maximum likelihood estimator considered by other researchers. In the case of small samples, the GMM estimators are more robust to deviations from normality assumptions, particularly to outliers.     

Structured parameter optimization method for the radial basis function-based state-dependent autoregressive model

An off-line structured nonlinear parameter optimization method (SNPOM) for accelerating the computational convergence of parameter estimation of the radial basis function-based state-dependent autoregressive (RBF-AR) model is proposed. Using the method, all the parameters of the RBF-AR model may be optimized automatically and simultaneously. The proposed method combines the advantages of the Levenberg-Marquardt algorithm in nonlinear parameter optimization and the least-squares method in linear parameter estimation. Case studies on two complex time series and a nonlinear chemical reaction process show that the proposed parameter optimization method exhibits significantly accelerated convergence when compared with the classic version of the Levenberg-Marquardt algorithm, and to some hybrid algorithms such as the evolutionary programming algorithm.     

Data filtering based least squares iterative algorithm for parameter identification of output error autoregressive systems

This letter derives a data filtering based least squares iterative identification algorithm for output error autoregressive systems. The basic idea is to use the data filtering technique to transform the original identification model to an equation error model and to estimate the parameters of this model. The proposed algorithm is more efficient and can produce more accurate parameter estimation than the existing least squares iterative algorithm.     

进化策略在极大似然法参数估计中的应用

采用极大似然法进行参数估计,是为了避免传统优化算法的缺点。本文采用进化策略算法与极大似然参数估计法相结合使参数估计变量不再受初始值影响,从而可获得全局更优的解。最后,以威布尔三参数分布为例进行参数估计。结果表明,该方法有求解精度高和收敛速度快的优点,从而使进化策略算法能更好地应用于数理统计中。     

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

Identifying a nonlinear radial basis function‐based state‐dependent autoregressive (RBF‐AR) time series model is the basis for solving the corresponding prediction and control problems. This paper studies some recursive parameter estimation algorithms for the RBF‐AR model. Considering the difficulty of the nonlinear optimal problem arising in estimating the RBF‐AR model, an overall forgetting gradient algorithm is deduced based on the negative gradient search. A numerical method with a forgetting factor is provided to solve the problem of determining the optimal convergence factor. In order to improve the parameter estimation accuracy, the multi‐innovation identification theory is applied to develop an overall multi‐innovation forgetting gradient (O‐MIFG) algorithm. The simulation results indicate that the estimation model based on the O‐MIFG algorithm can capture the dynamics of the RBF‐AR model very well.     

Estimation of the autoregressive parameters of a mixed autoregressive moving-average time series

The problem of estimating the autoregressive parameters of a mixed autoregressive moving-average (ARMA) time series (of known order) using the output data alone is treated. This problem is equivalent to the estimation of the denominator terms of the scalar transfer function of a stationary, linear discrete time system excited by an unobserved unenrrelated sequence input by employing only the observations of the scalar output. The solution of this problem solves the problem of the identification of the dynamics of a white-noise excited continuous-time linear stationary system using sampled data. The latter problem was suggested by Bartlett in 1946. The problem treated here has appeared before in the engineering literature. The earlier treatment yielded biased parameter estimates. An asymptotically unbiased estimator of the autoregressive parameters is obtained as the solution of a modified set of Yule-Walker equations. The asymptotic estimator covariance matrix behaves like a least-squares parameter estimate of an observation set with unknown error covariances. The estimators are also shown to be unbiased in the presence of additive independent observation noise of arbitrary finite correlation time. An example illustrates the performance of the estimating procedures.     

ECM-based maximum likelihood inference for multivariate linear mixed models with autoregressive errors

For the analysis of longitudinal data with multiple characteristics, we are devoted to providing additional tools for multivariate linear mixed models in which the errors are assumed to be serially correlated according to an autoregressive process. We present a computationally flexible ECM procedure for obtaining the maximum likelihood estimates of model parameters. A score test statistic for testing the existence of autocorrelation among within-subject errors of each characteristic is derived. The techniques for the estimation of random effects and the prediction of further responses given past repeated measures are also investigated. The methodology is illustrated through an application to a set of AIDS data and two small simulation studies.     

基于粒子滤波的非线性系统静态参数估计方法*

针对基于滤波方法的最大似然参数估计步长序列过于单一,算法收敛缓慢并很容易收敛于局部最优解的问题,提出了基于似然权值的在线EM参数估计算法(LWOEM)。通过粒子滤波方法实时估计系统的状态值变化,结合最大似然方法计算静态参数的点估计,然后通过计算更新参数的似然值来动态更新步长序列.与在线EM参数估计算法(OEM)的实验结果比较,表明该算法具有更好的适应性和收敛效果。     

Discrete variable stochastic approximation procedures and recursive autoregressive model identification

Two stochastic approximation procedures are proposed for finding a point attaining the maximum of a regression function defined and observable only at points on a set of discrete variables. The asymptotic convergence property of the procedures is discussed using the theorem of almost supermartingales. The procedures are applied to the recursive identification of autoregressive time series models. The identification procedure consists of a recursive order estimation stage and a recursive autoregressive parameter updating stage, and gives the true autoregressive model or the best autoregressive approximation model.     

基于EM算法的混合模型的参数估计

介绍了极大似然参数估计，然后介绍了混合模型极大似然参数估计的EM算法实现，最后利用计算机仿真实验验证了此算法的有效性和收敛性．     

Estimation of the order of autoregressive process

Though there are several methods for estimating the order of the autoregressive process, it is hoped that it can be estimated with high sensitivity. In this paper, an approach based on an information measure which characterizes the autoregressive process has boon made to the estimation of the order of the process. A mixing operator, which is found by adding an independent random variable to the sub-sequence of the original data sequence, is introduced. By applying the operator to the conditional entropies, we developed a statistic which estimates the order of the autoregressive process. Results of computer simulation are presented to verify and compare this algorithm with other methods.     

Feature extraction by autoregressive spectral analysis using maximum likelihood estimation: internal carotid arterial Doppler signals

Abstract: In this study, Doppler signals recorded from the internal carotid artery (ICA) of 97 subjects were processed by personal computer using classical and model-based methods. Fast Fourier transform (classical method) and autoregressive (model-based method) methods were selected for processing the ICA Doppler signals. The parameters in the autoregressive method were found by using maximum likelihood estimation. The Doppler power spectra of the ICA Doppler signals were obtained by using these spectral analysis techniques. The variations in the shape of the Doppler spectra as a function of time were presented in the form of sonograms in order to obtain medical information. These Doppler spectra and sonograms were then used to compare the applied methods in terms of their frequency resolution and the effects in determination of stenosis and occlusion in the ICA. Reliable information on haemodynamic alterations in the ICA can be obtained by evaluation of these sonograms.     

Consistency of modified LS estimation method for identifying 2-Dnoncausal SAR model parameters

Least squares (LS) and maximum likelihood (ML) are the two main methods for parameter estimation of two-dimensional (2D) noncausal simultaneous autoregressive (SAR) models. ML is asymptotically consistent and unbiased but computationally unattractive. On the other hand, conventional LS is computationally efficient but does not produce accurate parameter estimates for noncausal models. Recently, Zhao-Yu (1993) proposed a modified LS estimation method and was shown to be unbiased. In this paper we prove that, under certain assumptions, the method introduced by Zhao-Yu is also consistent     

Autoregressive spectral analysis when observations are missing

A new missing data algorithm ARFIL gives good results in spectral estimation. The log likelihood of a multivariate Gaussian random variable can always be written as a sum of conditional log likelihoods. For a complete set of autoregressive AR(p) data the best predictor in the likelihood requires only p previous observations. If observations are missing, the best AR predictor in the likelihood will in general include all previous observations. Using only those observations that fall within a finite time interval will approximate this likelihood. The resulting non-linear estimation algorithm requires no user provided starting values. In various simulations, the spectral accuracy of robust maximum likelihood methods was much better than the accuracy of other spectral estimates for randomly missing data.