FIR system identification using third- and fourth-order cumulants

A new set of equations relating the coefficients of a finite-impulse-response (FIR) system and the third- and forth-order cumulants of the system output are derived. Based on these equations, two new methods to estimate FIR parameters are presented. Simulation results show that these methods perform better than other recently published linear methods in the additive coloured Gaussian noise case. This improvement is due to the fact that they do not make use of any correlation information and that they employ several slices of third- and forth-order cumulants     

Caractérisation des images stationnaires par des modèles non gaussiens bidimensionnels à moyenne ajustée

In this paper, four batch least squares linear approaches are presented for identification of non minimum phase bidimensional non Gaussian moving average (MA) models, and a relationship between autocorrelation and cumulant sequences is given. One of the proposed methods is cumulant-based only but the others use both autocorrelations and m-th order cumulants (m. > 2). Three of them are derived from the Brillinger-Rosenblatt’s non linear relation by using the Tugnait’s closed-form solution. Also, we generalize to m-th order cumulants the 2-D version of Giannakis-Mende’s approach. By simulations, we test and compare the Tugnait’s closed-form solution and the proposed methods, and we evaluate the performance of our relationship in noisy environment. Finally, we propose to characterize textured images by a 2-D ma model witch will be identified using our approaches in noisy and free noise cases.     

Fitting MA models to linear non-Gaussian random fields using higherorder cumulants

A general (possibly nonminimum phase and/or asymmetric noncausal) two-dimensional (2-D) moving average (MA) model driven by a zero-mean i.i.d. 2-D sequence is considered. The input sequence is not observed. The signal observations may be noisy. We consider the problems of model order determination and model parameter estimation using the higher order (third- or fourth-order, for example) cumulants of the 2-D signal. Second-order statistics of the data can consistently identify only a smaller class of MA models. The proposed approaches are illustrated via computer simulations     

New approaches without postprocessing to FIR system identificationusing selected order cumulants

In this paper, we address the problem of identifying the parameters of the nonminimum-phase FIR system from the cumulants of noisy output samples. The system is driven by an unobservable, zero-mean, independent and identically distributed (i.i.d) non-Gaussian signal. The measurement noise may be white Gaussian, colored MA, ARMA Gaussian processes, or even real. For this problem, two novel methods are proposed. The methods are designed by using higher order cumulants with the following advantages. (i) Flexibility: method 1 employs two arbitrary adjacent order cumulants of output, whereas method 2 uses three cumulants of output: two cumulants with arbitrary orders and the other one with an order equal to the summation of the two orders minus one. Because of this flexibility, we can select cumulants with appropriate orders to accommodate different applications. (ii) Linearity: both the formulations in method 1 and method 2 are linear with respect to the unknowns, unlike the existing cumulant-based algorithms. The post-processing is thus avoided. Extensive experiments with ARMA Gaussian and three real noises show that the new algorithms, especially algorithm 1, perform the FIR system identification with higher efficiency and better accuracy as compared with the related algorithms in the literature     

Cumulant based identification approaches for nonminimum phase FIRsystems

Recursive and least squares methods for identification of non-minimum-phase linear time-invariant (NMP-LTI) FIR systems are developed. The methods utilize the second- and third-order cumulants of the output of the FIR system whose input is an independent, identically distributed (i.i.d.) non-Gaussian process. Since knowledge of the system order is of utmost importance to many system identification algorithms, new procedures for determining the order of an FIR system using only the output cumulants are also presented. To illustrate the effectiveness of the methods, various simulation examples are presented     

Adaptive blind equalization using second- and higher orderstatistics

This paper presents two classes of adaptive blind algorithms based on second- and higher order statistics. The first class contains fast recursive algorithms whose cost functions involve second and third- or fourth-order cumulants. These algorithms are stochastic gradient-based but have structures similar to the fast transversal filters (FTF) algorithms. The second class is composed of two stages: the first stage uses a gradient adaptive lattice (GAL) while the second stage employs a higher order-cumulant (HOC) based least mean squares (LMS) filter. The computational loads for these algorithms are all linearly proportional to the number of taps used. Furthermore, the second class, as various numerical examples indicate, yields very fast convergence rates and low steady state mean square errors (MSE) and intersymbol interference (ISI). MSE convergence analyses for the proposed algorithms are also provided and compared with simulation results     

Modeling (almost) periodic moving average processes using cyclicstatistics

Estimating parameters of almost cyclostationary non-Gaussian moving average (MA) processes using noisy output-only data is considered. It is shown that second-order cyclic correlations of the output are generally insufficient in uniquely characterizing almost periodically time-varying MA(q) models, while third-order and higher order cumulants can be used to estimate their model parameters within a scale factor. Both linear and nonlinear identification algorithms for fixed and time-varying order q(t) are presented. Statistical model order determination procedures are also derived. Implementation issues are discussed and resistance to noise is claimed when the signal of interest has cycles distinct from the additive noise. Simulations are performed to verify the theoretical results     

Sample cumulants of stationary processes: asymptotic results

In this paper, we present the formulas of the covariances of the second-, third-, and fourth-order sample cumulants of stationary processes. These expressions are then used to obtain the analytic performance of FIR system identification methods as a function of the coefficients and the statistics of the input sequence. The lower bound in the variance is also compared for different sets of sample statistics to provide insight about the information carried by each sample statistic. Finally, the effect that the presence of noise has on the accuracy of the estimates is studied analytically. The results are illustrated graphically with plots of the variance of the estimates as a function of the parameters or the signal-to-noise ratio. Monte Carlo simulations are also included to compare their results with the predicted analytic performance     

Recursive least squares identification methods for multivariate pseudo-linear systems using the data filtering

This paper concerns the parameter identification methods of multivariate pseudo-linear autoregressive systems. A multivariate recursive generalized least squares algorithm is presented as a comparison. By using the data filtering technique, a multivariate pseudo-linear autoregressive system is transformed into a filtered system model and a filtered noise model, and a filtering based multivariate recursive generalized least squares algorithm is developed for estimating the parameters of these two models. The proposed algorithm achieves a higher computational efficiency than the multivariate recursive generalized least squares algorithm, and the simulation results prove that the proposed method is effective.     

Extending HOC-based methods for identifying the diagonal parameters of quadratic systems

In this paper, methods developed for the linear case of identifying the diagonal parameters of quadratic systems are extended to nonlinear case. Firstly, nonlinear relationships between model kernels and output cumulants are presented. Secondly, the relationship linking output cumulants and the coefficients of systems presented in the linear case, is extended to the general case of nonlinear quadratic systems identification. According to this concept, two nonlinear approaches are developed, the first use the fourth-order cumulants, and the second combined the third- and fourth-order cumulants. The numerical simulation results, for various signal to noise ratio (SNR) and 200 Monte Carlo runs, show that the proposed approaches achieve better accuracy, as compared with the related algorithm in the literature. Furthermore, the second algorithm is more precise in high noise environment (smallest \(\mathrm{SNR}=0\) dB), but the first algorithm more efficient in the weak noise environment case (highest SNR \(\ge \) 8 dB) comparing to using others methods.     

一种遮挡检测的核最小二乘视觉跟踪算法

针对视觉跟踪中目标、背景的复杂变化问题,提出一种遮挡检测的核最小二乘视觉跟踪算法。首先,以带约束的核最小二乘方法建立视觉跟踪优化模型,训练阶段,循环移位基采样构造训练样本集,达到稠密采样目的,利用循环矩阵的优良特性,通过快速傅里叶变换高效计算核最小二乘问题;同时,提出了基于高阶累积量的遮挡、形变等复杂变化的检测方法,改进分类器的更新处理机制。实验结果表明,在各种具有挑战性的视频序列,与现有最好算法对比,在实时性和精度方面,本文所提算法都具有较优的性能。      

A new recursive pseudo least squares algorithm for ARMA filteringand modeling. II

For pt.I see ibid., vol.40, no.11, p.2766-74 (Nov. 1992). A recursive algorithm for ARMA (autoregressive moving average) filtering has been developed in a companion paper. These recursions are seen to have a lattice-like filter structure. The ARMA parameters, however, are not directly available from the coefficients of this filter. The problem of identification of the ARMA model from the coefficients of this filter is addressed here. Two new update relations for certain pseudoinverses are derived and used to obtain a recursive least squares algorithm for AR parameter estimation. Two methods for the estimation of the MA parameters are also presented. Numerical results demonstrate the usefulness of the proposed algorithms     

Higher order cumulants-based least squares for nonminimum-phasesystems identification

A third-order cumulants based adaptive recursive least-squares (CRLS) algorithm for the identification of time-invariant nonminimum phase systems, as well as time-variant nonminimum phase systems, has been successfully developed. As higher order cumulants preserve both the magnitude and the phase information of received signals, they have been considered as powerful signal processing tools for nonminimum phase systems. In this paper, the development of the CRLS algorithm is described and examined. A cost function based on the third-order cumulant and the third-order cross cumulant is defined for the development of the CRLS system identification algorithm. The CRLS algorithm is then applied to different moving average (MA) and autoregressive moving average (ARMA) models. In the case of identifying the parameters of an MA model, a direct application of the CRLS algorithm is adequate. When dealing with an ARMA model, the poles and the zeros are estimated separately. In estimating the zeros of the ARMA model, the construction of a residual time-series sequence for the MA part is required. Simulation results indicate that the CRLS algorithm is capable of identifying nonminimum phase and time-varying systems. In addition, because of the third-order cumulant properties, the CRLS algorithm can suppress Gaussian noise and is capable of providing an unbiased estimate in a noisy environment     

A nonlinear least square technique for simultaneous image registration and super-resolution.

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.     

An evaluation of the algorithms for determining local cerebral metabolic rates of glucose using positron emission tomography dynamic data

Measurement of the local cerebral metabolic rate of glucose (LCMRGlc) and the individual rate constant parameters of the [(18 )F]2-fluoro-2-deoxy-D-glucose (FDG) model can provide a clearer understanding and insight to the physiological processes in the human brain, and a quicker and more accurate means of diagnosis in clinical applications. A systematic study using simulated and clinical tissue time activity data is presented to evaluate several existing and newly developed major algorithms used for determining LCMRGlc and the individual rate constants from positron emission tomography dynamic data. The computational and statistical properties of the autoradiographic approach, weighted and unweighted nonlinear least squares methods, Patlak graphic approach, weighted integration method, linear least squares and generalized linear least squares methods are investigated and discussed in this paper.     

Identification of electrically stimulated quadriceps muscles inparaplegic subjects

This work establishes a method for the noninvasive in vivo identification of parametric models of electrically stimulated muscle in paralyzed individuals, when significant inertial loads and/or load transitions are present. The method used differs from earlier work, in that both the pulse width and stimulus period (interpulse interval) modulation are considered. A Hill-type time series model, in which the output is the product of two factors (activation and torque-angle) is used. In this coupled model, the activation dynamics depend upon velocity. Sequential nonlinear least squares methods are used in the parameter identification. The ability of the model, using identified time-varying parameters, to accurately predict muscle torque outputs is evaluated, along with the variability of the identified parameters. This technique can be used to determine muscle parameter models for biomechanical computer simulations, and for real-time adaptive control and monitoring of muscle response variations such as fatigue     

ARMA modeling of fourth-order cumulants and phase estimation

A new procedure is proposed for ARMA modeling of fourth-order cumulants and trispectrum estimation of non-Gaussian stationary random processes. The new procedure is applied to the identification of nonminimum phase systems for both phase and magnitude response estimation. It is demonstrated by means of comprehensive simulation examples that the ARMA approach exhibits improved performance over conventional trispectrum methods. ARMA model order selection criteria based on fourth-order cumulants are presented and their performance evaluated. The computational complexity of the ARMA and conventional trispectrum methods is also examined. The new procedure does not require knowledge of the non-Gaussian distribution.This work was supported by the Office of Naval Research under Contract No. ONR-N00014-86-K-0219.     

Time-Frequency ARMA Models and Parameter Estimators for Underspread Nonstationary Random Processes

Parsimonious parametric models for nonstationary random processes are useful in many applications. Here, we consider a nonstationary extension of the classical autoregressive moving-average (ARMA) model that we term the time-frequency autoregressive moving-average (TFARMA) model. This model uses frequency shifts in addition to time shifts (delays) for modeling nonstationary process dynamics. The TFARMA model and its special cases, the TFAR and TFMA models, are shown to be specific types of time-varying ARMA (AR, MA) models. They are attractive because of their parsimony for underspread processes, that is, nonstationary processes with a limited time-frequency correlation structure. We develop computationally efficient order-recursive estimators for the TFARMA, TFAR, and TFMA model parameters which are based on linear time-frequency Yule-Walker equations or on a new time-frequency cepstrum. Simulation results demonstrate that the proposed parameter estimators outperform existing estimators for time-varying ARMA (AR, MA) models with respect to accuracy and/or numerical efficiency. An application to the time-varying spectral analysis of a natural signal is also discussed.     

FIR system identification based on subspaces of a higher ordercumulant matrix

A new linear algebraic approach for identification of a nonminimum phase FIR system of known order using only higher order (>2) cumulants of the output process is proposed. It is first shown that a matrix formed from a set of cumulants of arbitrary order can be expressed as a product of structured matrices. The subspaces of this matrix are then used to obtain the parameters of the FIR system using a set of linear equations. Theoretical analysis and numerical simulation studies are presented to characterize the performance of the proposed methods     

A concurrent system for the computation of higher-order moments

The cumulants defined in terms of moments are basic to the study of higher-order statistics (HOS) of a stationary stochastic process. This paper presents a concurrent systolic array system for the computation of higher-order moments. The system allows for the simultaneous computation of the second-, third-, and fourth-order moments. The architecture achieves good speedup through its excellent exploitation of parallelism, pipelining, and reusability of some intermediate results. The computational complexity and system performance issues related to the architecture are discussed. The concurrent system is designed with the CMOS VLSI technology and is capable of operating at 3.9 MHz.