Iterative and sequential algorithms for multisensor signalenhancement

In problems of enhancing a desired signal in the presence of noise, multiple sensor measurements will typically have components from both the signal and the noise sources. When the systems that couple the signal and the noise to the sensors are unknown, the problem becomes one of joint signal estimation and system identification. The authors specifically consider the two-sensor signal enhancement problem in which the desired signal is modeled as a Gaussian autoregressive (AR) process, the noise is modeled as a white Gaussian process, and the coupling systems are modeled as linear time-invariant finite impulse response (FIR) filters. The main approach consists of modeling the observed signals as outputs of a stochastic dynamic linear system, and the authors apply the estimate-maximize (EM) algorithm for jointly estimating the desired signal, the coupling systems, and the unknown signal and noise spectral parameters. The resulting algorithm can be viewed as the time-domain version of the frequency-domain approach of Feder et al. (1989), where instead of the noncausal frequency-domain Wiener filter, the Kalman smoother is used. This approach leads naturally to a sequential/adaptive algorithm by replacing the Kalman smoother with the Kalman filter, and in place of successive iterations on each data block, the algorithm proceeds sequentially through the data with exponential weighting applied to allow adaption to nonstationary changes in the structure of the data. A computationally efficient implementation of the algorithm is developed. An expression for the log-likelihood gradient based on the Kalman smoother/filter output is also developed and used to incorporate efficient gradient-based algorithms in the estimation process     

Robust M-estimate adaptive filtering

An M-estimate adaptive filter for robust adaptive filtering in impulse noise is proposed. Instead of using the conventional least-square cost function, a new cost function based on an M-estimator is used to suppress the effect of impulse noise on the filter weights. The resulting optimal weight vector is governed by an M-estimate normal equation. A recursive least M-estimate (RLM) adaptive algorithm and a robust threshold estimation method are derived for solving this equation. The mean convergence performance of the proposed algorithm is also analysed using the modified Huber (1981) function (a simple but good approximation to the Hampel's three-parts-redescending M-estimate function) and the contaminated Gaussian noise model. Simulation results show that the proposed RLM algorithm has better performance than other recursive least squares (RLS) like algorithms under either a contaminated Gaussian or alpha-stable noise environment. The initial convergence, steady-state error, robustness to system change and computational complexity are also found to be comparable to the conventional RLS algorithm under Gaussian noise alone     

自适应序贯M估计算法及其性能分析

针对复杂噪声环境下的参数估计问题,提出了一种稳健的自适应序贯M估计算法(Adaptive Recursive M-Estimation,ARME),并从理论分析和Monte Carlo实验仿真两方面分析了该算法的收敛性、渐进无偏特性和稳健性.理论分析和仿真试验表明:在高斯白噪声背景下,ARME具有与序贯最小二乘算法(Recursive Least Square,RLS)相近的性能;在有突出干扰等非高斯噪声背景下,与RLS相比,ARME的参数估计收敛速度更快,估计误差更小,而且在稳健性上大大优于RLS.     

Image motion estimation algorithms using cumulants

A class of algorithms is presented that estimates the displacement vector from two successive image frames consisting of signal plus noise. In the model, the signals are assumed to be either non-Gaussian or (quasistationary) deterministic; and, via a consistency result for cumulant estimators, the authors unify the stochastic and deterministic signal viewpoints. The noise sources are assumed to be Gaussian (perhaps spatially and temporally correlated) and of unknown covariance. Viewing image motion estimation as a 2D time delay estimation problem, the displacement vector of a moving object is estimated by solving linear equations involving third-order auto-cumulants and cross-cumulants. Additionally, a block-matching algorithm is developed that follows from a cumulant-error optimality criterion. Finally, the displacement vector for each pel is estimated using a recursive algorithm that minimizes a mean 2D fourth-order cumulant criterion. Simulation results are presented and discussed.     

Stochastic convergence analysis of recursive successive over-relaxation algorithm in adaptive filtering

A stochastic convergence analysis of the parameter vector estimation obtained by the recursive successive over-relaxation (RSOR) algorithm is performed in mean sense and mean-square sense. Also, excess of mean-square error and misadjustment analysis of the RSOR algorithm is presented. These results are verified by ensemble-averaged computer simulations. Furthermore, the performance of the RSOR algorithm is examined using a system identification example and compared with other widely used adaptive algorithms. Computer simulations show that the RSOR algorithm has better convergence rate than the widely used gradient-based algorithms and gives comparable results obtained by the recursive least-squares RLS algorithm.     

基于压缩感知的分布式协同估计算法

为了降低分布式协同估计算法的计算量并改善其收敛性能,提出了基于压缩感知(CS)和递归最小二乘(RLS)的分布式协同估计算法.该算法在传统RLS分布式协同估计算法的基础上引入压缩感知技术,首先在压缩域中进行递归最小二乘运算,然后利用压缩感知重构算法得到未知参数向量的估计值.提出的算法能够在增量式策略和两种模式的扩散式策略下实现对未知向量的有效估计.理论分析和仿真结果表明,该算法一方面降低了RLS分布式协同估计算法的计算量,另一方面保持较快的收敛速度与良好的均方误差性能.     

色噪声下的多传感器量测融合

传感器量测噪声通常可表示为一阶时间相关的Markov随机序列。该文将单传感器的色噪声量测滤波方法推广到多传感器的情形,提出了6种色噪声下的多传感器量测融合算法。通过协方差分析技术和仿真实验,对各算法的滤波精度、计算量和使用灵活性等性能进行了比较性评估。由评估结果得出的两条算法选取原则对实际中量测融合算法的选择具有一定的指导意义。     

Time discretization of continuous-time filters and smoothers forHMM parameter estimation

In this paper we propose algorithms for parameter estimation of fast-sampled homogeneous Markov chains observed in white Gaussian noise. Our algorithms are obtained by the robust discretization of stochastic differential equations involved in the estimation of continuous-time hidden Markov models (HMM's) via the EM algorithm. We present two algorithms: the first is based on the robust discretization of continuous-time filters that were recently obtained by Elliott to estimate quantities used in the EM algorithm; the second is based on the discretization of continuous-time smoothers, yielding essentially the well-known Baum-Welch re-estimation equations. The smoothing formulas for continuous-time HMM's are new, and their derivation involves two-sided stochastic integrals. The choice of discretization results in equations which are identical to those obtained by deriving the results directly in discrete time. The filter-based EM algorithm has negligible memory requirements; indeed, independent of the number of observations. In comparison the smoother-based discrete-time EM algorithm requires the use of the forward-backward algorithm, which is a fixed-interval smoothing algorithm and has memory requirements proportional to the number of observations. On the other hand, the computational complexity of the filter-based EM algorithm is greater than that of the smoother-based scheme. However, the filters may be suitable for parallel implementation. Using computer simulations we compare the smoother-based and filter-based EM algorithms for HMM estimation. We provide also estimates for the discretization error     

A robust past algorithm for subspace tracking in impulsive noise

The PAST algorithm is an effective and low complexity method for adaptive subspace tracking. However, due to the use of the recursive least squares (RLS) algorithm in estimating the conventional correlation matrix, like other RLS algorithms, it is very sensitive to impulsive noise and the performance can be degraded substantially. To overcome this problem, a new robust correlation matrix estimate, based on robust statistics concept, is proposed in this paper. It is derived from the maximum-likelihood (ML) estimate of a multivariate Gaussian process in contaminated Gaussian noise (CG) similar to the M-estimates in robust statistics. This new estimator is incorporated into the PAST algorithm for robust subspace tracking in impulsive noise. Furthermore, a new restoring mechanism is proposed to combat the hostile effect of long burst of impulses, which sporadically occur in communications systems. The convergence of this new algorithm is analyzed by extending a previous ordinary differential equation (ODE)-based method for PAST. Both theoretical and simulation results show that the proposed algorithm offers improved robustness against impulsive noise over the PAST algorithm. The performance of the new algorithm in nominal Gaussian noise is very close to that of the PAST algorithm.     

A fast recursive algorithm for the maximum likelihood estimation ofthe parameters of a periodic signal

Describes a new recursive algorithm for the estimation of the parameters of a periodic signal in additive Gaussian white noise. These parameters are the period, and the complex amplitudes of the harmonics present. The proposed algorithm is based on recursive maximum likelihood (ML) algorithms for incomplete data as described by Titterington (1985) and others. These algorithms are of complexity O(NM), where N is the number of harmonics, and M is the signal length. The performance of the method is compared to that of the extended Kalman filter with the aid of simulations     

Identification of hidden Markov models for ion channel currents. I.Colored background noise

Signal processing based on hidden Markov models (HMM's) has been applied recently to the characterization of single ion channel currents as recorded with the patch clamp technique from living cells. The estimation of HMM parameters using the traditional forward-backward and Baum-Welch algorithms can be performed at signal-to-noise ratios (SNR's) that are too low for conventional analysis; however, the application of these algorithms relies on the assumption that the background noise is white. In this paper, the observed single channel current is modeled as a vector hidden Markov process. An extension of the forward-backward and Baum-Welch algorithms is described to model ion channel kinetics under conditions of colored noise like that seen in patch clamp recordings. Using simulated data, we demonstrate that the traditional algorithms result in biased estimates and that the vector HMM approach provides unbiased estimates of the parameters of the underlying hidden Markov scheme     

Stochastic gradient optimization of importance sampling for theefficient simulation of digital communication systems

Importance sampling (IS) techniques offer the potential for large speed-up factors for bit error rate (BER) estimation using Monte Carlo (MC) simulation. To obtain these speed-up factors, the IS parameters specifying the simulation probability density function (PDF) must be carefully chosen. With the increased complexity in communication systems, analytical optimization of the IS parameters can be virtually impossible. We present a new IS optimization algorithm based on stochastic gradient techniques. The formulation of the stochastic gradient descent (SGD) algorithm is more general and system-independent than other existing IS methodologies, and its applicability is not restricted to a specific PDF or biasing scheme. The effectiveness of the SGD algorithm is demonstrated by two examples of communication systems where the IS techniques have not been applied before. The first example is a communication system with diversity combining, slow nonselective Rayleigh fading channel, and noncoherent envelope detection. The second example is a binary baseband communication system with a static linear channel and a recursive least square (RLS) linear equalizer in the presence of additive white Gaussian noise (AWGN)     

Parameter estimation of cyclostationary AM time series withapplication to missing observations

Time series with systematic misses occur often in practice and can be modeled as amplitude modulated ARMA processes. With this as a motivating application, modeling of cyclostationary amplitude modulated time series is addressed in the paper. Assuming that the modulating sequence is (almost) periodic, parameter estimation algorithms are developed based on second- and higher order cumulants of the resulting cyclostationary observations, which may be corrupted by any additive stationary noise of unknown covariance. If unknown, the modulating sequence can be recovered even in the presence of additive (perhaps nonstationary and colored) Gaussian, or any symmetrically distributed, noise. If the ARMA process is nonGaussian, cyclic cumulants of order greater than three can identify (non)causal and (non)minimum phase models from partial noisy data. Simulation experiments corroborate the theoretical results     

高斯噪声中的参数盲估计

盲信号处理方法中常忽略噪声的影响,而实际问题中噪声的影响是存在的.本文主要讨论了在协方差矩阵未知的加性高斯噪声中混合系数的盲估计问题.本文以最大似然估计为基础,提出一种求解参数的最优化算法,给出了混合矩阵和协方差矩阵的计算式.采用高斯混合模型(GMM)来逼近源信号的概率密度函数,简化了算法中的积分,导出了一种基于EM算法的迭代式.仿真表明,算法不仅能稳定收敛,而且在低信噪比下也能获得良好性能.     

四阶累积量稀疏表示的DOA估计方法

针对现有稀疏重构DOA估计算法不能抑制噪声项以及在高斯色噪声背景下不再适用的问题,本文提出了基于四阶累积量稀疏重构的DOA估计方法。首先,利用接收数据的四阶累积量构建了稀疏表示模型,该模型抑制了噪声项;其次对四阶累计量矩阵进行奇异值分解,化简了稀疏表示模型,通过奇异值分解,不仅减小了数据规模,而且进一步抑制了噪声。对于稀疏表示模型的求解,先利用信号子空间与噪声子空间的正交特性选取权值矢量,然后利用加权l1范数法对模型求解实现DOA估计。理论分析和仿真实验表明本文算法在高斯白噪声和色噪声背景下均适用;能够处理非相干和相干信号,且在低信噪比条件下,对相干信号有更高的估计精度;较之同类的稀疏重构算法,本文算法具有较低的算法复杂度和更高的角度分辨力。      

高斯色噪声背景下的单通道信源数目估计算法

目前信源数目估计算法大都是基于多通道接收模型且对高斯色噪声抑制能力较差,而实际应用中单通道接收模型及色噪声环境非常普遍,因此研究色噪声背景下的单通道信源数目估计算法意义重大。针对现有算法的缺陷提出了一种基于构建信号时间快拍和四阶累积量矩阵的单通道信源数目估计算法。首先通过构建信号时间快拍实现单通道接收信号的升维得到矢量化空间,然后以此组信号空间构造出四阶累积量矩阵,并从理论上验证了该四阶累积量矩阵能有效抑制高斯白噪声及高斯色噪声的影响,最后对该矩阵进行奇异值分解并通过信息论准则估计出信源个数。仿真实验和实际信号实验都表明本文算法能较好地解决单通道信源数目估计问题,且能有效抑制高斯色噪声。      

Optimal reassignment-based complex nonlinear processing of vector discrete-continuous Markovian random processes

Methods of the Markovian estimation theory are applied to the synthesis of optimal and quasi-optimal algorithms that involve time-stepped reassignment of the values of the parameters of conditional a posteriori probability densities of filtered continuous processes. The algorithms are intended for nonlinear processing of a vector discrete-continuous Markovian random process whose continuous component is a vector diffusion Markovian process and whose discrete component is a two-component vector, each component of which is characterized by a simple several-position Markovian chain. It is assumed that one group of simultaneously processed signals is observed against the background of colored (time-correlated) noise, while the other group is observed in the presence of white Gaussian noise. Basic analytic relationships and the corresponding block diagram of a system for quasi-optimal complex nonlinear processing of signals typical of the GLONASS, GPS, Galileo, and QZSS satellite navigation systems and such satellite mobile-communications systems as the Globalstar system are presented.     

Adaptive Iterative Detectors for Phase-Uncertain Channels via Variational Bounding

The problem of iterative detection/decoding of data symbols transmitted over an additive white Gaussian noise (AWGN) channel in the presence of phase uncertainty is addressed in this paper. By modelling the phase uncertainty either as an unknown deterministic variable/process or random variable/ process with a known a priori probability density function, a number of non-Bayesian and Bayesian detection algorithms with various amount of suboptimality have been proposed in the literature to solve the problem. In this paper, a new set of suboptimal iterative detection algorithms is obtained by utilizing the variational bounding technique. Especially, applying the generic variational Bayesian (VB) framework, efficient iterative joint estimation and detection/decoding schemes are derived for the constant phase model as well as for the dynamic phase model. In addition, the relation of the VB-based approach to the optimal noncoherent receiver as well as to the classical approach via the expectation-maximization (EM) algorithm is provided. Performance of the proposed detectors in the presence of a strong dynamic phase noise is compared to the performance of the existing detectors. Furthermore, an incremental scheduling of the VB (or EM) algorithm is shown to reduce the overall complexity of the receiver.     

Adaptive nonlinear filters for narrow-band interference suppressionin spread-spectrum CDMA systems

This paper presents a novel nonlinear filter and parameter estimator for narrow band interference suppression in code division multiple access spread-spectrum systems. As in the article by Rusch and Poor (1994), the received sampled signal is modeled as the sum of the spread-spectrum signal (modeled as a finite state independently identically distributed (i.i.d.) process-here we generalize to a finite state Markov chain), narrow-band interference (modeled as a Gaussian autoregressive process), and observation noise (modeled as a zero-mean white Gaussian process). The proposed algorithm combines a recursive hidden Markov model (HMM) estimator, Kalman filter (KF), and the recursive expectation maximization algorithm. The nonlinear filtering techniques for narrow-band interference suppression presented in Rusch and Poor and our proposed HMM-KF algorithm have the same computational cost. Detailed simulation studies show that the HMM-KF algorithm outperforms the filtering techniques in Rusch and Poor. In particular, significant improvements in the bit error rate and signal-to-noise ratio (SNR) enhancement are obtained in low to medium SNR. Furthermore, in simulation studies we investigate the effect on the performance of the HMM-KF and the approximate conditional mean (ACM) filter in the paper by Rusch and Poor, when the observation noise variance is increased. As expected, the performance of the HMM-KF and ACM algorithms worsen with increasing observation noise and number of users. However, HMM-KF significantly outperforms ACM in medium to high observation noise     

A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis

This paper studies the problem of robust adaptive filtering in impulsive noise environment using a recursive least M-estimate algorithm (RLM). The RLM algorithm minimizes a robust M-estimator-based cost function instead of the conventional mean square error function (MSE). Previous work has showed that the RLM algorithm offers improved robustness to impulses over conventional recursive least squares (RLS) algorithm. In this paper, the mean and mean square convergence behaviors of the RLM algorithm under the contaminated Gaussian impulsive noise model is analyzed. A lattice structure-based fast RLM algorithm, called the Huber Prior Error Feedback-Least Squares Lattice (H-PEF-LSL) algorithm is derived. Part of the H-PEF-LSL algorithm was presented in ICASSP 2001. It has an order O(N) arithmetic complexity, where N is the length of the adaptive filter, and can be viewed as a fast implementation of the RLM algorithm based on the modified Huber M-estimate function and the conventional PEF-LSL adaptive filtering algorithm. Simulation results show that the transversal RLM and the H-PEF-LSL algorithms have better performance than the conventional RLS and other RLS-like robust adaptive algorithms tested when the desired and input signals are corrupted by impulsive noise. Furthermore, the theoretical and simulation results on the convergence behaviors agree very well with each other.