含时延和丢失数据不确定系统的FIR滤波器

针对状态空间模型中存在服从伯努利分布的时延和随机观测丢失的情况,基于极大似然法则,分别设计有限脉冲响应(finite impulse response, FIR)滤波器的慢速率批处理形式和快速率迭代形式.首先,将时延和数据丢失情况下的模型表述为服从伯努利分布的概率线性函数;然后,通过极大似然处理从而得到所提出极大似然FIR算法;最后,将在相同条件下的极大似然FIR估计、改进型卡尔曼滤波以及无偏FIR估计3种滤波方法进行对比,从估计误差、均方根误差和不确定性影响等角度进行比较分析.实验部分通过3-DOF直升机模型仿真,可发现所提出极大似然FIR估计方法在处理时延和数据丢失问题时更加有效,鲁棒性更高.     

Context inclusive function evaluation: a case study with EM-based multi-scale multi-granular image classification

Many statistical queries such as maximum likelihood estimation involve finding the best candidate model given a set of candidate models and a quality estimation function. This problem is common in important applications like land-use classification at multiple spatial resolutions from remote sensing raster data. Such a problem is computationally challenging due to the significant computation cost to evaluate the quality estimation function for each candidate model. For example, a recently proposed method of multi-scale, multi-granular classification has high computational overhead of function evaluation for various candidate models independently before comparison. In contrast, we propose an upper bound based context-inclusive approach that reduces computational overhead based on the context, i.e. the value of the quality estimation function for the best candidate model so far. We also prove that an upper bound exists for each candidate model and the proposed algorithm is correct. Experimental results using land-use classification at multiple spatial resolutions from satellite imagery show that the proposed approach reduces the computational cost significantly.     

Maximum Likelihood Estimation of the Cox–Ingersoll–Ross Model Using Particle Filters

This paper shows how to build in a computationally efficient way a maximum simulated likelihood procedure to estimate the Cox–Ingersoll–Ross model from multivariate time series. The advantage of this estimator is that it takes into account the exact likelihood function while avoiding the huge computational burden associated with MCMC methods and without the ad hoc assumption that certain bond yields are measured without error. The proposed methodology is implemented and tested on simulated data. For realistic parameter values the estimator seems to have good small sample properties, compared to the popular quasi maximum likelihood approach, even using moderate simulation sizes. The effect of simulation errors does not seem to undermine the estimation procedure.     

Variational Bayes solution of linear neural networks and its generalization performance

It is well known that in unidentifiable models, the Bayes estimation provides much better generalization performance than the maximum likelihood (ML) estimation. However, its accurate approximation by Markov chain Monte Carlo methods requires huge computational costs. As an alternative, a tractable approximation method, called the variational Bayes (VB) approach, has recently been proposed and has been attracting attention. Its advantage over the expectation maximization (EM) algorithm, often used for realizing the ML estimation, has been experimentally shown in many applications; nevertheless, it has not yet been theoretically shown. In this letter, through analysis of the simplest unidentifiable models, we theoretically show some properties of the VB approach. We first prove that in three-layer linear neural networks, the VB approach is asymptotically equivalent to a positive-part James-Stein type shrinkage estimation. Then we theoretically clarify its free energy, generalization error, and training error. Comparing them with those of the ML estimation and the Bayes estimation, we discuss the advantage of the VB approach. We also show that unlike in the Bayes estimation, the free energy and the generalization error are less simply related with each other and that in typical cases, the VB free energy well approximates the Bayes one, while the VB generalization error significantly differs from the Bayes one.     

Speech enhancement using a masking threshold constrained Kalman filter and its heuristic implementations

A masking threshold constrained Kalman filter for speech enhancement is derived in the paper. A key step in a traditional Kalman filter requires minimizing an estimation error variance between a clean signal and its estimation. Our new method is to minimize the estimation error variance under the constraint that the energy of the estimation error is smaller than a masking threshold, computed from both time-domain forward masking and frequency-domain simultaneous masking properties of human auditory systems. The new Kalman filter provides a theoretical base for the application of the masking properties in Kalman filtering for speech enhancement. Due to the high computation cost of the proposed perceptually constrained Kalman filter, a perceptual post-filter concatenated with a standard Kalman filter is also proposed as a heuristic alternative for real-time implementation. The post-filter is constructed to make the estimation error obtained from the Kalman filter lower than the masking threshold. A wavelet Kalman filter with post-filtering is introduced to further reduce the computational load. Experimental results with colored noise show that the new constrained Kalman filter method produces the best performance when compared with other recent methods, and that the proposed heuristics with post-filtering can also produce a significant performance gain over other recent methods.     

基于统计量的加权函数图像重建方法

针对图像重建的问题,提出了一种基于统计量的加权函数图像重建方法.考虑到退化图像不仅含有高斯噪声,且含有拉普拉斯噪声,利用最大似然估计的思想估计高斯噪声和拉普拉斯噪声的方差构造基于统计量的高斯和拉普拉斯权重函数;由于在图像重建过程中,噪声分布发生变化,整合L1,L2范数,设计了一种自适应加权函数;结合双边全变差(BTV)正则化算法,设计了一种自适应加权函数图像恢复方法.实验结果表明:相比基于L1-L2混合误差模型(HEM),方法的峰值信噪比(PSNR)和结构相似度(SSIM)分别平均提高了约2.07 dB,0.02,对含有多种噪声的退化图像能够取得比较理想的结果.     

On the estimation of transfer functions,regularizations and Gaussian processes—Revisited

Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input–output measurements. We formulate a classical regularization approach, focused on finite impulse response (FIR) models, and find that regularization is necessary to cope with the high variance problem. This basic, regularized least squares approach is then a focal point for interpreting other techniques, like Bayesian inference and Gaussian process regression. The main issue is how to determine a suitable regularization matrix (Bayesian prior or kernel). Several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets. Our findings based on the data bank are as follows. The classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach–the prediction error method/maximum likelihood (PEM/ML) approach. If the goal is to estimate a model of given order as well as possible, a low order model is often better estimated by the PEM/ML approach, and a higher order model is often better estimated by model reduction on a high order regularized FIR model estimated with careful regularization. Moreover, an optimal regularization matrix that minimizes the mean square error matrix is derived and studied. The importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach.     

模型辨识新方法及应用

本文提出了一种有效的模型辨识新方法．为了提高数值稳定性和计算效率，本文给出了一种递阶最大信息量（ＡＩＣ）新判据和参数估计新方法，使单输出系统的极大似然方法及模型辨识的优选判据计算量成倍减少．通过分析标量系统的ＡＩＣ标准，本文进一步导出了多输出情况下的ＡＩＣ标准，大大提高了计算效率．     

模型辨识新方法及应用

本文提出了一种有效的模型辨识新方法，为了提高数值稳定性和计算效率，本文给出了一种递阶最大信息量（ＡＩＣ）新判据和参数估计新方法，使单输出系统的极大似然方法及模型辨识的优选判据计算量成倍减少，通过分析量系统的ＡＩＣ标准，本文进一步导出了多输出情况下的ＡＩＣ标准，大大提高了计算效率。     

一种基于BOC信号的多径抑制技术

针对卫星导航接收机容易受多径信号干扰的影响,提出了一种基于最大似然估计的加权ELS（超前/滞后斜率）技术BOC多径减弱方法。该方法先经过最大似然估计得到直射信号估计值,再检测多个BOC信号自相关函数峰两边的斜率,可以有效地减小多径信号对环路的影响,性能优于传统的窄相关技术和ELS技术,并且只增加较少的计算复杂度。通过对BOC（10,5）的多径误差包络仿真分析表明：该方法能有效减小多径干扰引起的跟踪误差。     

Nonlinear filtering structure for image smoothing in mixed-noise environments

This paper introduces a new nonlinear filtering structure for filtering image data that have been corrupted by both impulsive and nonimpulsive additive noise. Like other nonlinear filters, the proposed filtering structure uses order-statistic operations to remove the effects of the impulsive noise. Unlike other filters, however, nonimpulsive noise is smoothed by using a maximum a posteriori estimation criterion. The prior model for the image is a novel Markov random-field model that models image edges so that they are accurately estimated while additive Gaussian noise is smoothed. The Markov random-field-based prior is chosen such that the filter has desirable analytical and computational properties. The estimate of the signal value is obtained at the unique minimum of the a posteriori log likelihood function. This function is convex so that the output of the filter can be easily computed by using either digital or analog computational methods. The effects of the various parameters of the model will be discussed, and the choice of the predetection order statistic filter will also be examined. Example outputs under various noise conditions will be given.     

Computing all roots of the likelihood equations of seemingly unrelated regressions

Seemingly unrelated regressions are statistical regression models based on the Gaussian distribution. They are popular in econometrics but also arise in graphical modeling of multivariate dependencies. In maximum likelihood estimation, the parameters of the model are estimated by maximizing the likelihood function, which maps the parameters to the likelihood of observing the given data. By transforming this optimization problem into a polynomial optimization problem, it was recently shown that the likelihood function of a simple bivariate seemingly unrelated regressions model may have several stationary points. Thus local maxima may complicate maximum likelihood estimation. In this paper, we study several more complicated seemingly unrelated regression models, and show how all stationary points of the likelihood function can be computed using algebraic geometry.     

Distributed algorithm for target localization in wireless sensor networks using RSS and AoA measurements

This paper addresses target localization problem in a cooperative 3-D wireless sensor network (WSN). We employ a hybrid system that fuses distance and angle measurements, extracted from the received signal strength (RSS) and angle-of-arrival (AoA) information, respectively. Based on range measurement model and simple geometry, we derive a novel non-convex estimator based on the least squares (LS) criterion. The derived non-convex estimator tightly approximates the maximum likelihood (ML) one for small noise levels. We show that the developed non-convex estimator is suitable for distributed implementation, and that it can be transformed into a convex one by applying a second-order cone programming (SOCP) relaxation technique. We also show that the developed non-convex estimator can be transformed into a generalized trust region sub-problem (GTRS) framework, by following the squared range (SR) approach. The proposed SOCP algorithm for known transmit powers is then generalized to the case where the transmit powers are different and not known. Furthermore, we provide a detailed analysis of the computational complexity of the proposed algorithms. Our simulation results show that the new estimators have excellent performance in terms of the estimation accuracy and convergence, and they confirm the effectiveness of combining two radio measurements.     

Learning outdoor color classification

We present an algorithm for color classification with explicit illuminant estimation and compensation. A Gaussian classifier is trained with color samples from just one training image. Then, using a simple diagonal illumination model, the illuminants in a new scene that contains some of the surface classes seen in the training image are estimated in a maximum likelihood framework using the expectation maximization algorithm. We also show how to impose priors on the illuminants, effectively computing a maximum a posteriori estimation. Experimental results are provided to demonstrate the performance of our classification algorithm in the case of outdoor images     

Multivariable process identification for mpc: the asymptotic method and its applications

In this work we will introduce the asymptotic method (ASYM) of identification and provide two case studies. The ASYM was developed for multivariable process identification for model based control. The method calculates time domain parametric models using frequency domain criterion. Fundamental problems, such as test signal design for control, model order/structure selection, parameter estimation and model error quantification, are solved in a systematic manner. The method can supply not only input/output model and unmeasured disturbance model which are asymptotic maximum likelihood estimates, but also the upper bound matrix for the model errors that can be used for model validation and robustness analysis. To demonstrate the use of the method for model predictive control (MPC), the identification of a Shell benchmark process (a simulated distillation column) and an industrial application to a crude unit atmospheric tower will be presented.     

Learning mixture models with the regularized latent maximum entropy principle

This paper presents a new approach to estimating mixture models based on a recent inference principle we have proposed: the latent maximum entropy principle (LME). LME is different from Jaynes' maximum entropy principle, standard maximum likelihood, and maximum a posteriori probability estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the expectation maximization (EM) algorithm can be developed. We show that a regularized version of LME (RLME), is effective at estimating mixture models. It generally yields better results than plain LME, which in turn is often better than maximum likelihood and maximum a posterior estimation, particularly when inferring latent variable models from small amounts of data.     

Terrain analysis using radar shape-from-shading

This paper develops a maximum a posteriori (MAP) probability estimation framework for shape-from-shading (SFS) from synthetic aperture radar (SAR) images. The aim is to use this method to reconstruct surface topography from a single radar image of relatively complex terrain. Our MAP framework makes explicit how the recovery of local surface orientation depends on the whereabouts of terrain edge features and the available radar reflectance information. To apply the resulting process to real world radar data, we require probabilistic models for the appearance of terrain features and the relationship between the orientation of surface normals and the radar reflectance. We show that the SAR data can be modeled using a Rayleigh-Bessel distribution and use this distribution to develop a maximum likelihood algorithm for detecting and labeling terrain edge features. Moreover, we show how robust statistics can be used to estimate the characteristic parameters of this distribution. We also develop an empirical model for the SAR reflectance function. Using the reflectance model, we perform Lambertian correction so that a conventional SFS algorithm can be applied to the radar data. The initial surface normal direction is constrained to point in the direction of the nearest ridge or ravine feature. Each surface normal must fall within a conical envelope whose axis is in the direction of the radar illuminant. The extent of the envelope depends on the corrected radar reflectance and the variance of the radar signal statistics. We explore various ways of smoothing the field of surface normals using robust statistics. Finally, we show how to reconstruct the terrain surface from the smoothed field of surface normal vectors. The proposed algorithm is applied to various SAR data sets containing relatively complex terrain structure.     

基于变方差概率模型和进化计算的WSN 定位算法

定位是无线传感器网络技术和应用的重要基础.基于接收信号强度(received signal strength,简称RSS)的定位方法是实际应用中比较重要的定位方法考虑到实际应用中不同地点RSS测量信号的方差有所不同这一特点,运用最大概率似然理论,建立了更加符合实际的基于RSS测量的概率定位模型.对于模型中目标表达式高度非线性不好求解的特点,运用进化计算理论设计出符合传感器通信特征的定位算法(location in probability maximum with evolutionary algorithm,简称PMEA)求解概率可能性最大的位置坐标点,并用随机过程在数学上证明了算法的收敛性.最后,通过对实际公开数据集的实验,证实所提出的概率模型和PMEA算法确实能够提高RSS测距定位的精度.     

基于最大似然估计的加权质心定位算法

为解决无线传感器网络中节点自身定位问题,针对接收信号强度指示（received signal strength indication,RSSI）测距误差大和质心定位算法精度低的问题,提出一种基于最大似然估计的加权质心定位算法．首先通过计算将估计距离与实际距离之间的最大似然估计值作为权值,然后在权值模型中,引进一个参数k优化未知节点周围锚节点分布,最后计算出未知节点的位置并加以修正．仿真结果表明,基于最大似然估计的加权质心算法具有定位精度高和成本低的特点,优于基于距离倒数的质心加权和基于RSSI倒数的质心加权算法,适用于大面积的室内定位．     

Low Angle Estimation: Models,Methods, and Bounds

Boman, K. and Stoica, P., Low Angle Estimation: Models, Methods, and Bounds. Digital Signal Processing11 (2001), 35–79.In this work we study the performance of elevation estimators and lower bounds of the estimation error variance for a low angle target in a smooth sea scenario using an antenna array. The article is structured around some key assumptions on multipath knowledge, signal parameterization and noise covariance. We prove that the Cramér–Rao bound is highly dependent on the multipath model, while it is the same for the different signal parameterizations, and that it is independent of the noise covariance. The Cramér–Rao bound is sometimes too optimistic and not achievable. The tighter Barankin bound is derived to predict the threshold behavior seen at low SNR. Simulations show that the maximum likelihood methods are statistically efficient and achieve the theoretical lower bound on error variance, in the case of high enough SNR. Finally we show that the bounds can be used to design an improved array structure and study the influence of multiple frequencies.