Augmenting learning function to Bayesian network inferences with maximum likelihood parameters

Computing the posterior probability distribution for a set of query variables by search result is an important task of inferences with a Bayesian network. Starting from real applications, it is also necessary to make inferences when the evidence is not contained in training data. In this paper, we are to augment the learning function to Bayesian network inferences, and extend the classical “search”-based inferences to “search + learning”-based inferences. Based on the support vector machine, we use a class of hyperplanes to construct the hypothesis space. Then we use the method of solving an optimal hyperplane to find a maximum likelihood hypothesis for the value not contained in training data. Further, we give a convergent Gibbs sampling algorithm for approximate probabilistic inference with the presence of maximum likelihood parameters. Preliminary experiments show the feasibility of our proposed methods.     

MIMO雷达最大似然参数估计

多输入多输出(MIMO)雷达使用多个天线同时发射多个独立探测信号,并使用多个天线接收目标回波信号.本文考虑了发射空域分集、相干接收MIMO雷达模型及其最大似然(ML)参数估计方法.基于最大似然准则,本文推导了两种渐近最大似然算法.仿真实验的结果表明,在均匀噪声模型中,其中一种渐近算法与基于延迟求和波束形成的最大似然算法性能接近,而另一种渐近算法性能略差,但具有较低的计算复杂度.而在非均匀噪声模型中,本文所提出的两种渐近最大似然算法的性能均优于基于延迟求和波束形成的最大似然算法.     

A recursive (on-line) maximum likelihood identification method

A recursive (on-line) identification algorithm is developed based upon the off-line maximum likelihood method by Åström and Bohlin. The basic idea of the algorithm consists in two modifications to the classical method. First an approximate noisemodel is applied to eliminate auto-regressive filtering in the computation of the noise-derivatives. Second, some approximations are introduced to make the direct recursive version of the iteration equations really on-line. The combination of the two modifications yields a compact on-line algorithm.     

Information hiding with maximum likelihood detector for correlated signals

In this paper, a new scaling based information hiding approach with high robustness against noise and gain attack is presented. The host signal is assumed to be stationary Gaussian with first-order autoregressive model. For data embedding, the host signal is divided into two parts, and just one patch is manipulated while the other one is kept unchanged for parameter estimation. A maximum likelihood (ML) decoder is proposed which uses the ratio of samples for decoding the watermarked data. Due to the decorrelating property of the proposed decoder, it is very efficient for watermarking highly correlated signals for which the decoding process is not straightforward. By calculating the distribution of the decision variable, the performance of the decoder is analytically studied. To verify the validity of the proposed algorithm, it is applied to artificial Gaussian autoregressive signals. Simulation results for highly correlated host signals confirm the robustness of our decoder.     

Development of moving window state and parameter estimators under maximum likelihood and Bayesian frameworks

Estimation of slowly varying model parameters/unmeasured disturbances is of paramount importance in process monitoring, fault diagnosis, model based advanced control and online optimization. The conventional approach to estimate drifting parameters is to artificially model them as a random walk process and estimate them simultaneously with the states. However, this may lead to a poorly conditioned problem, where the tuning of the random walk model becomes a non-trivial exercise. In this work, the moving window parameter estimator of Huang et al. [1] is recast as a moving window maximum likelihood (ML) estimator. The state can be estimated within the window using any recursive Bayesian estimator. It is assumed that, when the model parameters are perfectly known, the innovation sequence generated by the chosen Bayesian estimator is a Gaussian white noise process and is further used to construct a likelihood function that treats the model parameters as unknowns. This leads to a well conditioned problem where the only tuning parameter is the length of the moving window, which is much easier to select than selecting the covariance of the random walk model. The ML formulation is further modified to develop a maximum a posteriori (MAP) cost function by including arrival cost for the parameter. Efficacy of the proposed ML and MAP formulations has been demonstrated by conducting simulation studies and experimental evaluation. Analysis of the simulation and experimental results reveals that the proposed moving window ML and MAP estimators are capable of tracking the drifting parameters/unmeasured disturbances fairly accurately even when the measurements are available at multiple rates and with variable time delays.     

使用最速下降算法提高极大似然估计算法的节点定位精度

阐述了极大似然估计算法用于无线传感器网络节点自定位的原理;阐述了最速下降算法求非线性方程组最优解的原理;提出在距离测量误差较大的情况下,使用最速下降算法优化极大似然估计算法所得的节点定位值,并通过模拟实验证实其可行性。实验结果表明,在无须多余通信代价的条件下,优化处理使定位精度得到很大提高,且算法收敛快,计算代价小,适用于无线传感器网络的节点自定位。     

Averaging, maximum penalized likelihood and Bayesian estimation forimproving Gaussian mixture probability density estimates

We apply the idea of averaging ensembles of estimators to probability density estimation. In particular, we use Gaussian mixture models which are important components in many neural-network applications. We investigate the performance of averaging using three data sets. For comparison, we employ two traditional regularization approaches, i.e., a maximum penalized likelihood approach and a Bayesian approach. In the maximum penalized likelihood approach we use penalty functions derived from conjugate Bayesian priors such that an expectation maximization (EM) algorithm can be used for training. In all experiments, the maximum penalized likelihood approach and averaging improved performance considerably if compared to a maximum likelihood approach. In two of the experiments, the maximum penalized likelihood approach outperformed averaging. In one experiment averaging was clearly superior. Our conclusion is that maximum penalized likelihood gives good results if the penalty term in the cost function is appropriate for the particular problem. If this is not the case, averaging is superior since it shows greater robustness by not relying on any particular prior assumption. The Bayesian approach worked very well on a low-dimensional toy problem but failed to give good performance in higher dimensional problems.     

基于无线传感器网络的极大似然定位法的分析

对现在普遍应用的极大似然估计定位方法进行了分析,发现极大似然估计法参考方程(第n个方程)的锚节点(参照锚节点)测距误差和参照锚节点在定位区域内的位置对定位误差有重要影响:参照锚节点的测距误差越小,计算得到的定位精度越高;当所有方程所含误差相同时,参照锚节点在定位区域中心时定位误差小,在定位区域边缘时定位误差大;当参照锚...     

Cross-spectral visual simultaneous localization and mapping (SLAM) with sensor handover

In this work, we examine the classic problem of robot navigation via visual simultaneous localization and mapping (SLAM), but introducing the concept of dual optical and thermal (cross-spectral) sensing with the addition of sensor handover from one to the other. In our approach we use a novel combination of two primary sensors: co-registered optical and thermal cameras. Mobile robot navigation is driven by two simultaneous camera images from the environment over which feature points are extracted and matched between successive frames. A bearing-only visual SLAM approach is then implemented using successive feature point observations to identify and track environment landmarks using an extended Kalman filter (EKF). Six-degree-of-freedom mobile robot and environment landmark positions are managed by the EKF approach illustrated using optical, thermal and combined optical/thermal features in addition to handover from one sensor to another. Sensor handover is primarily targeted at a continuous SLAM operation during varying illumination conditions (e.g., changing from night to day). The final methodology is tested in outdoor environments with variation in the light conditions and robot trajectories producing results that illustrate that the additional use of a thermal sensor improves the accuracy of landmark detection and that the sensor handover is viable for solving the SLAM problem using this sensor combination.     

Dimension-reduced nonparametric maximum likelihood computation for interval-censored data

A general technique is proposed for efficient computation of the nonparametric maximum likelihood estimate (NPMLE) of a survival function. The main idea is to include a new support interval that has the largest gradient value between inclusively every two neighbouring support intervals in the support set at each iteration. It is thus able to expand the support set exponentially fast during the initial stage of computation and tends to produce the same support set of the NPMLE afterward. The use of the proposed technique needs to be combined with an algorithm that can effectively find and remove redundant support intervals, for example, the constrained Newton method, the iterative convex minorant algorithm and the subspace-based Newton method. Numerical studies show that the dimension-reducing technique works very well, especially for purely interval-censored data, where a significant computational improvement via dimension reduction is possible. Strengths and weaknesses of various algorithms are also discussed and demonstrated.     

Fuzzy classification maximum likelihood algorithms for mixed-Weibull distributions

In this paper we propose an efficient algorithm based on Yang’s (Fuzzy Sets Syst 57:365–337, 1993) concept, namely the fuzzy classification maximum likelihood (FCML) algorithm, to estimate the mixed-Weibull parameters. Compared with EM and Jiang and Murthy (IEEE Trans Reliab 44:477–488, 1995) methods, the proposed FCML algorithm presents better accuracy. Thus, we recommend FCML as another acceptable method for estimating the mixed-Weibull parameters.     

A maximum likelihood framework for determining moving edges

The determination of moving edges in an image sequence is discussed. An approach is proposed that relies on modeling principles and likely hypothesis testing techniques. A spatiotemporal edge in an image sequence is modeled as a surface patch in a 3-D spatiotemporal space. A likelihood ratio test enables its detection as well as simultaneous estimation of its related attributes. It is shown that the computation of this test leads to convolving the image sequence with a set of predetermined masks. The emphasis is on a restricted but widely relevant and useful case of surface patch, namely the planar one. In addition, an implementation of the procedure whose computation cost is merely equivalent to a spatial gradient operator is presented. This method can be of interest for motion-analysis schemes, not only for supplying spatiotemporal segmentation, but also for extracting local motion information. Moreover, it can cope with occlusion contours and important displacement magnitude. Experiments have been carried out with both synthetic and real images     

Entropy-type classification maximum likelihood algorithms for mixture models

Mixtures of distributions are popularly used as probability models for analyzing grouped data. Classification maximum likelihood (CML) is an important maximum likelihood approach to clustering with mixture models. Yang et al. extended CML to fuzzy CML. Although fuzzy CML presents better results than CML, it is always affected by the fuzziness index parameter. In this paper, we consider fuzzy CML with an entropy-regularization term to create an entropy-type CML algorithm. The proposed entropy-type CML is a parameter-free algorithm for mixture models. Some numerical and real-data comparisons show that the proposed method provides better results than some existing methods.     

Selection properties of type II maximum likelihood (empirical Bayes) in linear models with individual variance components for predictors

Maximum likelihood (ML) in the linear model overfits when the number of predictors (M) exceeds the number of objects (N). One of the possible solution is the relevance vector machine (RVM) which is a form of automatic relevance detection and has gained popularity in the pattern recognition machine learning community by the famous textbook of Bishop (2006). RVM assigns individual precisions to weights of predictors which are then estimated by maximizing the marginal likelihood (type II ML or empirical Bayes). We investigated the selection properties of RVM both analytically and by experiments in a regression setting.We show analytically that RVM selects predictors when the absolute z-ratio (|least squares estimate|/standard error) exceeds 1 in the case of orthogonal predictors and, for M = 2, that this still holds true for correlated predictors when the other z-ratio is large. RVM selects the stronger of two highly correlated predictors. In experiments with real and simulated data, RVM is outcompeted by other popular regularization methods (LASSO and/or PLS) in terms of the prediction performance. We conclude that type II ML is not the general answer in high dimensional prediction problems.In extensions of RVM to obtain stronger selection, improper priors (based on the inverse gamma family) have been assigned to the inverse precisions (variances) with parameters estimated by penalized marginal likelihood. We critically assess this approach and suggest a proper variance prior related to the Beta distribution which gives similar selection and shrinkage properties and allows a fully Bayesian treatment.     

Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method for errors-in-variables and output error identification

For identifying errors-in-variables models, the time domain maximum likelihood (TML) method and the sample maximum likelihood (SML) method are two approaches. Both methods give optimal estimation accuracy but under different assumptions. In the TML method, an important assumption is that the noise-free input signal is modelled as a stationary process with rational spectrum. For SML, the noise-free input needs to be periodic. It is interesting to know which of these assumptions contain more information to boost the estimation performance. In this paper, the estimation accuracy of the two methods is analyzed statistically for both errors-in-variables (EIV) and output error models (OEM). Numerical comparisons between these two estimates are also done under different signal-to-noise ratios (SNRs). The results suggest that TML and SML have similar estimation accuracy at moderate or high SNR for EIV. For OEM identification, these two methods have the same accuracy at any SNR.     

扫描辐射源的最大似然定位算法

针对运动扫描辐射源的扫描速率未知的情况,提出一种最大似然定位算法.该算法将各个时差量测分别投影到目标状态空间,通过对目标状态进行修正、迭代和融合实现对扫描速率和目标状态的最大似然估计,同时避免联合估计方法对运动目标的定位精度不足的问题.此外,针对扫描辐射源定位的特点,重新推导分析误差项,从而改进瞬时定位算法.仿真结果表明,所提出的最大似然定位算法及其改进方法是有效的,后者的定位精度接近CRLB.     

Accuracy of linear multiple-input multiple-output (MIMO) models obtained by maximum likelihood estimation

In this paper, we study the accuracy of linear multiple-input multiple-output (MIMO) models obtained by maximum likelihood estimation. We present a frequency-domain representation for the information matrix for general linear MIMO models. We show that the variance of estimated parametric models for linear MIMO systems satisfies a fundamental integral trade-off. This trade-off is expressed as a multivariable ‘water-bed’ effect. An extension to spectral estimation is also discussed.     

Model-based classification with dissimilarities: a maximum likelihood approach

Most of classification problems concern applications with objects lying in an Euclidean space, but, in some situations, only dissimilarities between objects are known. We are concerned with supervised classification analysis from an observed dissimilarity table, which task is classifying new unobserved or implicit objects (only known through their dissimilarity measures with previously classified ones forming the training data set) into predefined classes. This work concentrates on developing model-based classifiers for dissimilarities which take into account the measurement error w.r.t. Euclidean distance. Basically, it is assumed that the unobserved objects are unknown parameters to estimate in an Euclidean space, and the observed dissimilarity table is a random perturbation of their Euclidean distances of gaussian type. Allowing the distribution of these perturbations to vary across pairs of classes in the population leads to more flexible classification methods than usual algorithms. Model parameters are estimated from the training data set via the maximum likelihood (ML) method, and allocation is done by assigning a new implicit object to the group in the population and positioning in the Euclidean space maximizing the conditional group likelihood with the estimated parameters. This point of view can be expected to be useful in classifying dissimilarity tables that are no longer Euclidean due to measurement error or instabilities of various types. Two possible structures are postulated for the error, resulting in two different model-based classifiers. First results on real or simulated data sets show interesting behavior of the two proposed algorithms, ant the respective effects of the dissimilarity type and of the data intrinsic dimension are investigated. For these latter two aspects, one of the constructed classifiers appears to be very promising. Interestingly, the data intrinsic dimension seems to have a much less adverse effect on our classifiers than initially feared, at least for small to moderate dimensions.