Maximum Likelihood Estimation for Stochastic Differential Equations Using Sequential Gaussian-Process-Based Optimization

Stochastic differential equations (SDEs) are used as statistical models in many disciplines. However, intractable likelihood functions for SDEs make inference challenging, and we need to resort to simulation-based techniques to estimate and maximize the likelihood function. While importance sampling methods have allowed for the accurate evaluation of likelihoods at fixed parameter values, there is still a question of how to find the maximum likelihood estimate. In this article, we propose an efficient Gaussian-process-based method for exploring the parameter space using estimates of the likelihood from an importance sampler. Our technique accounts for the inherent Monte Carlo variability of the estimated likelihood, and does not require knowledge of gradients. The procedure adds potential parameter values by maximizing the so-called expected improvement, leveraging the fact that the likelihood function is assumed to be smooth. Our simulations demonstrate that our method has significant computational and efficiency gains over existing grid- and gradient-based techniques. Our method is applied to the estimation of ocean circulation from Lagrangian drift data in the South Atlantic ocean.     

一种具有特殊滤波器的递推极大似然法

本文提出一种采用特殊滤波器的递推极大似然法，该法不需单独估计滤波器的参数与阶次，而只需递推估计模型参数，其算法简单，计算量小，辨识精度高，收敛性好，并可用于在线辨识和实时数据处理。文中还通过计算机仿真与传统的参数辨识方法进行了分析比较。     

Serial Inspection Plan in the Presence of Inspection Errors: Maximum Likelihood and Maximum Entropy Approaches

A high-quality inspection must ensure most of the detectable faults are, indeed, detected. In practice, however, the effectiveness of inspections varies widely from inspector to inspector and, in some cases, multiple inspectors are assigned to inspect the same product in a sequential manner. For such a serial inspection, we propose two new methods that can estimate the number of undetected faults in the product. In a numerical analysis, we compare the performances of the maximum likelihood estimator and the maximum entropy method with that of an existing method. The maximum entropy method is shown to perform very well, particularly when the detection probabilities are not the same among the inspectors.     

Aids for Fitting the Gamma Distribution by Maximum Likelihood

This paper presents a new table and some approximating polynomials especially designed to facilitate maximum likelihood estimation of the parameters of the gamma distribution, and also applicable to the a-parameter Type V; it discusses methods of computing the sampling variances of the likelihood estimators; it illustrates the use of the tables in a numerical example; it mentions applications to the Erlang distribution and difficulties of application to the general Type III. Finally it inquires when the numbers extracted from the tables are maximum likelihood estimates, and what they are estimates of.     

Comparison of methods to extract an asymmetry parameter from data

Several methods to extract an asymmetry parameter in an event distribution function are discussed and compared in terms of statistical precision and applicability. These methods are: simple counting rate asymmetries, event weighting procedures and the unbinned extended maximum likelihood method. It is known that weighting methods reach the same figure of merit (FOM) as the likelihood method in the limit of vanishing asymmetries. This article presents an improved weighting procedure reaching the FOM of the likelihood method for arbitrary asymmetries. Cases where the maximum likelihood method is not applicable are also discussed.     

Calculation of hazard indices for highway-railway crossings in Canada

This paper presents a method for deriving the distribution of future accidents at a particular crossing based on several crossing characteristics. The accident history of a crossing is assumed to be a Poisson process, with its parameter a linear function of certain predictor variables. This parameter, the hazard index, is formed by solving maximum likelihood normal equations using an iterative process. Test procedures are given for the model using asymptotic results. Ramifications of these results, relative to the present government allocation methods for crossing upgrading are discussed.     

Maximum Likelihood Estimation of the Parameters of the Gamma Distribution and Their Bias

The numerical technique of the maximum likelihood method to estimate the parameters of Gamma distribution is examined. A convenient table is obtained to facilitate the maximum likelihood estimation of the parameters and the estimates of the variance-covariance matrix. The bias of the estimates is investigated numerically. The empirical result indicates that the bias of both parameter estimates produced by the maximum likelihood method is positive.     

The Maximum Likelihood Estimation Under Curtailed Three Class Attributes Plans

In this paper Semi-curtailed and Fully-curtailed three class attributes plans are introduced. The maximum likelihood estimators of the proportion of bad units and that of marginal units under these plans are obtained. The efficiency of these estimators in terms of saving in inspection is discussed. A graphical procedure to simplify the execution of these plans is provided.     

Maximum Likelihood Estimators of Regression Coefficients for the Case of Autocorrelated Residuals

The classical linear regression model is extended to include the case in which the residuals are dependent with covariance matrix

where

A system of equations in the maximum likelihood estimators for the regression coefficients, a, and γ is derived and an iterative procedure for solving the system of equations is developed.     

On A Maximum Likelihood Method for Clustering

Behaviormetrika - In this paper, we propose a new clustering method based on the concept of maximum likelihood (ML) estimation. In general, the problem of local minima arises when we try to use the...     

APP roximate Maximum Likelihood Estimates From Grouped Dlata

In this paper methods are developed for obtaining approximate maximum likelihood, m.l., estimates of parameters of distribution functions, d.f., under conditions where the dat'a are grouped. Both the cases of a multivariate d.f. under equal grouping on all co-ordinates and a one-dimensional d.f. under uneqlual grouping are considered. These are extensions to Lindley's results of 1949.

The procedure requires that for any particular d.f. and grouping set-up, a correction factor, usually depending on the grouping width, be calculated. This factor when added to a specific initial estimate should provide reasonably close approximations to the m.1. estimates under grouping. The sampling variances of these approximations are also obtained and they are found to depend, in part, on the squares of the interval widths.     

Approximate solution of singular integro-differential equations in elastic contact problems

A method for the numerical solution of singular integro-differential equations is proposed. The approximate solution is sought in the form of the sum of a power series with unknown coefficients multiplied by a special term which controls the appropriate solution behaviour near and at the edges of the interval. The coefficients are to be determined from a system of linear algebraic equations. The method is applied to the solution of a contact problem of a disk inserted in an infinite elastic plane. Exact analytical solution is obtained for the particular case when the disk is of the same material as the plane. Comparison is made between the exact and the approximate solutions as well as with the solutions previously available in literature. The stability and the accuracy of the present method is investigated under variation of the parameters involved. The applicability of the method to the case when the boundary conditions for the unknown function are nonzero is discussed along with an illustrative example. A FORTRAN subroutine for the numerical solution of singular integro-differential equations is also provided.     

Likelihood ratio gradient estimation for dynamic reliability applications

This paper investigates the issue of performing a first-order sensitivity analysis in the setting of dynamic reliability. The likelihood ratio (LR) derivative/gradient estimation method is chosen to fulfill the mission. Its formulation and implementation in the system-based Monte Carlo approach that is commonly used in dynamic reliability applications is first given. To speed up the simulation, we then apply the LR method within the framework of Z-VISA, a biasing (or importance sampling) method we have developed recently. A widely discussed dynamic reliability example (a holdup tank) is studied to test the effectiveness and behaviors of the LR method when applied to dynamic reliability problems and also the effectiveness of the Z-VISA biasing technique for reducing the variance of LR derivative estimators.     

Improved finite difference method for equilibrium problems based on differentiation of the partial differential equations and the boundary conditions

A numerical algorithm for producing high-order solutions for equilibrium problems is presented. The approximated solutions are improved by differentiating both the governing partial differential equations and their boundary conditions. The advantages of the proposed method over standard finite difference methods are: the possibility of using arbitrary meshes; the possibility of using simultaneously approximations with different (distinct) orders of accuracy at different locations in the problem domain; an improvement in approximating the boundary conditions; the elimination of the need for ‘fictitious’ or ‘external’ nodal points in treating the boundary conditions. Furthermore, the proposed method is capable of reaching approximate solutions which are more accurate than other finite difference methods, when the same number of nodal points participate in the local scheme. A computer program was written for solving two-dimensional problems in elasticity. The solutions of a few examples clearly illustrate these advantages.     

Exploiting Structure of Maximum Likelihood Estimators for Extreme Value Threshold Selection

To model the tail of a distribution, one has to define the threshold above or below which an extreme value model produces a suitable fit. Parameter stability plots, whereby one plots maximum likelihood estimates of supposedly threshold-independent parameters against threshold, form one of the main tools for threshold selection by practitioners, principally due to their simplicity. However, one repeated criticism of these plots is their lack of interpretability, with pointwise confidence intervals being strongly dependent across the range of thresholds. In this article, we exploit the independent-increments structure of maximum likelihood estimators to produce complementary plots with greater interpretability, and suggest a simple likelihood-based procedure that allows for automated threshold selection. Supplementary materials for this article are available online.     

Maximum Likelihood Estimation for Periodic Autoregressive Moving Average Models

A useful class of models for seasonal time series that cannot be filtered or standardized to achieve second-order stationarity is that of periodic autoregressive moving average (PARMA) models, which are extensions of ARMA models that allow periodic (seasonal) parameters. An approximation to the exact likelihood for Gaussian PARMA processes is developed, and a straightforward algorithm for its maximization is presented. The algorithm is tested on several periodic ARMA(l, 1) models through simulation studies and is compared to moment estimation via the seasonal Yule–Walker equations. Applicability of the technique is demonstrated through an analysis of a seasonal stream-flow series from the Rio Caroni River in Venezuela.     

Maximum Likelihood Estimation for the Multinomial Distribution Using Geometric Progamming

The problem of obtaining maximum likelihood estimates for the multinomial distribution is considered. Maximum likelihood estimation is applied to the particular problem of estimating overlap sizes created by interlocking sampling frames. In this paper, geometric programming is discusseda as a method of solving the likelihood function and is applied to a practical example of estimating overlap sizes.     

A Combined Sensitive Matrix Method and Maximum Likelihood Method for Uncertainty Inverse Problems

The uncertainty inverse problems with insufficiency and imprecision in the input and/or output parameters are widely existing and unsolved in the practical engineering. The insufficiency refers to the partly known parameters in the input and/or output, and the imprecision refers to the measurement errors of these ones. In this paper, a combined method is proposed to deal with such problems. In this method, the imprecision of these known parameters can be described by probability distribution with a certain mean value and variance. Sensitive matrix method is first used to transform the insufficient formulation in the input and/or output to a resolvable one, and then the mean values of these unknown parameters can be identified by maximizing the likelihood of the measurements. Finally, to quantify the uncertainty propagation, confidence intervals of the obtained solutions are calculated based on linearization and Monte Carlo methods. Two numerical examples are presented to demonstrate the effectiveness of the present method.     

Likelihood Distributions for Estimating Functions when Both Variables are Subject to Error

When the data for estimating a function contains errors in the independent variable, the likelihood contains the unknown true values of the independent variable as nuisance parameters. These can be eliminated from the likelihood to give a new likelihood which resembles a normal distribution with variances which depend on the unknown function. The resulting least squares equations have variable weights, and must be solved by an iterative procedure.     

Estimation of density functions and upcrossing-rates from samples

This paper assesses techniques for estimating univariate probability density functions, and highlights strengths and weaknesses of several methods. Applications are given for non-Gaussian distributions typical of the dynamic response of nonlinear systems. It is concluded that of the methods considered the maximum likelihood method is generally the most suitable when predicting extreme values. The moment and maximum likelihood methods are adapted for estimation of upcrossing-rate functions.