Exact filters for Newton–Raphson parameter estimation algorithms for continuous-time partially observed stochastic systems

This paper presents explicit finite-dimensional filters for implementing Newton–Raphson (NR) parameter estimation algorithms. The models which exhibit nonlinear parameter dependence are stochastic, continuous-time and partially observed. The implementation of the NR algorithm requires evaluation of the log-likelihood gradient and the Fisher information matrix. Fisher information matrices are important in bounding the estimation error from below, via the Cramer–Rao bound. The derivations are based on relations between incomplete and complete data, likelihood, gradient and Hessian likelihood functions, which are derived using Girsanov's measure transformations.     

Iterative weighted least squares algorithms for neural networks classifiers

This paper discusses learning algorithms of layered neural networks from the standpoint of maximum likelihood estimation. At first we discuss learning algorithms for the most simple network with only one neuron. It is shown that Fisher information of the network, namely minus expected values of Hessian matrix, is given by a weighted covariance matrix of input vectors. A learning algorithm is presented on the basis of Fisher's scoring method which makes use of Fisher information instead of Hessian matrix in Newton's method. The algorithm can be interpreted as iterations of weighted least squares method. Then these results are extended to the layered network with one hidden layer. Fisher information for the layered network is given by a weighted covariance matrix of inputs of the network and outputs of hidden units. Since Newton's method for maximization problems has the difficulty when minus Hessian matrix is not positive definite, we propose a learning algorithm which makes use of Fisher information matrix, which is non-negative, instead of Hessian matrix. Moreover, to reduce the computation of full Fisher information matrix, we propose another algorithm which uses only block diagonal elements of Fisher information. The algorithm is reduced to an iterative weighted least squares algorithm in which each unit estimates its own weights by a weighted least squares method. It is experimentally shown that the proposed algorithms converge with fewer iterations than error back-propagation (BP) algorithm.     

A new computationally efficient fixed-interval,discrete-time smoother

The Rauch-Tung-Streibel smoother recursion is used to derive a new smoother algorithm based upon a decomposition of the linear model dynamical equation and maximizing use of rank 1 matrix modification. This new algorithm, it turns out, parallels Bierman's forward recursive square-root information filter/backward recursive U-D factorized covariance algorithm. The new result features computational efficiency, reliance on numerically stable matrix modification algorithms, and reduced computer storage.     

Adaptive method of realizing natural gradient learning for multilayer perceptrons

The natural gradient learning method is known to have ideal performances for on-line training of multilayer perceptrons. It avoids plateaus, which give rise to slow convergence of the backpropagation method. It is Fisher efficient, whereas the conventional method is not. However, for implementing the method, it is necessary to calculate the Fisher information matrix and its inverse, which is practically very difficult. This article proposes an adaptive method of directly obtaining the inverse of the Fisher information matrix. It generalizes the adaptive Gauss-Newton algorithms and provides a solid theoretical justification of them. Simulations show that the proposed adaptive method works very well for realizing natural gradient learning.     

改进的非线性连续-离散系统的极大似然参数估计及其应用

对Jategaonkar等人给出的同时计及过程及观测噪声的非线性连续-离散系统的极大似 然算法从两个方面进行了改进:1)给出了计算灵敏度的最佳摄动有限差分算法,避免了普通 有限差分法计算灵敏度矩阵时需人为选择参数摄动量大小而带来的缺点;2)给出了具有快 速三角化平方根滤波的极大似然算法,提高了原算法的数值稳定性.上述改进算法经应用于 飞行器系统参数估计证明了方法的有效性.     

Efficient Monte Carlo computation of Fisher information matrix using prior information

The Fisher information matrix (FIM) is a critical quantity in several aspects of mathematical modeling, including input selection and confidence region calculation. Analytical determination of the FIM in a general setting, especially in nonlinear models, may be difficult or almost impossible due to intractable modeling requirements or/and intractable high-dimensional integration.To circumvent these difficulties, a Monte Carlo simulation based technique, known as resampling algorithm, is usually recommended, in which values of the log-likelihood function or its exact stochastic gradient computed based on a set of pseudo-data vectors are used. The current work proposes an extension of this resampling algorithm in order to enhance the statistical qualities of the estimator of the FIM. This modified resampling algorithm is useful in those cases when some elements of the FIM are analytically known from prior information and the rest of the elements are unknown. The estimator of the FIM resulting from the proposed algorithm simultaneously preserves the analytically known elements and reduces variances of the estimators of the unknown elements. This is achieved by capitalizing on the information contained in the known elements.     

改进的非线性连续－离散系统的极大似然参数估计及其应用

对Ｊａｔｅｇａｏｎｋａｒ等人给出的同时计及过程及观测噪声的非线性连续－离散系统的极大似然算法从两个方面进行了改进：１）给出了计算灵敏度的最佳摄动有限差分算法，避免了普通有限差分法计算灵敏度矩阵时需人为选择参数摄动量大小而带来的缺点；２）给出了具有快速三角化平方根滤波的极大似然算法，提高了原算法的数值稳定性．上述改进算法经应用于飞行器系统参数估计证明了方法的有效性。     

On scalarized calculation of the likelihood function in array square-root filtering algorithms

An efficient method of scalarized calculation of the logarithmic likelihood function based on the array square-root implementation methods for Kalman filtering formulas was proposed. The algorithms of this kind were shown to be more stable to the roundoff errors than the conventional Kalman filter. The measurement scalarization technique enables a substantial reduction in the computational complexity of the algorithm. Additionally, the new implementations are classified with the array filtering algorithms and thereby are oriented to the parallel calculations. Computational results corroborated effectiveness of the new algorithm.     

Collaborative target tracking in WSNs using the combination of maximum likelihood estimation and Kalman filtering

Target tracking using wireless sensor networks requires efficient collaboration among sensors to tradeoff between energy consumption and tracking accuracy. This paper presents a collaborative target tracking approach in wireless sensor networks using the combination of maximum likelihood estimation and the Kalman filter. The cluster leader converts the received nonlinear distance measurements into linear observation model and approximates the covariance of the converted measurement noise using maximum likelihood estimation, then applies Kalman filter to recursively update the target state estimate using the converted measurements. Finally, a measure based on the Fisher information matrix of maximum likelihood estimation is used by the leader to select the most informative sensors as a new tracking cluster for further tracking. The advantages of the proposed collaborative tracking approach are demonstrated via simulation results.     

有色噪声下的平方根UKF在天文自主导航中的应用

针对由星敏感器和光学导航相机组成的卫星天文自主导航系统, 传统的平方根UKF不能很好地解决测量噪声为有色噪声情况下的非线性滤波问题, 导致导航系统的精度下降. 为此, 提出了一种有色噪声情况下的平方根UKF方法. 同时, 为了避免在数值计算的过程中, 由于舍入误差而破坏误差协方差矩阵的正定性和对称性, 在整个递推计算过程中, 借鉴平方根Kalman滤波理论, 采用协方差矩阵平方根进行递推计算, 改善滤波算法的稳定性, 协方差矩阵的平方根更新用cholesky分解和qr分解来计算. 将该方法应用于卫星天文自主导航系统中, 实验仿真结果表明, 相对于传统的平方根UKF算法, 所设计的平方根UKF算法能够很好地解决测量噪声为有色噪声情况下估计精度低问题.     

The Poisson-exponential lifetime distribution

In this paper we proposed a new two-parameters lifetime distribution with increasing failure rate. The new distribution arises on a latent complementary risk problem base. The properties of the proposed distribution are discussed, including a formal proof of its probability density function and explicit algebraic formulae for its reliability and failure rate functions, quantiles and moments, including the mean and variance. A simple EM-type algorithm for iteratively computing maximum likelihood estimates is presented. The Fisher information matrix is derived analytically in order to obtaining the asymptotic covariance matrix. The methodology is illustrated on a real data set.     

确定采样型强跟踪滤波飞机舵面故障诊断与隔离

为了克服扩展多模型自适应估计中扩展卡尔曼滤波的理论局限性,多重渐消因子强跟踪改进引起的滤波发散现象以及多维高斯故障概率计算量大等问题,本文将一类基于确定解析采样近似方法的非线性次优高斯滤波与多模型自适应估计相结合,提出了改进的多重渐消因子强跟踪非线性滤波快速故障诊断方法.确定采样型滤波克服了扩展卡尔曼滤波的理论局限性;推导了等效多重渐消因子计算方法,避免了非线性系统雅克比矩阵的计算,提高了故障突变时的跟踪性能;提出了基于平方根分解的改进的一步预测协方差更新方程,保证了滤波稳定性;提出了基于欧几里得范数简化的故障概率计算方法,降低了计算量.通过对比仿真验证了3种不同非线性滤波算法及其强跟踪改进算法的有效性,故障诊断方法跟踪性强、速度快、精度高,具有较好的鲁棒性和稳定性.     

改进型平方根无迹卡尔曼滤波及其在无轴承永磁同步电机无速度传感器运行中的应用

平方根无迹卡尔曼滤波(SRUKF)解决了标准无迹卡尔曼滤波(UKF)中由于误差协方差阵负定而引起的滤波发散问题, 保证了算法的数值稳定性, 但仍存在对模型参数变化的鲁棒性差、收敛速度慢及对突变状态的跟踪能力低等缺陷. 因此, 本文提出一种改进SRUKF滤波, 通过引入时变渐消因子和弱化因子, 实时修正滤波增益矩阵和误差协方差平方根矩阵, 实现残差序列正交, 确保SRUKF滤波保持对目标实际状态的准确跟踪. 将该算法在无轴承永磁同步电机无速度传感器矢量控制系统中进行仿真研究. 结果表明: 改进SRUKF非线性近似精度、数值稳定性及滤波精度更高, 在系统状态突变或负载扰动时, 鲁棒性更强, 能够有效实现转速及转子角度的准确估计, 确保转子稳定悬浮运行.     

Fisher scoring: An interpolation family and its Monte Carlo implementations

The Fisher scoring method is widely used for likelihood maximization, but its application can be difficult in situations where the expected information matrix is not available in closed form or when parameters have constraints. In this paper, we describe an interpolation family that generalizes the Fisher scoring method and propose a general Monte Carlo approach that makes these generalized methods also applicable in such situations. With this approach, random samples are generated from the iteratively estimated models and used to provide estimates of the expected information. As a result, the likelihood function can be optimized by repeatedly solving weighted linear regression problems. Specific extensions of this general approach to fitting multivariate normal mixtures and to fitting mixed-effects models with a single discrete random effect are also described. Numerical studies show that the proposed algorithms are fast and reliable to use, as compared with the classical expectation-maximization algorithm.     

An algorithm for propagating the square-root covariance matrix in triangular form

A method for propagating the square root of the state error covariance matrix in lower triangular form is described. The algorithm can be combined with any triangular square-root measurement update algorithm to obtain a triangular square-root sequential estimation algorithm. The triangular square-root algorithm compares favorably with the convential sequential estimation algorithm with regard to computation time.     

Pathological behaviors of fisher confidence bounds for Weibull distribution

The Weibull distribution is widely used in reliability engineering. To estimate its parameters and associated reliability indices, the maximum likelihood (ML) approach is often employed, and the associated Fisher information matrix is used to obtain the confidence bounds on the reliability indices that are of interest. The estimates and the confidence bounds usually behave similarly in terms of monotonic and asymptotic properties. However, the confidence bounds may behave differently under certain circumstances. As a result, the Fisher matrix approach may not always be preferred in obtaining the desired confidence bounds. This paper provides some properties of Fisher confidence bounds for the Weibull distribution. These properties can be used as guidelines when implementing the ML approach and Fisher information matrix to analyze failure time data and plan life tests.     

Progressively Type-II censored competing risks data from Lomax distributions

A competing risks model based on Lomax distributions is considered under progressive Type-II censoring. Maximum likelihood estimates for the distribution parameters are established. Moreover, the expected Fisher information matrix is computed and optimal Fisher information based censoring plans are discussed. In particular, it turns out that the optimal censoring scheme depends on the particular parametrization of the Lomax distributions.     

一种基于信息融合的粒子滤波跟踪算法

针对粒子滤波在复杂背景下容易造成跟踪目标丢失的问题,提出一种基于多特征信息融合的粒子滤波算法。该方法同时利用灰度和梯度信息描述目标,有效提高了复杂场景下对目标描述的可靠性;在此基础上,推导出多信息融合的观测似然函数,将两种信息融合在一起,使得融合算法能根据当前跟踪形势自适应调整各信息的加权,实现了信息间的优势互补。实验结果表明,该算法鲁棒性较高,明显提高了跟踪精度。     

The use of a modified Fedorov exchange algorithm to optimise sampling times for population pharmacokinetic experiments

We propose a new algorithm for optimising sampling times for population pharmacokinetic experiments using D-optimality. The algorithm was used in conjunction with the population Fisher information matrix as implemented in MATLAB (PFIM 1.1 and 1.2) to evaluate population pharmacokinetic designs. The new algorithm based on the classical Fedorov exchange algorithm optimises the determinant of the population Fisher information matrix. The performance of the new algorithm has been compared with other existing algorithms including simplex, simulated annealing and adaptive random search. The new algorithm performed better especially when dealing with complex designs at the expense of longer computing times.     

Maximum likelihood estimation via the extended covariance and combined square-root filters

The method of maximum likelihood is a general method for parameter estimation and is often used in system identification. To implement it, it is necessary to maximize the likelihood function, which is usually done using the gradient approach. It involves the computation of the likelihood gradient with respect to unknown system parameters. For linear stochastic system models this leads to the implementation of the Kalman filter, which is known to be numerically unstable. The aim of this work is to present new efficient algorithms for likelihood gradient evaluation. They are more reliable in practice and improve robustness of computations against roundoff errors. All algorithms are derived in measurement and time updates form. The comparison with the conventional Kalman filter approach and results of numerical experiments are given.