Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs

In population pharmacokinetic studies, the precision of parameter estimates is dependent on the population design. Methods based on the Fisher information matrix have been developed and extended to population studies to evaluate and optimize designs. In this paper we propose simple programming tools to evaluate population pharmacokinetic designs. This involved the development of an expression for the Fisher information matrix for nonlinear mixed-effects models, including estimation of the variance of the residual error. We implemented this expression as a generic function for two software applications: S-PLUS and MATLAB. The evaluation of population designs based on two pharmacokinetic examples from the literature is shown to illustrate the efficiency and the simplicity of this theoretic approach. Although no optimization method of the design is provided, these functions can be used to select and compare population designs among a large set of possible designs, avoiding a lot of simulations.     

Iterative weighted least squares algorithms for neural networks classifiers

This paper discusses learning algorithms of layered neural networks from the standpoint of maximum likelihood estimation. At first we discuss learning algorithms for the most simple network with only one neuron. It is shown that Fisher information of the network, namely minus expected values of Hessian matrix, is given by a weighted covariance matrix of input vectors. A learning algorithm is presented on the basis of Fisher's scoring method which makes use of Fisher information instead of Hessian matrix in Newton's method. The algorithm can be interpreted as iterations of weighted least squares method. Then these results are extended to the layered network with one hidden layer. Fisher information for the layered network is given by a weighted covariance matrix of inputs of the network and outputs of hidden units. Since Newton's method for maximization problems has the difficulty when minus Hessian matrix is not positive definite, we propose a learning algorithm which makes use of Fisher information matrix, which is non-negative, instead of Hessian matrix. Moreover, to reduce the computation of full Fisher information matrix, we propose another algorithm which uses only block diagonal elements of Fisher information. The algorithm is reduced to an iterative weighted least squares algorithm in which each unit estimates its own weights by a weighted least squares method. It is experimentally shown that the proposed algorithms converge with fewer iterations than error back-propagation (BP) algorithm.     

Maximum likelihood estimation using square root information filters

The maximum likelihood parameter estimation algorithm is known to provide optimal estimates for linear time-invariant dynamic systems. However, the algorithm is computationally expensive and requires evaluations of the gradient of a log likelihood function and the Fisher information matrix. By using the square-root information filter, a numerically reliable algorithm to compute the required gradient and the Fisher information matrix is developed. The algorithm is a significant improvement over the methods based on the conventional Kalman filter. The square-root information filter relies on the use of orthogonal transformations that are well known for numerical reliability. This algorithm can be extended to real-time system identification and adaptive control     

基于Fisher块对角LNMF的彩色人脸识别

为提高对彩色人脸的识别率,提出一种基于Fisher块对角局部非负矩阵分解(LNMF)的识别算法。采用块对角矩阵编码彩色图像不同通道的颜色信息,在LNMF算法中增加块对角约束和Fisher判别约束,对不同通道的颜色信息同时进行计算并融入人脸的类别信息,用于提取人脸特征。在CVL和PIE彩色人脸数据库上的实验结果验证了该识别算法的有效性。     

基于改进LDA算法的人脸识别

提出一种基于改进LDA的人脸识别算法,该算法克服传统LDA算法的缺点,重新定义样本类间离散度矩阵和Fisher准则,从而保留住最有辨别力的信息,增强算法的识别率.实验结果证明该算法是可行的,与传统的PCA LDA算法比较,具有较高的识别率.     

Difficulty of singularity in population coding

Fisher information has been used to analyze the accuracy of neural population coding. This works well when the Fisher information does not degenerate, but when two stimuli are presented to a population of neurons, a singular structure emerges by their mutual interactions. In this case, the Fisher information matrix degenerates, and the regularity condition ensuring the Cramér-Rao paradigm of statistics is violated. An animal shows pathological behavior in such a situation. We present a novel method of statistical analysis to understand information in population coding in which algebraic singularity plays a major role. The method elucidates the nature of the pathological case by calculating the Fisher information. We then suggest that synchronous firing can resolve singularity and show a method of analyzing the binding problem in terms of the Fisher information. Our method integrates a variety of disciplines in population coding, such as nonregular statistics, Bayesian statistics, singularity in algebraic geometry, and synchronous firing, under the theme of Fisher information.     

一种基于Fisher鉴别极小准则的特征提取方法

特征提取是模式识别研究领域的一个热点．为了更好地解决人脸识别中的特征提取问题,定义了一种新的基于Fisher鉴别极小准则的特征提取方法,并且提出了类间散布矩阵零空间的概念,解决了先前Fisher线性变换方法中的最终特征维数受类别数的限制．在人脸数据库上的实验结果验证了该算法的有效性．     

The Poisson-exponential lifetime distribution

In this paper we proposed a new two-parameters lifetime distribution with increasing failure rate. The new distribution arises on a latent complementary risk problem base. The properties of the proposed distribution are discussed, including a formal proof of its probability density function and explicit algebraic formulae for its reliability and failure rate functions, quantiles and moments, including the mean and variance. A simple EM-type algorithm for iteratively computing maximum likelihood estimates is presented. The Fisher information matrix is derived analytically in order to obtaining the asymptotic covariance matrix. The methodology is illustrated on a real data set.     

一种新的图像特征抽取方法研究

对最佳鉴别矢量的求解方法进行了研究，根据矩阵的分块理论和优化理论，在一定的条件下，从理论上得到类间散布矩阵和总体散布矩阵的一种简洁表示方法，提出了求解最佳鉴别矢量的一种新算法，该算法的优点是计算量明显减少。ORL人脸数据库的数值实验，验证了上述论断的正确性。实验结果表明，虽然识别率与分块维数之间存在非线性关系，但可以通过选择适当的分块维数来获得较高的识别率。类间散布矩阵和总体散布矩阵的一种简洁表示方法适合于一切使用Fisher鉴别准则的模式识别问题。     

一种新的L1度量Fisher线性判别分析研究

Fisher线性判别分析（Fisher Linear Discriminant Analysis,FLDA）是一种典型的监督型特征提取方法,旨在最大化Fisher准则,寻求最优投影矩阵。在标准Fisher准则中,涉及到的度量为[L2]范数度量,此度量通常缺乏鲁棒性,对异常值点较敏感。为提高鲁棒性,引入了一种基于[L1]范数度量的FLDA及其优化求解算法。实验结果表明：在很多情形下,相比于传统的[L2]范数FLDA,[L1]范数FLDA具有更好的分类精度和鲁棒性。     

Some new designs of 2-D array for matrix multiplication andtransitive closure

We present some new regular iterative algorithms for matrix multiplication and transitive closure. With these algorithms, by spacetime mapping the 2-D arrays with 2N-1 and upper bound [(3N-1)/2] execution times for matrix multiplication can be obtained. Meanwhile, we can derive a 2-D array with 4N-2 execution rime for transitive closure based on the sequential Warshall-Floyd algorithm. All these new 2-D arrays for matrix multiplication and transitive closure have the advantages of faster and more regular than other previous designs     

Design of optimal multioutput sampling schedules: Computation of gradients and Hessians

The design of optimal sampling schedules in a dynamic experiment is often accomplished by maximization of a scalar function of the Fisher information matrix, using complex optimization algorithms. In this note exact analytic expressions for the gradients and Hessians of two common design criteria (D- andL-optimality) are derived. The results allow the use of a wealth of efficient gradient search techniques in such designs.     

LBP特征和改进Fisher准则的人脸识别

为了进一步提高人脸识别系统的性能,在LDRC算法的基础上进行改进,并将改进LDRC算法的准则函数应用到Fisher分类器中,提出了一种新的基于LBP特征和改进Fisher准则的人脸识别算法。该算法提取每幅人脸图像的标准LBP直方图特征：把提取到的LBP特征输入到改进后的Fisher分类器中,得到最佳投影矩阵和投票结果矩阵;求解出投票结果矩阵的最大值所对应的类别号,将其作为最终的识别结果;分别在FERET和AR人脸库中进行实验检测,结果表明与传统的特征提取方法相比,给出的方案可以使人脸识别率得到显著提高。     

传感器优化布置的距离系数-Fisher信息准则

以损伤参数识别为目标,基于传统Fisher信息准则的传感器优化布置会出现测点局部聚集现象,导致信息冗余,不利于损伤定位。针对此问题,首先以反映信息独立程度的距离系数对候选自由度的Fisher信息矩阵进行加权修正;然后以修正后的有效信息矩阵行列式最大化为目标,采用逐步累加的方法得到基于距离系数-Fisher信息准则的传感器优化布置方案。采用该方法对一个16自由度剪切型弹簧质量模型进行传感器优化布置。结果表明,该方法能有效避免测点聚集现象,解决信息冗余问题。     

Exact filters for Newton–Raphson parameter estimation algorithms for continuous-time partially observed stochastic systems

This paper presents explicit finite-dimensional filters for implementing Newton–Raphson (NR) parameter estimation algorithms. The models which exhibit nonlinear parameter dependence are stochastic, continuous-time and partially observed. The implementation of the NR algorithm requires evaluation of the log-likelihood gradient and the Fisher information matrix. Fisher information matrices are important in bounding the estimation error from below, via the Cramer–Rao bound. The derivations are based on relations between incomplete and complete data, likelihood, gradient and Hessian likelihood functions, which are derived using Girsanov's measure transformations.     

Discrete sizing/layout/topology optimization of truss structures with an advanced Jaya algorithm

Discrete optimization of truss structures is a hard computing problem with many local minima. Metaheuristic algorithms are naturally suited for discrete optimization problems as they do not require gradient information. A recently developed method called Jaya algorithm (JA) has proven itself very efficient in continuous engineering problems. Remarkably, JA has a very simple formulation and does not utilize algorithm-specific parameters. This study presents a novel JA formulation for discrete optimization of truss structures under stress and displacement constraints. The new algorithm, denoted as discrete advanced JA (DAJA), implements efficient search mechanisms for generating new trial designs including discrete sizing, layout and topology optimization variables. Besides the JA’s basic concept of moving towards the best design of the population and moving away from the worst design, DAJA tries to form a set of descent directions in the neighborhood of each candidate designs thus generating high quality trial designs that are very likely to improve current population. Results collected in seven benchmark problems clearly demonstrate the superiority of DAJA over other state-of-the-art metaheuristic algorithms and multi-stage continuous–discrete optimization formulations.     

Population coding and decoding in a neural field: a computational study

This study uses a neural field model to investigate computational aspects of population coding and decoding when the stimulus is a single variable. A general prototype model for the encoding process is proposed, in which neural responses are correlated, with strength specified by a gaussian function of their difference in preferred stimuli. Based on the model, we study the effect of correlation on the Fisher information, compare the performances of three decoding methods that differ in the amount of encoding information being used, and investigate the implementation of the three methods by using a recurrent network. This study not only rediscovers main results in existing literatures in a unified way, but also reveals important new features, especially when the neural correlation is strong. As the neural correlation of firing becomes larger, the Fisher information decreases drastically. We confirm that as the width of correlation increases, the Fisher information saturates and no longer increases in proportion to the number of neurons. However, we prove that as the width increases further--wider than (sqrt)2 times the effective width of the turning function--the Fisher information increases again, and it increases without limit in proportion to the number of neurons. Furthermore, we clarify the asymptotic efficiency of the maximum likelihood inference (MLI) type of decoding methods for correlated neural signals. It shows that when the correlation covers a nonlocal range of population (excepting the uniform correlation and when the noise is extremely small), the MLI type of method, whose decoding error satisfies the Cauchy-type distribution, is not asymptotically efficient. This implies that the variance is no longer adequate to measure decoding accuracy.     

不确定规划中一种观察信息高效约简算法

在不确定规划中,可通过观察周围的信息来区分多个状态,但周围的观察信息较多,因此如何从大量的观察信息中筛选必须的信息非常重要。以往算法是在直接搜索过程中增加一些剪枝条件来达到优化的目的,存在一定的局限性。在对观察信息约简研究中,为提高搜索效率,设计一种高效的不确定规划中观察信息约筒算法。该算法将规划问题转化为求解O—l矩阵的覆盖问题,使用数据结构十字链表来表示O-1矩阵,通过维护十字链表并采用启发式函数来加速求解一个最小观察变量集。实验结果表明,该算法不仅能够找最小观察变量集,而且运行速度超过同类算法。     

Obtaining more information from conjoint experiments by best-worst choices

Conjoint choice experiments elicit individuals’ preferences for the attributes of a good by asking respondents to indicate repeatedly their most preferred alternative in a number of choice sets. However, conjoint choice experiments can be used to obtain more information than that revealed by the individuals’ single best choices. A way to obtain extra information is by means of best-worst choice experiments in which respondents are asked to indicate not only their most preferred alternative but also their least preferred one in each choice set. To create D-optimal designs for these experiments, an expression for the Fisher information matrix for the maximum-difference model is developed. Semi-Bayesian D-optimal best-worst choice designs are derived and compared with commonly used design strategies in marketing in terms of the D-optimality criterion and prediction accuracy. Finally, it is shown that best-worst choice experiments yield considerably more information than choice experiments.     

Optimum experimental designs for dynamic systems in the presence of correlated errors

The problem of determining an optimal measurement time schedule for identification of unknown parameters in multiresponse systems when correlations between observations occur is considered. The measurement process is performed by collecting data at discrete time instants from several outputs. An observation plan is proposed based on a scalar measure of the Fisher information matrix as the design criterion quantifying the accuracy of parameter estimators. A numerical procedure is proposed to determine approximations of optimum designs in the case of correlated measurement errors. The approach is illustrated with an example of the multi-output system of equations describing a chemical kinetic reaction.