Gelfand-Yaglom-Perez theorem for generalized relative entropy functionals

The measure-theoretic definition of Kullback-Leibler relative-entropy (or simply KL-entropy) plays a basic role in defining various classical information measures on general spaces. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic definitions are key to extending the ergodic theorems of information theory to non-discrete cases. A fundamental theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem [M.S. Pinsker, Information and Information Stability of Random Variables and Process, 1960, Holden-Day, San Francisco, CA (English ed., 1964, translated and edited by Amiel Feinstein), Theorem. 2.4.2] which states that measure-theoretic relative-entropy equals the supremum of relative-entropies over all measurable partitions. This paper states and proves the GYP-theorem for Rényi relative-entropy of order greater than one. Consequently, the result can be easily extended to Tsallis relative-entropy.     

Inferencia bayesiana en mixturas: metodos aproximados

Resumen últimamente se ha dedicado una gran atención a las técnicas de aproximación de mixturas de distribuciones. En este trabajo se consideran problemas de estimación de los parámetros de mezcla en una mixtura finita, desde una metodología bayesiana, que conducen a problemas de aproximación de mixturas finitas y se proponen dos nuevos métodos de aproximación. Bajo ciertas condiciones se demuestra que ambos métodos son asintóticamente equivalentes a un tercer método, de aplicación mucho más sencilla. El trabajo se concluye con un estudio de simulación en el que se analiza la bondad de los métodos de aproximación que aquí se exponen.      

Distributions most nearly compatible with given families of conditional distributions

Consider a discrete bivariate random variable (X, Y) with possible values 1, 2, ...,I forX and 1, 2, ...J forY. Suppose that putative families of conditional distributions, forX given values ofY and ofY given values ofX, are available. After reviewing conditions for compatibiity of such conditional specifications of the distribution of (X, Y), attention is focussed on the incompatible case. The Kullback-Leibler information function is shown to provide a convenient measure of inconsistency. Using it, algorithms are provided for computing the joint distribution for (X, Y) that is least discrepant from the given inconsistent conditional specifications. Other discrepancy measures are briefly discussed.     

基于对称KL距离的自适应粒子滤波算法

针对粒子滤波算法中粒子数自适应的问题,提出了一种新的算法.将当前滤波时刻的粒子随机划分为粒子数相同的两个粒子群,并采用对称KL距离方法计算他们之间的信息距离,然后根据信息距离的大小决定增加或者减少下一时刻参与滤波的粒子数,从而实现了滤波过程中粒子数目的自适应.该方法在确保一定滤波精度的基础上,能够减少滤波过程中需要的粒子数,为降低粒子滤波算法的时间复杂度提供了新的途径.仿真结果表明了算法的有效性.     

融合复小波特征和局部二值模式的纹理聚类

为了解决图像检索中聚类问题,对图像作双树旋转复小波变换,再对变换后的系数通过广义高斯模型建模后,计算Kullback-Leibler距离;对图像采用局部二值模式,计算图像之间的对数似然距离.将这2种距离采用事先计算得到的加权因子进行融合得到新的距离.在此距离的基础上构建邻接矩阵,对邻接矩阵采用谱聚类的方法进行聚类运算.实验证明,由于双树旋转复小波变换和局部二值模式之间存在互补性,在聚类过程中将2种特征距离结合起来,能够有效地提高聚类的正确性.     

Bayesian projection approaches to variable selection in generalized linear models

A Bayesian approach to variable selection which is based on the expected Kullback-Leibler divergence between the full model and its projection onto a submodel has recently been suggested in the literature. For generalized linear models an extension of this idea is proposed by considering projections onto subspaces defined via some form of L₁ constraint on the parameter in the full model. This leads to Bayesian model selection approaches related to the lasso. In the posterior distribution of the projection there is positive probability that some components are exactly zero and the posterior distribution on the model space induced by the projection allows exploration of model uncertainty. Use of the approach in structured variable selection problems such as ANOVA models is also considered, where it is desired to incorporate main effects in the presence of interactions. Projections related to the non-negative garotte are able to respect the hierarchical constraints. A consistency result is given concerning the posterior distribution on the model induced by the projection, showing that for some projections related to the adaptive lasso and non-negative garotte the posterior distribution concentrates on the true model asymptotically.     

基于Kullback-Leibler分歧的变分滤波WSNs贝叶斯移动定位跟踪西

为了在降低资源能耗和带宽占用情况下,提高无线传感器网络WSNs移动目标定位跟踪的精度,提出了基于Kullback-Leibler分歧的变分滤波的WSNs贝叶斯移动目标定位跟踪算法。首先,利用高斯和Wishart分布在不考虑速度限制和方向移动限制情况下,构建WSNs移动定位的贝叶斯状态演化模型,并基于路径损耗模型构建移动目标定位的观测模型;其次,利用Kullback-Leibler分歧构建变分滤波的误差计算模型,通过周围激活节点实现移动节点目标的位置估计,设计了递归概率计算过程综合预测和更新两个过程,并实现了定位和目标跟踪的同步化;最后,通过仿真验证了所提模型在跟踪精度和资源节约上的优势。     

An alternative objective function for fitting regression trees to functional response variables

Analyses of systems that can be represented by functional responses are becoming common in many scientific disciplines. Functional regression trees (FRT) provide a methodology for modelling such systems. Recent work has focused on fitting models where the response variable is a probability density function, using a splitting criterion that is based on the sum of dissimilarities between the densities. We suggest a different criterion based on deviations of the densities from their mean. We provide motivation and justification for this criterion, and demonstrate its superior performance using an extensive simulation exercise. We discuss the computational aspects of the FRT procedure and show that substantial speed gains can be made through use of a dissimilarity matrix. Our results show that the proposed splitting criterion outperforms both the original and a splitting criterion based on Euclidean distance. Pointwise standard error curves for a predicted functional response can be generated through the fitting procedure, which we demonstrate in a case study with a forestry data set. Supplementary materials are available.     

Automatic transaction of signal via statistical modeling

The statistical information processing can be characterized by the likelihood function defined by giving an explicit form for an approximation to the true distribution. This mathematical representation, which is usually called a model, is built based on not only the current data but also prior knowledge on the object and the objective of the analysis. Akaike^2,3) showed that the log-likelihood can be considered as an estimate of the Kullback-Leibler (K-L) information which measures the similarity between the predictive distribution of the model and the true distribution. Akaike information criterion (AIC) is an estimate of the K-L information and makes it possible to evaluate and compare the goodness of many models objectively. In consequence, the minimum AIC procedure allows us to develop automatic modeling and signal extraction procedures. In this article, we give a simple explanation of statistical modeling based on the AIC and demonstrate four examples of applying the minimum AIC procedure to an automatic transaction of signals observed in the earth sciences. Genshiro, Kitagawa, Ph.D.: He is a Professor in the Department of Prediction and Control at the Institute of Statistical Mathematics. He is currently Deputy Director of the Institute of Statistical Mathematics and Professor of Statistical Science at the Graduate University for Advanced Study. He obtained his Ph.D. from the Kyushu University in 1983. His primary research interests are in time series analysis, non-Gaussian nonlinear filtering, and statistical modeling. He has published over 50 research papers. He was awarded the 2nd Japan Statistical Society Prize in 1997. Tomoyuki Higuchi, Ph.D.: He is an Associate Professor in the Department of Prediction and Control at the Institute of Statistical Mathematics. He is currently an Associate Professor of Statistical Science at the Graduate University for Advanced Study. He obtained his Ph.D. from the University of Tokyo in 1989. His research interests are in statistical modeling of space-time data, stochastic optimization techniques, and data mining. He has published over 30 research papers.     

改进的TOPSIS模型在地下水水质评价中的应用

地下水水质评价是保护水环境实现地下水资源可持续利用的重要基础工作。为了克服使用TOPSIS模型评价水质时常受到多指标决策的局限性,对原模型进行了两点改进:一是采用以主观赋权(AHP法)与客观赋权(熵权法)相结合确定组合权重;二是采用Kullback-Leibler距离代替传统方法中的欧式距离计算贴近度。利用改进后的TOPSIS模型,分别对邯郸市、焦作市地下水水质进行评价,评价结果分别与参考文献对比基本一致,表明改进后的TOPSIS模型,计算方法基本合理可行,具有一定的探索价值。