Ising图模型概率推理的计算复杂性

图模型概率推理的主要任务是通过对联合概率分布进行变量求和来计算配分函数、变量边缘概率分布、条件 概率分布等。图模型概率推理计算复杂性及近似概率推理的计算复杂性是一重要的理论问题,也是设计概率推理算 法和近似概率推理算法的理论基础。研究了Ising图模型概率推理的计算复杂性,包括概率推理的难解性及不可近似 性。具体地,通过构建#2 SA"I'问题到Icing图模型概率推理问题的多项式时间计数归约,证明在一般 Ising图模型上 计算配分函数、变量边缘概率分布、条件概率分布的概率推理问题是#P难的,同时证明Icing图模型近似概率推理问 题是NP难的,即一般Icing图模型上的概率推理问题是难解且不可近似的。     

基于均值场计算树的Ising图模型消息族传播算法

提出基于均值场计算树的Ising图模型消息族传播算法。首先定义Ising图模型均值场计算树和均值场剪枝计算树概念来描述Ising图模型均值场推理方法的迭代计算过程。然后基于均值场计算树设计Ising图模型消息族传播算法,指出沿着计算树自底向上逐层进行消息族传播,可计算根节点变量的边缘概率分布族。同时证明基于均值场剪枝计算树的消息族传播算法可计算出变量边缘概率分布的界,即此时的边缘概率分布族包括边缘概率分布精确值。最后通过数值实验验证消息族传播算法的有效性和边缘概率分布界的紧致性。     

基于Ising计算树的均值场区间传播算法

基于不完全泛函迭代,设计一个均值场区间传播算法,可给出变量期望界.首先,定义Ising均值场计算树模型来表示Ising均值场迭代计算过程.然后,基于Ising计算树设计均值场区间传播算法,通过在计算树上进行消息区间传播,计算出根变量簇变量期望区间.同时证明在2层计算树上区间传播算法给出的变量期望区间包含期望精确值,即给出变量期望界.最后,通过对比实验验证该算法的有效性和期望界的紧致性.     

高斯均值场变分推理的收敛性和精确性

高斯马尔可夫随机场模型是具有马尔可夫性质、符合多元高斯分布的概率模型.均值场变分方法是图模型最基本的变分近似推理方法.基于指数族变分近似推理框架,分析了高斯马尔可夫随机场模型均值场变分近似推理的收敛性和精确性,证明了均值场变分近似推理关于一阶均值参数是收敛的.进一步给出了模型的各个变量不完全独立时,对数配分函数的最优下界和迭代误差的解析式.最后,通过数值模拟实验,验证了理论分析的结果.     

概率图模型推理方法的研究进展

近年来概率图模型已成为不确定性推理的研究热点,在人工智能、机器学习与计算机视觉等领域有广阔的应用前景.根据网络结构与查询问题类型的不同,系统地综述了概率图模型的推理算法.首先讨论了贝叶斯网络与马尔可夫网络中解决概率查询问题的精确推理算法与近似推理算法,其中主要介绍精确推理中的VE算法、递归约束算法和团树算法,以及近似推理中的变分近似推理和抽样近似推理算法,并给出了解决MAP查询问题的常用推理算法;然后分别针对混合网络的连续与混合情况阐述其推理算法,并分析了暂态网络的精确推理、近似推理以及混合情况下的推理;最后指出了概率图模型推理方法未来的研究方向.     

多Agent动态影响图的一种混合近似推理算法

多Agent动态影响图模型适合于对动态环境中多Agent问题进行建模,Agent之间结构关系被表示成局部的概率因式形式.概率图模型推理所面临的一个主要问题是难以实现近似推理的精度和复杂性之间的均衡.近似推理方法可提高推理精度,但同时也会带来推理精度的损失.BK和粒子滤波(PF)是动态概率模型两种重要的近似推理算法,BK算法有较高的计算效率但会引入较大的误差,PF可以近似任意分布但存在计算的高维问题.结合BK和PF的优点,提出多Agent动态影响图(MADIDs)的一种混合近似推理算法.根据概率图模型的可分解性,将MADIDs分解生成用于推理的原型联合树,混合近似推理算法在规模复杂度较小的团上执行PF推理以达到局部最佳估计,而在其他的团上执行BK推理,为了减小推理误差引入了分割团.仿真实验表明混合近似推理算法是MADIDs模型的一种有效推理方法,与BK和PF算法相比,该算法显著提高了推理精度,且可以实现推理精度和时间复杂性之间的均衡.     

限制树宽的图的最小标记生成数算法

本文研究了图的最小标记生成树问题。首先介绍在一般图上基于搜索树的最小标记生成树的算法;然后考虑了限制树宽的图,得到了效率更高的算法。该算法在树宽为常数的情况下,时间复杂度关于图的顶点个数为多项式,从而也证明了最小标记生成树在限制树宽的图上属于确定参数可解问题。     

3-Set Packing参数化计数问题的复杂性及近似算法

Set Packing参数化计数问题即在一个3-Set Packing实例中统计所有大小为k的不同packing的个数。首先证明了该问题的计算复杂性是#W[1]-难的,表明该问题不大可能存在固定参数可解的精确算法(除非#W[1]=FPT)。然后,通过拓展3-D Matching参数化计数问题的算法对3-Set Packing参数化计数问题提出了一个基于Monte-Carlo自适应覆盖算法和着色技术的随机近似算法。     

概率图模型中的变分近似推理方法

概率图模型将图论和概率论相结合,为多个变量之间复杂依赖关系的表示提供了统一的框架,在计算机视觉、自然语言处理和计算生物学等领域有着广泛的应用.概率推理(包括计算边缘概率和计算最大概率状态等问题)是概率图模型研究及应用的核心问题.本文主要介绍概率图模型近似推理方法中变分推理的最新研究成果.在变分近似推理的框架下,系统地归纳了概率图模型推理问题的基本研究思路,综述了目前主要的近似推理方法,并分析了近似算法的单调性、收敛性和全局性等性质.最后,对概率图模型近似推理方法的研究方向和应用前景作了展望.     

反馈集问题的研究进展

反馈集问题是经典的NP难问题,在电路测试、操作系统解死锁、分析工艺流程、生物计算等领域都有重要应用,按照反馈集中元素类型可分为反馈顶点集(FVS)问题和反馈边集(FAS)问题。人们利用线性规划和局部搜索等技术设计了一系列关于FVS和FAS问题的近似算法,并基于分枝一剪枝策略和加权分治技术提出了FVS问题的精确算法。随着参数计算理论的发展,近年来参数化反馈集问题引起了人们的重视,并取得了很大突破。目前已经证明了无向图和有向图中FVS问题和FAS问题都是固定参数可解的(FPT)。利用树分解、分支搜索、迭代压缩等技术,对无向图FVS问题提出了一系列FPT算法。针对某些特殊的应用,人们开展了对具有特殊性质的图上FVS问题的研究,提出了一些多项式时间可解的精确算法。现首先介绍了在无向图中关于FVS问题的近似算法与精确算法,然后具体分析了FVS问题的参数化算法。进一步阐述了关于有向图和特殊图上FVS问题的研究现状,介绍了FAS问题的研究成果。基于对反馈集问题研究现状的分析,提出了今后FVS问题研究中值得关注的几个方面。     

Latent-space variational bayes

Variational Bayesian Expectation-Maximization (VBEM), an approximate inference method for probabilistic models based on factorizing over latent variables and model parameters, has been a standard technique for practical Bayesian inference. In this paper, we introduce a more general approximate inference framework for conjugate-exponential family models, which we call Latent-Space Variational Bayes (LSVB). In this approach, we integrate out model parameters in an exact way, leaving only the latent variables. It can be shown that the LSVB approach gives better estimates of the model evidence as well as the distribution over latent variables than the VBEM approach, but in practice, the distribution over latent variables has to be approximated. As a practical implementation, we present a First-order LSVB (FoLSVB) algorithm to approximate this distribution over latent variables. From this approximate distribution, one can estimate the model evidence and the posterior over model parameters. The FoLSVB algorithm is directly comparable to the VBEM algorithm and has the same computational complexity. We discuss how LSVB generalizes the recently proposed collapsed variational methods [20] to general conjugate-exponential families. Examples based on mixtures of Gaussians and mixtures of Bernoullis with synthetic and real-world data sets are used to illustrate some advantages of our method over VBEM.     

On Learning Conditional Random Fields for Stereo

Until recently, the lack of ground truth data has hindered the application of discriminative structured prediction techniques to the stereo problem. In this paper we use ground truth data sets that we have recently constructed to explore different model structures and parameter learning techniques. To estimate parameters in Markov random fields (MRFs) via maximum likelihood one usually needs to perform approximate probabilistic inference. Conditional random fields (CRFs) are discriminative versions of traditional MRFs. We explore a number of novel CRF model structures including a CRF for stereo matching with an explicit occlusion model. CRFs require expensive inference steps for each iteration of optimization and inference is particularly slow when there are many discrete states. We explore belief propagation, variational message passing and graph cuts as inference methods during learning and compare with learning via pseudolikelihood. To accelerate approximate inference we have developed a new method called sparse variational message passing which can reduce inference time by an order of magnitude with negligible loss in quality. Learning using sparse variational message passing improves upon previous approaches using graph cuts and allows efficient learning over large data sets when energy functions violate the constraints imposed by graph cuts.     

Parameterized complexity of coloring problems: Treewidth versus vertex cover

We compare the fixed parameter complexity of various variants of coloring problems (including List Coloring, Precoloring Extension, Equitable Coloring, L(p,1)-Labeling and Channel Assignment) when parameterized by treewidth and by vertex cover number. In most (but not all) cases we conclude that parametrization by the vertex cover number provides a significant drop in the complexity of the problems.     

Bayesian retrieval in associative memories with storage errors

It is well known that for finite-sized networks, one-step retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have hitherto only had heuristic justification. We show how they emerge naturally from considerations of probabilistic inference under conditions of noisy and partial input and a corrupted weight matrix. We start from the conditional probability distribution over possible patterns for retrieval. We develop two approximate, but tractable, iterative retrieval methods. One performs maximum likelihood inference to find the single most likely pattern, using the conditional probability as a Lyapunov function for retrieval. The second method makes a mean field assumption to optimize a tractable estimate of the full conditional probability distribution. In the absence of storage errors, both models become very similar to the Willshaw model.     

AND/OR search spaces for graphical models

The paper introduces an AND/OR search space perspective for graphical models that include probabilistic networks (directed or undirected) and constraint networks. In contrast to the traditional (OR) search space view, the AND/OR search tree displays some of the independencies present in the graphical model explicitly and may sometimes reduce the search space exponentially. Indeed, most algorithmic advances in search-based constraint processing and probabilistic inference can be viewed as searching an AND/OR search tree or graph. Familiar parameters such as the depth of a spanning tree, treewidth and pathwidth are shown to play a key role in characterizing the effect of AND/OR search graphs vs. the traditional OR search graphs. We compare memory intensive AND/OR graph search with inference methods, and place various existing algorithms within the AND/OR search space.     

On Making a Distinguished Vertex of Minimum Degree by Vertex Deletion

For directed and undirected graphs, we study how to make a distinguished vertex the unique minimum-(in)degree vertex through deletion of a minimum number of vertices. The corresponding NP-hard optimization problems are motivated by applications concerning control in elections and social network analysis. Continuing previous work for the directed case, we show that the problem is W[2]-hard when parameterized by the graph’s feedback arc set number, whereas it becomes fixed-parameter tractable when combining the parameters “feedback vertex set number” and “number of vertices to delete”. For the so far unstudied undirected case, we show that the problem is NP-hard and W[1]-hard when parameterized by the “number of vertices to delete”. On the positive side, we show fixed-parameter tractability for several parameterizations measuring tree-likeness. In particular, we provide a dynamic programming algorithm for graphs of bounded treewidth and a vertex-linear problem kernel with respect to the parameter “feedback edge set number”. On the contrary, we show a non-existence result concerning polynomial-size problem kernels for the combined parameter “vertex cover number and number of vertices to delete”, implying corresponding non-existence results when replacing vertex cover number by treewidth or feedback vertex set number.     

Bayesian estimation of beta mixture models with variational inference

Bayesian estimation of the parameters in beta mixture models (BMM) is analytically intractable. The numerical solutions to simulate the posterior distribution are available, but incur high computational cost. In this paper, we introduce an approximation to the prior/posterior distribution of the parameters in the beta distribution and propose an analytically tractable (closed form) Bayesian approach to the parameter estimation. The approach is based on the variational inference (VI) framework. Following the principles of the VI framework and utilizing the relative convexity bound, the extended factorized approximation method is applied to approximate the distribution of the parameters in BMM. In a fully Bayesian model where all of the parameters of the BMM are considered as variables and assigned proper distributions, our approach can asymptotically find the optimal estimate of the parameters posterior distribution. Also, the model complexity can be determined based on the data. The closed-form solution is proposed so that no iterative numerical calculation is required. Meanwhile, our approach avoids the drawback of overfitting in the conventional expectation maximization algorithm. The good performance of this approach is verified by experiments with both synthetic and real data.