基于结构分析的局部Gibbs抽样自动推理算法

提出一种基于结构分析的局部Gibbs抽样的贝叶斯网络推理算法(S-LGSI).S-LGSI算法基于联合树算法的概率图模型分析思想,对贝叶斯网络进行精确分解,然后根据查询结点和证据结点生成具有强相关性的局部网络模型,进而对局部网络模型进行Gibbs抽样推理.与当前基于抽样的其它近似推理算法相比,该算法降低推理的计算维数.同时,由于局部抽样模型包含了与查询结点相关的重要信息,因此该算法保证局部抽样推理的精度.算法分析和在Alarm网的实验结果表明,S-LGSI算法较显著降低时间复杂度,同时也提高推理精度.S-LGSI算法应用于上海证券交易所股票网络的推理结果与实际情况基本一致,表现出较强的实用性.     

多Agent动态影响图的一种混合近似推理算法

多Agent动态影响图模型适合于对动态环境中多Agent问题进行建模,Agent之间结构关系被表示成局部的概率因式形式.概率图模型推理所面临的一个主要问题是难以实现近似推理的精度和复杂性之间的均衡.近似推理方法可提高推理精度,但同时也会带来推理精度的损失.BK和粒子滤波(PF)是动态概率模型两种重要的近似推理算法,BK算法有较高的计算效率但会引入较大的误差,PF可以近似任意分布但存在计算的高维问题.结合BK和PF的优点,提出多Agent动态影响图(MADIDs)的一种混合近似推理算法.根据概率图模型的可分解性,将MADIDs分解生成用于推理的原型联合树,混合近似推理算法在规模复杂度较小的团上执行PF推理以达到局部最佳估计,而在其他的团上执行BK推理,为了减小推理误差引入了分割团.仿真实验表明混合近似推理算法是MADIDs模型的一种有效推理方法,与BK和PF算法相比,该算法显著提高了推理精度,且可以实现推理精度和时间复杂性之间的均衡.     

一种基于近似因子的在线概率知识库推理方法

概率知识库中的推理技术是近年来的研究热点。目前大多数系统的推理主要基于批处理的方式实现,并不适用于在线查询场景。针对此本文提出了一种基于近似因子的在线概率知识库推理方法,它可重复利用已推断结果计算查询变量的边缘概率。该算法首先提取查询变量的子图（含已推断变量）;接着在此子图上添加近似因子以模拟子图外其余变量的影响;最后采用团树算法推断查询变量的边缘概率。实验结果表明,相较于已有算法,本文算法可在时间和精度上取得较好的权衡。.     

概率图模型学习技术研究进展

概率图模型能有效处理不确定性推理,从样本数据中准确高效地学习概率图模型是其在实际应用中的关键问题.概率图模型的表示由参数和结构两部分组成,其学习算法也相应分为参数学习与结构学习.本文详细介绍了基于概率图模型网络的参数学习与结构学习算法,并根据数据集是否完备而分别讨论各种情况下的参数学习算法,还针对结构学习算法特点的不同把结构学习算法归纳为基于约束的学习、基于评分搜索的学习、混合学习、动态规划结构学习、模型平均结构学习和不完备数据集的结构学习.并总结了马尔科夫网络的参数学习与结构学习算法.最后指出了概率图模型学习的开放性问题以及进一步的研究方向.     

Ising图模型概率推理的计算复杂性

图模型概率推理的主要任务是通过对联合概率分布进行变量求和来计算配分函数、变量边缘概率分布、条件 概率分布等。图模型概率推理计算复杂性及近似概率推理的计算复杂性是一重要的理论问题,也是设计概率推理算 法和近似概率推理算法的理论基础。研究了Ising图模型概率推理的计算复杂性,包括概率推理的难解性及不可近似 性。具体地,通过构建#2 SA"I'问题到Icing图模型概率推理问题的多项式时间计数归约,证明在一般 Ising图模型上 计算配分函数、变量边缘概率分布、条件概率分布的概率推理问题是#P难的,同时证明Icing图模型近似概率推理问 题是NP难的,即一般Icing图模型上的概率推理问题是难解且不可近似的。     

基于概率图的三支决策模型研究

概率图模型是一类用图形模式表达基于概率关系的模型的总称,用该模型解决损失代价问题已成为当前的研究热点。结合概率图和三支决策理论,提出了基于概率图的三支决策模型。该模型通过对数据进行分析,构造其Bayes网络;并根据模型中节点的相互依赖关系,计算出条件概率分布函数;结合查询变量的先验概率和三支决策损失代价函数,建立了相应的决策规则,给出了概率推理决策中代价最小化问题的一种解决方法。最后通过教学评估实例验证了该模型的有效性。     

概率图模型中的变分近似推理方法

概率图模型将图论和概率论相结合,为多个变量之间复杂依赖关系的表示提供了统一的框架,在计算机视觉、自然语言处理和计算生物学等领域有着广泛的应用.概率推理(包括计算边缘概率和计算最大概率状态等问题)是概率图模型研究及应用的核心问题.本文主要介绍概率图模型近似推理方法中变分推理的最新研究成果.在变分近似推理的框架下,系统地归纳了概率图模型推理问题的基本研究思路,综述了目前主要的近似推理方法,并分析了近似算法的单调性、收敛性和全局性等性质.最后,对概率图模型近似推理方法的研究方向和应用前景作了展望.     

基于Markov毯分解的抽样近似推理算法

现有的贝叶斯推理算法不同程度地存在推理精度低或推理时间长的问题。文中提出一种基于Markov毯分解的抽样近似推理算法(LSIA-MB)。LSIA-MB算法利用HITON_MB算法寻找查询结点的Markov毯, 进而利用动态规划方法学习边的后验概率, 确定变量之间的因果关系, 获得一个关于查询结点的Markov局部网络模型。最后, 在Markov局部模型上执行Gibbs Sampling。通过对Markov局部模型的抽样, 极大降低推理的计算维数。同时, 由于Markov局部网络模型包含与目标结点相关的完整信息, 从而保证局部抽样推理的精度。算法分析和在标准Alarm网的实验结果均表明, LSIA-MB算法降低推理时间, 且提高推理精度。LSIA-MB算法在上海股票交易网络上的推理预测结果显示出较强的实用性。     

概率图模型研究进展综述

概率图模型作为一类有力的工具,能够简洁地表示复杂的概率分布,有效地(近似)计算边缘分布和条件分布,方便地学习概率模型中的参数和超参数.因此,它作为一种处理不确定性的形式化方法,被广泛应用于需要进行自动的概率推理的场合,例如计算机视觉、自然语言处理.回顾了有关概率图模型的表示、推理和学习的基本概念和主要结果,并详细介绍了这些方法在两种重要的概率模型中的应用.还回顾了在加速经典近似推理算法方面的新进展.最后讨论了相关方向的研究前景.     

Ising图模型概率推理的参数化复杂性

Ising图模型概率推理的主要工作是通过变量求和来计算配分函数和边缘概率分布。传统计算复杂性理论证明Ising图模型精确概率推理是NP难的,并且Ising图模型近似概率推理是NP难的。研究了Ising图模型精确概率推理和Ising均值场近似概率推理的参数化复杂性。首先证明了不同参数的Ising图模型概率推理的参数化复杂性定理,指出基于变量个数或图模型树宽的参数化概率推理问题是固定参数可处理的。然后证明了Ising均值场的参数化复杂性定理,指出基于自由分布树宽、迭代次数和变量个数的参数化Icing均值场是固定参数可处理的;进一步,当Ising图模型参数满足Ising均值场迭代式压缩条件时,基于自由分布树宽和迭代次数的参数化Ising均值场是固定参数可处理的。     

PrDB: managing and exploiting rich correlations in probabilistic databases

Due to numerous applications producing noisy data, e.g., sensor data, experimental data, data from uncurated sources, information extraction, etc., there has been a surge of interest in the development of probabilistic databases. Most probabilistic database models proposed to date, however, fail to meet the challenges of real-world applications on two counts: (1) they often restrict the kinds of uncertainty that the user can represent; and (2) the query processing algorithms often cannot scale up to the needs of the application. In this work, we define a probabilistic database model, PrDB, that uses graphical models, a state-of-the-art probabilistic modeling technique developed within the statistics and machine learning community, to model uncertain data. We show how this results in a rich, complex yet compact probabilistic database model, which can capture the commonly occurring uncertainty models (tuple uncertainty, attribute uncertainty), more complex models (correlated tuples and attributes) and allows compact representation (shared and schema-level correlations). In addition, we show how query evaluation in PrDB translates into inference in an appropriately augmented graphical model. This allows us to easily use any of a myriad of exact and approximate inference algorithms developed within the graphical modeling community. While probabilistic inference provides a generic approach to solving queries, we show how the use of shared correlations, together with a novel inference algorithm that we developed based on bisimulation, can speed query processing significantly. We present a comprehensive experimental evaluation of the proposed techniques and show that even with a few shared correlations, significant speedups are possible.     

Approximate inference for dynamic Bayesian networks: sliding window approach

Dynamic Bayesian networks (DBNs) are probabilistic graphical models that have become a ubiquitous tool for compactly describing statistical relationships among a group of stochastic processes. A suite of elaborately designed inference algorithms makes it possible for intelligent systems to use a DBN to make inferences in uncertain conditions. Unfortunately, exact inference or even approximation in a DBN has been proved to be NP-hard and is generally computationally prohibitive. In this paper, we investigate a sliding window framework for approximate inference in DBNs to reduce the computational burden. By introducing a sliding window that moves forward as time progresses, inference at any time is restricted to a quite narrow region of the network. The main contributions to the sliding window framework include an exploration of its foundations, explication of how it operates, and the proposal of two strategies for adaptive window size selection. To make this framework available as an inference engine, the interface algorithm widely used in exact inference is then integrated with the framework for approximate inference in DBNs. After analyzing its computational complexity, further empirical work is presented to demonstrate the validity of the proposed algorithms.     

An Introduction to Variational Methods for Graphical Models

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.     

Representing and processing lineages over uncertain data based on the Bayesian network

Processing lineages (also called provenances) over uncertain data consists in tracing the origin of uncertainty based on the process of data production and evolution. In this paper, we focus on the representation and processing of lineages over uncertain data, where we adopt Bayesian network (BN), one of the popular and important probabilistic graphical models (PGMs), as the framework of uncertainty representation and inferences. Starting from the lineage expressed as Boolean formulae for SPJ (Selection–Projection–Join) queries over uncertain data, we propose a method to transform the lineage expression into directed acyclic graphs (DAGs) equivalently. Specifically, we discuss the corresponding probabilistic semantics and properties to guarantee that the graphical model can support effective probabilistic inferences in lineage processing theoretically. Then, we propose the function-based method to compute the conditional probability table (CPT) for each node in the DAG. The BN for representing lineage expressions over uncertain data, called lineage BN and abbreviated as LBN, can be constructed while generally suitable for both safe and unsafe query plans. Therefore, we give the variable-elimination-based algorithm for LBN's exact inferences to obtain the probabilities of query results, called LBN-based query processing. Then, we focus on obtaining the probabilities of inputs or intermediate tuples conditioned on query results, called LBN-based inference query processing, and give the Gibbs-sampling-based algorithm for LBN's approximate inferences. Experimental results show the efficiency and effectiveness of our methods.     

Credal networks

This paper presents a complete theory of credal networks, structures that associate convex sets of probability measures with directed acyclic graphs. Credal networks are graphical models for precise/imprecise beliefs. The main contribution of this work is a theory of credal networks that displays as much flexibility and representational power as the theory of standard Bayesian networks. Results in this paper show how to express judgements of irrelevance and independence, and how to compute inferences in credal networks. A credal network admits several extensions—several sets of probability measures comply with the constraints represented by a network. Two types of extensions are investigated. The properties of strong extensions are clarified through a new generalization of d-separation, and exact and approximate inference methods are described for strong extensions. Novel results are presented for natural extensions, and linear fractional programming methods are described for natural extensions. The paper also investigates credal networks that are defined globally through perturbations of a single network.     

对弧进行删除的Bayes网络近似方法

弧的删除是一种对Bayes网络模型进行近似的方法。文中以Kullback-Leibler偏差作为近似网络和原网络概率分布误差的测度,给出了近似网络在此测度意义下的最优参数。同时,也给出了通过对原网络删除多条弧进行近似的启发式算法,当给定一个误差上界时,可以使用此算法寻找满足误差要求的近似网络。