一种基于近似因子的在线概率知识库推理方法

概率知识库中的推理技术是近年来的研究热点。目前大多数系统的推理主要基于批处理的方式实现,并不适用于在线查询场景。针对此本文提出了一种基于近似因子的在线概率知识库推理方法,它可重复利用已推断结果计算查询变量的边缘概率。该算法首先提取查询变量的子图（含已推断变量）;接着在此子图上添加近似因子以模拟子图外其余变量的影响;最后采用团树算法推断查询变量的边缘概率。实验结果表明,相较于已有算法,本文算法可在时间和精度上取得较好的权衡。.     

概率生成模型变分推理方法综述

概率生成模型是知识表示的重要方法,在该模型上计算似然函数的概率推理问题一般是难解的.变分推理是重要的确定性近似推理方法,具有较快的收敛速度、坚实的理论基础．尤其随着大数据时代的到来,概率生成模型变分推理方法受到工业界和学术界的极大关注.综述了多种概率生成模型变分推理框架及最新进展,具体包括：首先综述了概率生成模型变分推理一般框架及基于变分推理的生成模型参数学习过程;然后对于条件共轭指数族分布,给出了具有解析优化式的变分推理框架及该框架下可扩展的随机化变分推理;进一步,对于一般概率分布,给出了基于随机梯度的黑盒变分推理框架,并简述了该框架下多种变分推理算法的具体实现;最后分析了结构化变分推理,通过不同方式丰富变分分布提高推理精度并改善近似推理一致性.此外,展望了概率生成模型变分推理的发展趋势.     

支持短时交通流量预测的概率图模型构建与推理

短时交通流量预测,是交通系统信息化和智能化交通运输管理技术领域研究的关键问题.目前的方法对历史数据具有较高的依赖程度,或者具有较高的计算成本,或者不能有效反映实际中较复杂的交通网络及各结点之间的相互关系、以及依赖的不确定性,或者多种模型的组合使得预测方法较复杂.贝叶斯网是一种重要的概率图模型,本文以交通网络结构为基础,利用概率图模型在不确定性知识表示和推理方面的良好性质,考虑路口交通流量及其预测的时序依赖特征,构建了带有时序条件依赖关系的交通贝叶斯网.进而针对短时交通流量预测的实时性和高效性要求,提出了基于Gibbs采样的交通贝叶斯网近似概率推理算法,并进行交通流量的短时预测.实验结果表明,本文提出的交通贝叶斯网构建、近似推理以及相应的短时交通流量的预测方法,具有高效性、准确性和可用性.     

概率图模型推理方法的研究进展

近年来概率图模型已成为不确定性推理的研究热点,在人工智能、机器学习与计算机视觉等领域有广阔的应用前景.根据网络结构与查询问题类型的不同,系统地综述了概率图模型的推理算法.首先讨论了贝叶斯网络与马尔可夫网络中解决概率查询问题的精确推理算法与近似推理算法,其中主要介绍精确推理中的VE算法、递归约束算法和团树算法,以及近似推理中的变分近似推理和抽样近似推理算法,并给出了解决MAP查询问题的常用推理算法;然后分别针对混合网络的连续与混合情况阐述其推理算法,并分析了暂态网络的精确推理、近似推理以及混合情况下的推理;最后指出了概率图模型推理方法未来的研究方向.     

概率图模型研究进展综述

概率图模型作为一类有力的工具,能够简洁地表示复杂的概率分布,有效地(近似)计算边缘分布和条件分布,方便地学习概率模型中的参数和超参数.因此,它作为一种处理不确定性的形式化方法,被广泛应用于需要进行自动的概率推理的场合,例如计算机视觉、自然语言处理.回顾了有关概率图模型的表示、推理和学习的基本概念和主要结果,并详细介绍了这些方法在两种重要的概率模型中的应用.还回顾了在加速经典近似推理算法方面的新进展.最后讨论了相关方向的研究前景.     

概率图模型表示理论

概率图模型结合概率论与图论的知识,利用图来表示与模型有关的变量的联合概率分布。近年它已成为不确定性推理的研究热点,在人工智能、机器学习和计算机视觉等领域有广阔的应用前景。主要研究概率图模型的表示方法,讨论如何利用概率网络中的独立性来简化联合概率分布的方法表示。首先介绍了单个节点上的条件概率分布的表示模型及其引起的独立性,包括表格CPD、确定性CPD、特定上下文CPD、因果影响CPD、高斯模型和混合模型,并把单个分布模型推广到指数分布族中。然后详细介绍贝叶斯网络中的独立性以及图与概率分布的关系,讨论了高斯分布和指数分布族的贝叶斯网络表示理论。再详细描述马尔可夫网络的参数化问题及其独立性,也讨论高斯分布和指数分布族的马尔可夫网络表示理论。还给出两种局部有向图模型:条件随机场和链图。并且描述基于模板的概率模型表示,包括动态贝叶斯网络和状态观测模型这两种暂态模型,以及盘模型和概率关系模型这两种对象关系领域的有向概率模型,而且给出对象关系领域的无向表示。最后对概率图模型表示理论和方法所面临的问题及前景进行展望。     

异构信息网络数据上的融合概率图模型

针对异构信息网络中存在多种数据目标类型,以及多种数据连接关系的问题,将多个主题模型和马尔科夫逻辑网相结合,提出了一种融合概率图模型。多个主题模型可以分别描述不同数据目标子空间的主题分布,实现对多种数据目标的预处理。用一阶逻辑子句描述的连接规则构建马尔科夫逻辑网,将每个主题模型中的不同数据目标连接起来。通过使用Gibbs采样,可以对异构网络进行参数学习和推理。在国际通用的异构信息网络DBLP数据集上的实验结果表明,使用融合概率图模型能够更好地表示不同的数据目标和连接关系。实验对比了4种典型的分类方法,多次采样得到的分类结果稳定,对作者、文章和会议取得了较好的分类结果。     

复小波域混合概率图模型的超声医学图像分割

针对存在大量不规则斑点噪声、目标边缘弱化的超声医学图像分割中较难识别目标的问题, 提出了一种复小波域中混合概率图模型的超声医学图像分割算法.采用具有近似平移不变性和良好方向选择性的双树复小波变换(Dual tree-complex wavelet transform, DT-CWT)提取超声医学图像6个方向的高频特征信息; 其次, 为关联目标的弱特征信息并抑制统计独立的高频噪声, 构建了复小波域混合概率图模型; 尺度间"父—子"节点间标记采用贝叶斯网络进行建模, 尺度内邻域间标记采用马尔科夫随机场(Markov random field, MRF)无向图建模, 对复小波域中同尺度的特征系数采用高斯混合模型建模, 尺度内同标记的观测特征采用高斯模型建模; 最后, 用迭代条件模式(Iterated conditional mode, ICM)实现MRF中误分割率最小的能量函数最优解, 获取标记场, 实现超声医学图像分割.实验结果从视觉效果和定量分析两方面验证表明, 本文算法能有效地提取超声图像的弱目标信息, 较好地定位目标区域, 具有较高的分割精度和鲁棒性.     

Ising图模型概率推理的计算复杂性

图模型概率推理的主要任务是通过对联合概率分布进行变量求和来计算配分函数、变量边缘概率分布、条件 概率分布等。图模型概率推理计算复杂性及近似概率推理的计算复杂性是一重要的理论问题,也是设计概率推理算 法和近似概率推理算法的理论基础。研究了Ising图模型概率推理的计算复杂性,包括概率推理的难解性及不可近似 性。具体地,通过构建#2 SA"I'问题到Icing图模型概率推理问题的多项式时间计数归约,证明在一般 Ising图模型上 计算配分函数、变量边缘概率分布、条件概率分布的概率推理问题是#P难的,同时证明Icing图模型近似概率推理问 题是NP难的,即一般Icing图模型上的概率推理问题是难解且不可近似的。     

概率图模型学习技术研究进展

概率图模型能有效处理不确定性推理,从样本数据中准确高效地学习概率图模型是其在实际应用中的关键问题.概率图模型的表示由参数和结构两部分组成,其学习算法也相应分为参数学习与结构学习.本文详细介绍了基于概率图模型网络的参数学习与结构学习算法,并根据数据集是否完备而分别讨论各种情况下的参数学习算法,还针对结构学习算法特点的不同把结构学习算法归纳为基于约束的学习、基于评分搜索的学习、混合学习、动态规划结构学习、模型平均结构学习和不完备数据集的结构学习.并总结了马尔科夫网络的参数学习与结构学习算法.最后指出了概率图模型学习的开放性问题以及进一步的研究方向.     

An Introduction to Variational Methods for Graphical Models

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.     

Models and Methods of Finite-Dimensional Variational Inequalities

Algorithms for solving finite-dimensional inequalities are studied. The emphasis is on numerical methods based on the optimization approach. Examples of economic equilibrium models that assist in solving variational inequalities are presented.     

Graphical Inference for Multiple Intrusion Detection

In this paper, we consider vulnerabilities of networked systems and develop a multiple intrusion detection system (MIDS) which operates by running belief propagation on an appropriately constructed weighted bipartite graph. In this bipartite graph, one set of nodes represents the different types of intrusions that are possible, the other set of nodes represents the set of significant measures that are available, and the (weighted) connections represent the dependence of a certain measure on a particular type of intrusion. We assume that the effect of each active intrusion on a particular significant measure is superimposed on the normal operation of that measure; thus, we are able to obtain a complete representation of the overall bipartite graph model by superimposing the simpler graphs associated with each individual intrusion. The key ingredient of our MIDS is the development of a modified belief propagation max-product algorithm (MPA) that avoids the exponential complexity of the original MPA by limiting, during the iteration process, the number of active intrusions that are connected to a particular measure. Our simulation results indicate that the proposed MIDS performs well in detecting both single and multiple intrusions with a very low false alarm rate.      

An Approximate Numerical Methods for Mathematical and Physical Studies for Covid-19 Models

The advancement in numerical models of serious resistant illnesses is a key research territory in different fields including the nature and the study of disease transmission. One of the aims of these models is to comprehend the elements of conduction of these infections. For the new strain of Covid-19 (Coronavirus), there has been no immunization to protect individuals from the virus and to forestall its spread so far. All things being equal, control procedures related to medical services, for example, social distancing or separation, isolation, and travel limitations can be adjusted to control this pandemic. This article reveals some insights into the dynamic practices of nonlinear Coronavirus models dependent on the homotopy annoyance strategy (HPM). We summon a novel sign stream chart that is utilized to depict the Coronavirus model. Through the numerical investigations, it is uncovered that social separation of the possibly tainted people who might be conveying the infection and the healthy virus-free people can diminish or interrupt the spread of the infection. The mathematical simulation results are highly concurrent with the statistical forecasts. The free balance and dependability focus for the Coronavirus model is discussed and the presence of a consistently steady arrangement is demonstrated.     

Tuning the Learning Rate for Stochastic Variational Inference

Stochastic variational inference (SVI) can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback–Leibler (KL) divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models: latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.     

Towards Autoencoding Variational Inference for Aspect-Based Opinion Summary

Aspect-based Opinion Summary (AOS), consisting of aspect discovery and sentiment classification steps, has recently been emerging as one of the most crucial data mining tasks in e-commerce systems. Along this direction, the LDA-based model is considered as a notably suitable approach, since this model offers both topic modeling and sentiment classification. However, unlike traditional topic modeling, in the context of aspect discovery, it is often required some initial seed words, whose prior knowledge is not easy to be incorporated into LDA models. Moreover, LDA approaches rely on sampling methods, which need to load the whole corpus into memory, making them hardly scalable. In this research, we study an alternative approach for AOS problem, based on Autoencoding Variational Inference (AVI). Firstly, we introduce the Autoencoding Variational Inference for Aspect Discovery (AVIAD) model, which extends the previous work of Autoencoding Variational Inference for Topic Models (AVITM) to embed prior knowledge of seed words. This work includes enhancement of the previous AVI architecture and also modification of the loss function. Ultimately, we present the Autoencoding Variational Inference for Joint Sentiment/Topic (AVIJST) model. In this model, we substantially extend the AVI model to support the JST model, which performs topic modeling for corresponding sentiment. The experimental results show that our proposed models enjoy higher topic coherent, faster convergence time and better accuracy on sentiment classification, as compared to their LDA-based counterparts.     

Graphical Methods,Inductive Causal Inference,and Econometrics: A Literature Review

Recent work with graphical methods for inductive causal inference with observational econometric data is reviewed and compared with earlier work. Two alternative algorithms are described. Caveats on applications are discussed.     

一种变分优化问题的近似解法

针对传统变分法求解困难的问题,提出一种变分优化问题的近似解法。根据最小二乘近似解法的简便性与粒子群优化算法参数少的特性,在最小二乘近似解法的求解过程中引入粒子群优化算法,并给出求解流程。数值仿真实验结果表明,该算法计算过程简单,优化效果较好。     

Variational Methods for Normal Integration

The need for an efficient method of integration of a dense normal field is inspired by several computer vision tasks, such as shape-from-shading, photometric stereo, deflectometry. Inspired by edge-preserving methods from image processing, we study in this paper several variational approaches for normal integration, with a focus on non-rectangular domains, free boundary and depth discontinuities. We first introduce a new discretization for quadratic integration, which is designed to ensure both fast recovery and the ability to handle non-rectangular domains with a free boundary. Yet, with this solver, discontinuous surfaces can be handled only if the scene is first segmented into pieces without discontinuity. Hence, we then discuss several discontinuity-preserving strategies. Those inspired, respectively, by the Mumford–Shah segmentation method and by anisotropic diffusion, are shown to be the most effective for recovering discontinuities.