Understanding the scalability of Bayesian network inference using clique tree growth curves

One of the main approaches to performing computation in Bayesian networks (BNs) is clique tree clustering and propagation. The clique tree approach consists of propagation in a clique tree compiled from a BN, and while it was introduced in the 1980s, there is still a lack of understanding of how clique tree computation time depends on variations in BN size and structure. In this article, we improve this understanding by developing an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from BNs, specifically: (i) the ratio of the number of a BN's non-root nodes to the number of root nodes, and (ii) the expected number of moral edges in their moral graphs. Analytically, we partition the set of cliques in a clique tree into different sets, and introduce a growth curve for the total size of each set. For the special case of bipartite BNs, there are two sets and two growth curves, a mixed clique growth curve and a root clique growth curve. In experiments, where random bipartite BNs generated using the BPART algorithm are studied, we systematically increase the out-degree of the root nodes in bipartite Bayesian networks, by increasing the number of leaf nodes. Surprisingly, root clique growth is well-approximated by Gompertz growth curves, an S-shaped family of curves that has previously been used to describe growth processes in biology, medicine, and neuroscience. We believe that this research improves the understanding of the scaling behavior of clique tree clustering for a certain class of Bayesian networks; presents an aid for trade-off studies of clique tree clustering using growth curves; and ultimately provides a foundation for benchmarking and developing improved BN inference and machine learning algorithms.     

LEARNING STRUCTURED BAYESIAN NETWORKS: COMBINING ABSTRACTION HIERARCHIES AND TREE-STRUCTURED CONDITIONAL PROBABILITY TABLES

Context-specific independence representations, such as tree-structured conditional probability distributions, capture local independence relationships among the random variables in a Bayesian network (BN). Local independence relationships among the random variables can also be captured by using attribute-value hierarchies to find an appropriate abstraction level for the values used to describe the conditional probability distributions. Capturing this local structure is important because it reduces the number of parameters required to represent the distribution. This can lead to more robust parameter estimation and structure selection, more efficient inference algorithms, and more interpretable models. In this paper, we introduce Tree-Abstraction-Based Search (TABS), an approach for learning a data distribution by inducing the graph structure and parameters of a BN from training data. TABS combines tree structure and attribute-value hierarchies to compactly represent conditional probability tables. To construct the attribute-value hierarchies, we investigate two data-driven techniques: a global clustering method, which uses all of the training data to build the attribute-value hierarchies, and can be performed as a preprocessing step; and a local clustering method, which uses only the local network structure to learn attribute-value hierarchies. We present empirical results for three real-world domains, finding that (1) combining tree structure and attribute-value hierarchies improves the accuracy of generalization, while providing a significant reduction in the number of parameters in the learned networks, and (2) data-derived hierarchies perform as well or better than expert-provided hierarchies.     

一种用于信度网推理的高效三角化算法

信度网是不确定性知识表达和推理的有力工具。信度网的精确推理是NPC问题,计算的主要困难在于将信度网三角化并构造一棵最小权的join tree上。此项研究提出了一种新的三角化算法MsLB-Triang,该算法同时利用了无向图三角化的Direc性质与LB-单纯性质,在生成的三角化图的总权以及增加边的数目上均明显优于目前广泛采用的Min.Weight Heuristic算法。     

Maximal prime subgraph decomposition of Bayesian networks

The authors present a method for decomposition of Bayesian networks into their maximal prime subgraphs. The correctness of the method is proven and results relating the maximal prime subgraph decomposition (MPD) to the maximal complete subgraphs of the moral graph of the original Bayesian network are presented. The maximal prime subgraphs of a Bayesian network can be organized as a tree which can be used as the computational structure for LAZY propagation. We also identify a number of tasks performed on Bayesian networks that can benefit from MPD. These tasks are: divide and conquer triangulation, hybrid propagation algorithms combining exact and approximative inference techniques, and incremental construction of junction trees. We compare the proposed algorithm with standard algorithms for decomposition of undirected graphs into their maximal prime subgraphs. The discussion shows that the proposed algorithm is simpler, more easy to comprehend, and it has the same complexity as the standard algorithms.     

基于评分函数的贝叶斯网络结构融合算法

利用贝叶斯网络进行因果关系推理已广泛应用于人工智能领域。基于约束方法从观测数据中构建贝叶斯网络通常得到的是其马尔科夫等价类，因存在无向边而无法进行有效的因果推断。为此，基于贝叶斯网络评分函数，并结合集成学习提出了一种模型融合算法，通过对不同的网络结构加权融合，以减少网络中无向边的个数，进而提高其可推断性。实验结果表明，不仅显著减少了无向边条数，也提高了最终网络结构的学习效果，验证了算法的有效性。     

An Introduction to Variational Methods for Graphical Models

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.     

DUCF: Distributed load balancing Unequal Clustering in wireless sensor networks using Fuzzy approach

Data gathering in wireless sensor networks (WSN) consumes more energy due to large amount of data transmitted. In direct transmission (DT) method, each node has to transmit its generated data to the base station (BS) which leads to higher energy consumption and affects the lifetime of the network. Clustering is one of the efficient ways of data gathering in WSN. There are various kinds of clustering techniques, which reduce the overall energy consumption in sensor networks. Cluster head (CH) plays a vital role in data gathering in clustered WSN. Energy consumption in CH node is comparatively higher than other non CH nodes because of its activities like data aggregation and transmission to BS node. The present day clustering algorithms in WSN use multi-hopping mechanism which cost higher energy for the CH nodes near to BS since it routes the data from other CHs to BS. Some CH nodes may die earlier than its intended lifetime due to its overloaded work which affects the performance of the WSN. This paper contributes a new clustering algorithm, Distributed Unequal Clustering using Fuzzy logic (DUCF) which elects CHs using fuzzy approach. DUCF forms unequal clusters to balance the energy consumption among the CHs. Fuzzy inference system (FIS) in DUCF uses the residual energy, node degree and distance to BS as input variables for CH election. Chance and size are the output fuzzy parameters in DUCF. DUCF assigns the maximum limit (size) of a number of member nodes for a CH by considering its input fuzzy parameters. The smaller cluster size is assigned for CHs which are nearer to BS since it acts as a router for other distant CHs. DUCF ensures load balancing among the clusters by varying the cluster size of its CH nodes. DUCF uses Mamdani method for fuzzy inference and Centroid method for defuzzification. DUCF performance was compared with well known algorithms such as LEACH, CHEF and EAUCF in various network scenarios. The experimental results indicated that DUCF forms unequal clusters which ensure load balancing among clusters, which again improves the network lifetime compared with its counterparts.     

对弧进行删除的Bayes网络近似方法

弧的删除是一种对Bayes网络模型进行近似的方法。文中以Kullback-Leibler偏差作为近似网络和原网络概率分布误差的测度,给出了近似网络在此测度意义下的最优参数。同时,也给出了通过对原网络删除多条弧进行近似的启发式算法,当给定一个误差上界时,可以使用此算法寻找满足误差要求的近似网络。     

概率图模型推理方法的研究进展

近年来概率图模型已成为不确定性推理的研究热点,在人工智能、机器学习与计算机视觉等领域有广阔的应用前景.根据网络结构与查询问题类型的不同,系统地综述了概率图模型的推理算法.首先讨论了贝叶斯网络与马尔可夫网络中解决概率查询问题的精确推理算法与近似推理算法,其中主要介绍精确推理中的VE算法、递归约束算法和团树算法,以及近似推理中的变分近似推理和抽样近似推理算法,并给出了解决MAP查询问题的常用推理算法;然后分别针对混合网络的连续与混合情况阐述其推理算法,并分析了暂态网络的精确推理、近似推理以及混合情况下的推理;最后指出了概率图模型推理方法未来的研究方向.     

基于结构分析的局部Gibbs抽样自动推理算法

提出一种基于结构分析的局部Gibbs抽样的贝叶斯网络推理算法(S-LGSI).S-LGSI算法基于联合树算法的概率图模型分析思想,对贝叶斯网络进行精确分解,然后根据查询结点和证据结点生成具有强相关性的局部网络模型,进而对局部网络模型进行Gibbs抽样推理.与当前基于抽样的其它近似推理算法相比,该算法降低推理的计算维数.同时,由于局部抽样模型包含了与查询结点相关的重要信息,因此该算法保证局部抽样推理的精度.算法分析和在Alarm网的实验结果表明,S-LGSI算法较显著降低时间复杂度,同时也提高推理精度.S-LGSI算法应用于上海证券交易所股票网络的推理结果与实际情况基本一致,表现出较强的实用性.     

一种带有可控阈值参数的分簇路由优化算法

针对传感网中大量冗余数据导致通信频繁中断的问题,提出一种带有可控阈值参数的分簇路由优化算法。引入蚁群算法中的适应度函数和启发式函数,使得下一跳簇首节点的选择更具针对性,实现网络路由树的建立与事件域节点的分布式成簇。利用可控阈值参数和变异系数对网络路由所选最短路径进行优化,保证节点能量消耗较低的同时全网延时最小,并通过全局信息素的更新策略抑制长链路的产生,以均衡全网能量并延长网络生存周期。实验结果表明,该算法与DMOA和MTTA算法在抑制网络能量消耗和延长网络生存周期方面进行对比,其性能指标平均提升了13.72%和12.06%。     

粒子寻优和最小生成树聚类下的WSN能量优化

为了均衡分簇无线传感器网络节点能量负载,提高网络的能量利用效率,提出了一种粒子寻优和最小生成树聚类规则的能量优化算法（OMST）。该算法为了使得簇头的能量负载能够得到均衡,采用基于粒子寻优的方法来进行适应值求解,通过适应值对比来求得最佳簇头,以减少簇内节点的传输能耗。同时,提出一种最小生成树聚类规则的簇首数量选择方法,该方法基于剩余能量和距离因素来选择最优的簇首数量,在保证数据传输质量的同时最小化网络总能量的消耗量。仿真结果表明,相比一种新型差分进化的无线传感器网络聚类算法和多层节能及距离感知的无线传感器网络聚类算法,OMST算法的节点平均能量效率分别提高了16.7%和6.4%,网络节点存活数量分别提高了24.1%和13.7%。     

传感器网络中节点个数约束查询处理算法

对传感器网络中一类新查询--节点个数约束查询,提出能量有效的查询处理算法.算法主要由查询下发和结果回收两部分构成.查询下发算法首先根据节点个数约束查询的特点提出相关节点选择以及基于Steiner树的查询下发算法.然后对该下发算法以及一种基于洪泛的能量有效查询下发算法的能量消耗进行分析,并对比两种算法的能量消耗从中选择适当的下发算法.结果回收算法提出直接和间接两种结果回收方式,并给出两种方式在进行结果回收时能够节省能量的条件.仿真实验表明,提出的能量有效节点个数约束查询处理算法能够在满足用户查询精度的同时,使其能量消耗低于其他查询处理算法.     

Computational Properties of Two Exact Algorithms for Bayesian Networks

This paper studies computational properties of two exact inference algorithms for Bayesian networks, namely the clique tree propagation algorithm (CTP)¹ and the variable elimination algorithm (VE). VE permits pruning of nodes irrelevant to a query while CTP facilitates sharing of computations among different queries. Experiments have been conducted to empirically compare VE and CTP. We found that, contrary to common beliefs, VE is often more efficient than CTP, especially in complex networks.     

Research of energy efficient clustering algorithm for multilayer wireless heterogeneous sensor networks prediction research

In designing wireless sensor networks of image transmitting, it is important to reduce energy dissipation and prolong network lifetime. This paper presents the research on existing clustering algorithm applied in heterogeneous sensor networks and then puts forward an energy-efficient prediction clustering algorithm, which is adaptive to sensor networks with energy and objects heterogeneous. This algorithm enables the nodes to select the cluster head according to factors such as energy and communication cost, thus the nodes with higher residual energy have higher probability to become a cluster head than those with lower residual energy, so that the network energy can be dissipated uniformly. In order to reduce energy consumption when broadcasting in clustering phase and prolong network lifetime, an energy consumption prediction model is established for regular data acquisition nodes. Simulation results and the application in image clustering show that compared with current clustering algorithms, this algorithm can achieve longer sensor network lifetime, higher energy efficiency, and superior network monitoring quality.     

基于节点向量表达的复杂网络社团划分算法

社团结构划分对复杂网络研究在理论和实践上都非常重要.借鉴分布式词向量理论,提出一种基于节点向量表达的复杂网络社团划分方法（CDNEV）.为了构建网络节点的分布式向量,提出启发式随机游走模型.利用节点启发式随机游走得到的节点序列作为上下文,采用SkipGram模型学习节点的分布式向量.选择局部度中心节点作为K-Means算法的聚类中心点,然后用K-Means算法进行聚类,最终得到社团结构.在真实和模拟两种网络上做了丰富的实验,与主流的全局社团划分算法和局部社团划分算法作了比较.在真实网络上CDNEV算法的F1指标比其他算法平均提高19%;在模拟网络上,F1指标则可以提高15%.实验结果表明,相对其他算法,CDNEV算法的精度和效率都较高.     

EBNO: Evolution of cost-sensitive Bayesian networks

The last decade has seen an increase in the attention paid to the development of cost-sensitive learning algorithms that aim to minimize misclassification costs while still maintaining accuracy. Most of this attention has been on cost-sensitive decision tree learning, whereas relatively little attention has been paid to assess if it is possible to develop better cost-sensitive classifiers based on Bayesian networks. Hence, this paper presents EBNO, an algorithm that utilizes Genetic algorithms to learn cost-sensitive Bayesian networks, where genes are utilized to represent the links between the nodes in Bayesian networks and the expected cost is used as a fitness function. An empirical comparison of the new algorithm has been carried out with respect to (a) an algorithm that induces cost-insensitive Bayesian networks to provide a base line, (b) ICET, a well-known algorithm that uses Genetic algorithms to induce cost-sensitive decision trees, (c) use of MetaCost to induce cost-sensitive Bayesian networks via bagging (d) use of AdaBoost to induce cost-sensitive Bayesian networks, and (e) use of XGBoost, a gradient boosting algorithm, to induce cost-sensitive decision trees. An empirical evaluation on 28 data sets reveals that EBNO performs well in comparison with the algorithms that produce single interpretable models and performs just as well as algorithms that use bagging and boosting methods.     

Portfolios in Stochastic Local Search: Efficiently Computing Most Probable Explanations in Bayesian Networks

Portfolio methods support the combination of different algorithms and heuristics, including stochastic local search (SLS) heuristics, and have been identified as a promising approach to solve computationally hard problems. While successful in experiments, theoretical foundations and analytical results for portfolio-based SLS heuristics are less developed. This article aims to improve the understanding of the role of portfolios of heuristics in SLS. We emphasize the problem of computing most probable explanations (MPEs) in Bayesian networks (BNs). Algorithmically, we discuss a portfolio-based SLS algorithm for MPE computation, Stochastic Greedy Search (SGS). SGS supports the integration of different initialization operators (or initialization heuristics) and different search operators (greedy and noisy heuristics), thereby enabling new analytical and experimental results. Analytically, we introduce a novel Markov chain model tailored to portfolio-based SLS algorithms including SGS, thereby enabling us to analytically form expected hitting time results that explain empirical run time results. For a specific BN, we show the benefit of using a homogenous initialization portfolio. To further illustrate the portfolio approach, we consider novel additive search heuristics for handling determinism in the form of zero entries in conditional probability tables in BNs. Our additive approach adds rather than multiplies probabilities when computing the utility of an explanation. We motivate the additive measure by studying the dramatic impact of zero entries in conditional probability tables on the number of zero-probability explanations, which again complicates the search process. We consider the relationship between MAXSAT and MPE, and show that additive utility (or gain) is a generalization, to the probabilistic setting, of MAXSAT utility (or gain) used in the celebrated GSAT and WalkSAT algorithms and their descendants. Utilizing our Markov chain framework, we show that expected hitting time is a rational function—i.e. a ratio of two polynomials—of the probability of applying an additive search operator. Experimentally, we report on synthetically generated BNs as well as BNs from applications, and compare SGS’s performance to that of Hugin, which performs BN inference by compilation to and propagation in clique trees. On synthetic networks, SGS speeds up computation by approximately two orders of magnitude compared to Hugin. In application networks, our approach is highly competitive in Bayesian networks with a high degree of determinism. In addition to showing that stochastic local search can be competitive with clique tree clustering, our empirical results provide an improved understanding of the circumstances under which portfolio-based SLS outperforms clique tree clustering and vice versa.     

基于三角形斯坦纳树的分区连通性恢复算法

无线传感器网络中的节点由于自身能量的消耗,及外部因素影响会导致节点出现大规模的失效,从而把无线传感器网络分割成几个独立的不能相互通信的分区。为恢复网络,重建分区之间的通信链路,提出基于三角形斯坦纳树连通恢复算法。该算法首先利用传统算法实现分区连通,然后通过构建三角形斯坦纳树以减少部署的中继节点数量。与现有的一些算法相比,该方法形成的网络拓扑不仅减少了部署中继节点的数量,能够使分区重新连通,而且能够减少网络通信的能量消耗。实验结果表明,所提方法相对于传统算法在构建网络拓扑时更加有效。