Learning Bayesian networks for discrete data

Bayesian networks have received much attention in the recent literature. In this article, we propose an approach to learn Bayesian networks using the stochastic approximation Monte Carlo (SAMC) algorithm. Our approach has two nice features. Firstly, it possesses the self-adjusting mechanism and thus avoids essentially the local-trap problem suffered by conventional MCMC simulation-based approaches in learning Bayesian networks. Secondly, it falls into the class of dynamic importance sampling algorithms; the network features can be inferred by dynamically weighted averaging the samples generated in the learning process, and the resulting estimates can have much lower variation than the single model-based estimates. The numerical results indicate that our approach can mix much faster over the space of Bayesian networks than the conventional MCMC simulation-based approaches.     

基于预测能力的连续贝叶斯网络结构学习

通过对连续随机变量之间预测能力及其计算方法的讨论,提出基于预测能力的连续贝叶斯网络结构学习方法。该方法包括两个步骤,每个步骤都伴随环路检验。首先建立初始贝叶斯网络结构,其次调整初始贝叶斯网络结构,包括增加丢失的弧、删除多余的弧及调整弧的方向,并使用模拟数据进行了对比实验,结果表明该方法非常有致。     

Learning tractable NAT-modeled Bayesian networks

Annals of Mathematics and Artificial Intelligence - Bayesian networks (BNs) encode conditional independence to avoid combinatorial explosion on the number of variables, but are subject to...     

用于风险管理的贝叶斯网络学习

结合专家知识和数据进行贝叶斯网络学习.首先利用专家知识建立初始贝叶斯网络结构和参数;然后基于变量之间基本依赖关系、基本结构和依赖分析方法,对初始贝叶斯网络结构进行修正和调整,得到新的贝叶斯网络结构;最后将由专家和数据确定的参数合成为新的参数,得到融合专家知识和数据的贝叶斯网络.该方法可避免现有的贝叶斯网络学习过于依赖数据、对数据的数量和质量要求过高等问题.     

Method of probabilistic inference from learning data in Bayesian networks

Bayesian networks (BN) are a powerful tool for various data-mining systems. The available methods of probabilistic inference from learning data have shortcomings such as high computation complexity and cumulative error. This is due to a partial loss of information in transition from empiric information to conditional probability tables. The paper presents a new simple and exact algorithm for probabilistic inference in BN from learning data. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 93–99, May–June 2007.     

Learning bipartite Bayesian networks under monotonicity restrictions

ABSTRACT

Learning parameters of a probabilistic model is a necessary step in machine learning tasks. We present a method to improve learning from small datasets by using monotonicity conditions. Monotonicity simplifies the learning and it is often required by users. We present an algorithm for Bayesian Networks parameter learning. The algorithm and monotonicity conditions are described, and it is shown that with the monotonicity conditions we can better fit underlying data. Our algorithm is tested on artificial and empiric datasets. We use different methods satisfying monotonicity conditions: the proposed gradient descent, isotonic regression EM, and non-linear optimization. We also provide results of unrestricted EM and gradient descent methods. Learned models are compared with respect to their ability to fit data in terms of log-likelihood and their fit of parameters of the generating model. Our proposed method outperforms other methods for small sets, and provides better or comparable results for larger sets.     

Learning with privileged information using Bayesian networks

For many supervised learning applications, additional information, besides the labels, is often available during training, but not available during testing. Such additional information, referred to the privileged information, can be exploited during training to construct a better classifier. In this paper, we propose a Bayesian network (BN) approach for learning with privileged information. We propose to incorporate the privileged information through a three-node BN. We further mathematically evaluate different topologies of the three-node BN and identify those structures, through which the privileged information can benefit the classification. Experimental results on handwritten digit recognition, spontaneous versus posed expression recognition, and gender recognition demonstrate the effectiveness of our approach.     

Learning action probabilities from delayed reinforcement

A reinforcement scheme for learning automata, applicable to real situations where the reinforcement received from the environment is delayed, is presented. The scheme divides the state space into regions following the boxes approach of Michie and Chambers. Each region maintains estimates of the reward characteristics of the environment and contains a local automaton that updates action probabilities whenever the system state enters it. Estimates of reward characteristics are obtained using reinforcement received during the period of eligibility. Results obtained through computer simulation of the inverted pendulum problem are compared with the adaptive critic learning developed by Barto et al. (1983).     

基于贝叶斯网络理论的道德图学习*

贝叶斯网络的道德图是一种马尔可夫网络,是进行随机变量之间依赖关系分析、推理及预测的有力工具。基于道德图和贝叶斯网络间的密切联系,提出了一种基于贝叶斯网络理论进行道德图学习的方法。实验表明该方法能够显著提高道德图学习效率和可靠性,适合于多变量稀疏道德图学习。     

Learning Bayesian networks with a hybrid convergent method

During the past few years, a variety of methods have been developed for learning probabilistic networks from data, among which the heuristic single link forward or backward searches are widely adopted to reduce the search space. A major drawback of these search heuristics is that they can not guarantee to converge to the right networks even if a sufficiently large data set is available. This motivates us to explore an algorithm that will not suffer from this problem. We first identify an asymptotic property of different score metrics, based on which we then present a hybrid learning method that can be proved to be asymptotically convergent. We show that the algorithm, when employing the information criterion and the Bayesian metric, guarantees to converge in a very general way and is computationally feasible. Evaluation of the algorithm with simulated data is given to demonstrate the capability of the algorithm     

具有丢失数据的贝叶斯网络结构学习算法

学习具有丢失数据的贝叶斯网络结构主要采用结合 EM 算法的打分一搜索方法,其效率和可靠性比较低.针对此问题建立一个新的具有丢失数据的贝叶斯网络结构学习算法.该方法首先用 Kullback-Leibler(KL)散度来表示同一结点的各个案例之间的相似程度,然后根据 Gibbs 取样来得出丢失数据的取值.最后,用启发式搜索完成贝叶斯网络结构的学习.该方法能够有效避免标准 Gibbs 取样的指数复杂性问题和现有学习方法存在的主要问题.     

基于因果语义定向的贝叶斯网络结构学习

基于变量之间基本依赖关系、基本结构、d-separation标准、依赖分析思想和混合定向策略,给出了一种有效实用的贝叶斯网络结构学习方法,不需要结点有序,并能避免打分-搜索方法存在的指数复杂性,以及现有依赖分析方法的大量高维条件概率计算等问题。     

小数据集条件下贝叶斯网络自适应参数学习方法

针对小数据集条件下贝叶斯网络参数学习问题,约束最大似然(CML)和定性最大后验概率(QMAP)方法是两种约束适用性较好的方法.当样本数量、约束数量、参数位置不同时,上述两种方法互有优劣,进而导致方法上的难以选择.因此,本文提出一种自适应参数学习方法:首先,利用CML和QMAP方法学习得到两组参数;然后,基于拒绝–接受采样和空间最大后验概率思想自定义计算得到样本权重、约束权重、参数位置权重;最后,基于上述参数和权重计算得到新的参数解.实验表明:在任何条件下,本文方法计算得到参数的精度接近甚至优于CML和QMAP方法的最优解.     

Bayesian estimation of survival and capture probabilities using logit link and photoidentification data

We develop a series of Bayesian statistical models for estimating survival of wild populations monitored using capture–recapture experiments and photoidentification data. The proposed methodology is based on Cormack–Jolly–Seber model [Cormack, R.M., 1964. Estimates of survival from the sighting of marked animals. Biometrika 51, 429–438; Jolly, G.M., 1965. Explicit estimates from capture–recapture data with both death and immigration—stochastic model. Biometrika 52, 225–247; Seber, G.A.F., 1965. A note on the multiple recapture census. Biometrika 52, 249–259]. Besides time effects in capture probabilities, the proposed models allow taking into account heterogeneity in capture probability caused by the existence of different groups of individuals in the population. For that purpose, the capture probabilities are fitted using a logistic model. Additionally, it is also possible to estimate group-specific survival rates. Goodness of fit is evaluated using Bayes factor methodology. The models are applied to an 11-year photoidentification capture–recapture experiment for bowhead whales, Balaena mysticetus. The best model provides an estimate close to the one obtained by Zeh et al. [2002. Survival of bowhead whales, Balaena mysticetus, estimated from 1981–1998 photoidentification data. Biometrics 58, 832–840] using the Jolly–Seber model, but accounting for heterogeneity in capture probabilities improves precision.     

Bayesian estimation of survival and capture probabilities using logit link and photoidentification data

We develop a series of Bayesian statistical models for estimating survival of wild populations monitored using capture-recapture experiments and photoidentification data. The proposed methodology is based on Cormack-Jolly-Seber model [Cormack, R.M., 1964. Estimates of survival from the sighting of marked animals. Biometrika 51, 429-438; Jolly, G.M., 1965. Explicit estimates from capture-recapture data with both death and immigration—stochastic model. Biometrika 52, 225-247; Seber, G.A.F., 1965. A note on the multiple recapture census. Biometrika 52, 249-259]. Besides time effects in capture probabilities, the proposed models allow taking into account heterogeneity in capture probability caused by the existence of different groups of individuals in the population. For that purpose, the capture probabilities are fitted using a logistic model. Additionally, it is also possible to estimate group-specific survival rates. Goodness of fit is evaluated using Bayes factor methodology. The models are applied to an 11-year photoidentification capture-recapture experiment for bowhead whales, Balaena mysticetus. The best model provides an estimate close to the one obtained by Zeh et al. [2002. Survival of bowhead whales, Balaena mysticetus, estimated from 1981-1998 photoidentification data. Biometrics 58, 832-840] using the Jolly-Seber model, but accounting for heterogeneity in capture probabilities improves precision.     

面向大规模数据集的贝叶斯网络参数学习算法

贝叶斯网络的学习可以分为结构学习和参数学习。期望最大化（EM）算法通常用于不完整数据的参数学习,但是由于EM算法计算相对复杂,存在收敛速度慢和容易局部最大化等问题,传统的EM算法难于处理大规模数据集。研究了EM算法的主要问题,采用划分数据块的方法将大规模数据集划分为小的样本集来处理,降低了EM算法的计算量,同时也提高了计算精度。实验证明,该改进的EM算法具有较高的性能。     

Learning Bayesian networks for clustering by means of constructive induction

The purpose of this paper is to present and evaluate a heuristic algorithm for learning Bayesian networks for clustering. Our approach is based upon improving the Naive-Bayes model by means of constructive induction. A key idea in this approach is to treat expected data as real data. This allows us to complete the database and to take advantage of factorable closed forms for the marginal likelihood. In order to get such an advantage, we search for parameter values using the EM algorithm or another alternative approach that we have developed: a hybridization of the Bound and Collapse method and the EM algorithm, which results in a method that exhibits a faster convergence rate and a more effective behaviour than the EM algorithm. Also, we consider the possibility of interleaving runnings of these two methods after each structural change. We evaluate our approach on synthetic and real-world databases.     

Bayesian networks based rare event prediction with sensor data

A Bayesian network is a powerful graphical model. It is advantageous for real-world data analysis and finding relations among variables. Knowledge presentation and rule generation, based on a Bayesian approach, have been studied and reported in many research papers across various fields. Since a Bayesian network has both causal and probabilistic semantics, it is regarded as an ideal representation to combine background knowledge and real data. Rare event predictions have been performed using several methods, but remain a challenge. We design and implement a Bayesian network model to forecast daily ozone states. We evaluate the proposed Bayesian network model, comparing it to traditional decision tree models, to examine its utility.     

Learning Bayesian networks using various datasources and applications to financial analysis

 This paper illustrates opportunities of using Bayesian networks in fundamental financial analysis. In it, we will present an application based on construction of a Bayesian network from a database of financial reports collected for the years 1993–97. We will focus on one sector of the Czech economy – engineering – presenting an example that use the constructed Bayesian network in the sector financial analysis. In addition, we will deal with the rating analysis and show how to perform this kind of analysis by means of neural and Bayesian networks. This work was supported by the grant VS96008 of the Ministry of Education of the Czech Republic.     

Maximum entropy and least square error minimizing procedures for estimating missing conditional probabilities in Bayesian networks

Conditional probability tables (CPT) in many Bayesian networks often contain missing values. The problem of missing values in CPT is a very common problem and occurs due to the lack of data on certain scenarios that are observed in the real world but are missing in the training data. The current approaches of addressing the problem of missing values in CPT are very restrictive in that they assume certain probability distributions for estimating missing values. Recently, maximum entropy (ME) approaches have been used to learn features of probability distribution functions from the observed data. The ME approaches do not require any data distribution assumptions and are shown to work well for several non-parametric distributions. The ME and least square (LS) error minimizing approaches can be used for estimating missing values in CPT for Bayesian networks. The applications of ME and LS approaches for estimating missing CPT require researchers to solve difficult constrained non-linear optimization problems. These difficult constrained non-linear optimization problems can be solved using genetic algorithms.