Learning Bayesian networks for discrete data

Bayesian networks have received much attention in the recent literature. In this article, we propose an approach to learn Bayesian networks using the stochastic approximation Monte Carlo (SAMC) algorithm. Our approach has two nice features. Firstly, it possesses the self-adjusting mechanism and thus avoids essentially the local-trap problem suffered by conventional MCMC simulation-based approaches in learning Bayesian networks. Secondly, it falls into the class of dynamic importance sampling algorithms; the network features can be inferred by dynamically weighted averaging the samples generated in the learning process, and the resulting estimates can have much lower variation than the single model-based estimates. The numerical results indicate that our approach can mix much faster over the space of Bayesian networks than the conventional MCMC simulation-based approaches.     

基于预测能力的连续贝叶斯网络结构学习

通过对连续随机变量之间预测能力及其计算方法的讨论,提出基于预测能力的连续贝叶斯网络结构学习方法。该方法包括两个步骤,每个步骤都伴随环路检验。首先建立初始贝叶斯网络结构,其次调整初始贝叶斯网络结构,包括增加丢失的弧、删除多余的弧及调整弧的方向,并使用模拟数据进行了对比实验,结果表明该方法非常有致。     

Learning tractable NAT-modeled Bayesian networks

Annals of Mathematics and Artificial Intelligence - Bayesian networks (BNs) encode conditional independence to avoid combinatorial explosion on the number of variables, but are subject to...     

Method of probabilistic inference from learning data in Bayesian networks

Bayesian networks (BN) are a powerful tool for various data-mining systems. The available methods of probabilistic inference from learning data have shortcomings such as high computation complexity and cumulative error. This is due to a partial loss of information in transition from empiric information to conditional probability tables. The paper presents a new simple and exact algorithm for probabilistic inference in BN from learning data. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 93–99, May–June 2007.     

Learning Bayesian networks with a hybrid convergent method

During the past few years, a variety of methods have been developed for learning probabilistic networks from data, among which the heuristic single link forward or backward searches are widely adopted to reduce the search space. A major drawback of these search heuristics is that they can not guarantee to converge to the right networks even if a sufficiently large data set is available. This motivates us to explore an algorithm that will not suffer from this problem. We first identify an asymptotic property of different score metrics, based on which we then present a hybrid learning method that can be proved to be asymptotically convergent. We show that the algorithm, when employing the information criterion and the Bayesian metric, guarantees to converge in a very general way and is computationally feasible. Evaluation of the algorithm with simulated data is given to demonstrate the capability of the algorithm     

Learning action probabilities from delayed reinforcement

A reinforcement scheme for learning automata, applicable to real situations where the reinforcement received from the environment is delayed, is presented. The scheme divides the state space into regions following the boxes approach of Michie and Chambers. Each region maintains estimates of the reward characteristics of the environment and contains a local automaton that updates action probabilities whenever the system state enters it. Estimates of reward characteristics are obtained using reinforcement received during the period of eligibility. Results obtained through computer simulation of the inverted pendulum problem are compared with the adaptive critic learning developed by Barto et al. (1983).     

具有丢失数据的贝叶斯网络结构学习算法

学习具有丢失数据的贝叶斯网络结构主要采用结合 EM 算法的打分一搜索方法,其效率和可靠性比较低.针对此问题建立一个新的具有丢失数据的贝叶斯网络结构学习算法.该方法首先用 Kullback-Leibler(KL)散度来表示同一结点的各个案例之间的相似程度,然后根据 Gibbs 取样来得出丢失数据的取值.最后,用启发式搜索完成贝叶斯网络结构的学习.该方法能够有效避免标准 Gibbs 取样的指数复杂性问题和现有学习方法存在的主要问题.     

基于因果语义定向的贝叶斯网络结构学习

基于变量之间基本依赖关系、基本结构、d-separation标准、依赖分析思想和混合定向策略,给出了一种有效实用的贝叶斯网络结构学习方法,不需要结点有序,并能避免打分-搜索方法存在的指数复杂性,以及现有依赖分析方法的大量高维条件概率计算等问题。     

Bayesian networks based rare event prediction with sensor data

A Bayesian network is a powerful graphical model. It is advantageous for real-world data analysis and finding relations among variables. Knowledge presentation and rule generation, based on a Bayesian approach, have been studied and reported in many research papers across various fields. Since a Bayesian network has both causal and probabilistic semantics, it is regarded as an ideal representation to combine background knowledge and real data. Rare event predictions have been performed using several methods, but remain a challenge. We design and implement a Bayesian network model to forecast daily ozone states. We evaluate the proposed Bayesian network model, comparing it to traditional decision tree models, to examine its utility.     

Bayesian estimation of survival and capture probabilities using logit link and photoidentification data

We develop a series of Bayesian statistical models for estimating survival of wild populations monitored using capture-recapture experiments and photoidentification data. The proposed methodology is based on Cormack-Jolly-Seber model [Cormack, R.M., 1964. Estimates of survival from the sighting of marked animals. Biometrika 51, 429-438; Jolly, G.M., 1965. Explicit estimates from capture-recapture data with both death and immigration—stochastic model. Biometrika 52, 225-247; Seber, G.A.F., 1965. A note on the multiple recapture census. Biometrika 52, 249-259]. Besides time effects in capture probabilities, the proposed models allow taking into account heterogeneity in capture probability caused by the existence of different groups of individuals in the population. For that purpose, the capture probabilities are fitted using a logistic model. Additionally, it is also possible to estimate group-specific survival rates. Goodness of fit is evaluated using Bayes factor methodology. The models are applied to an 11-year photoidentification capture-recapture experiment for bowhead whales, Balaena mysticetus. The best model provides an estimate close to the one obtained by Zeh et al. [2002. Survival of bowhead whales, Balaena mysticetus, estimated from 1981-1998 photoidentification data. Biometrics 58, 832-840] using the Jolly-Seber model, but accounting for heterogeneity in capture probabilities improves precision.     

Learning Bayesian networks using various datasources and applications to financial analysis

 This paper illustrates opportunities of using Bayesian networks in fundamental financial analysis. In it, we will present an application based on construction of a Bayesian network from a database of financial reports collected for the years 1993–97. We will focus on one sector of the Czech economy – engineering – presenting an example that use the constructed Bayesian network in the sector financial analysis. In addition, we will deal with the rating analysis and show how to perform this kind of analysis by means of neural and Bayesian networks. This work was supported by the grant VS96008 of the Ministry of Education of the Czech Republic.     

Argument diagram extraction from evidential Bayesian networks

Bayesian networks (BN) and argumentation diagrams (AD) are two predominant approaches to legal evidential reasoning, that are often treated as alternatives to one another. This paper argues that they are, instead, complimentary and proposes the beginnings of a method to employ them in such a manner. The Bayesian approach tends to be used as a means to analyse the findings of forensic scientists. As such, it constitutes a means to perform evidential reasoning. The design of Bayesian networks that accurately and comprehensively represent the relationships between investigative hypotheses and evidence remains difficult and sometimes contentious, however. Argumentation diagrams are representations of reasoning, and are used as a means to scrutinise reasoning (among other applications). In evidential reasoning, they tend to be used to represent and scrutinise the way humans reason about evidence. This paper examines how argumentation diagrams can be used to scrutinise Bayesian evidential reasoning by developing a method to extract argument diagrams from BN.     

Maximum entropy and least square error minimizing procedures for estimating missing conditional probabilities in Bayesian networks

Conditional probability tables (CPT) in many Bayesian networks often contain missing values. The problem of missing values in CPT is a very common problem and occurs due to the lack of data on certain scenarios that are observed in the real world but are missing in the training data. The current approaches of addressing the problem of missing values in CPT are very restrictive in that they assume certain probability distributions for estimating missing values. Recently, maximum entropy (ME) approaches have been used to learn features of probability distribution functions from the observed data. The ME approaches do not require any data distribution assumptions and are shown to work well for several non-parametric distributions. The ME and least square (LS) error minimizing approaches can be used for estimating missing values in CPT for Bayesian networks. The applications of ME and LS approaches for estimating missing CPT require researchers to solve difficult constrained non-linear optimization problems. These difficult constrained non-linear optimization problems can be solved using genetic algorithms.     

Learning Bayesian network parameters under incomplete data with domain knowledge

Bayesian networks (BNs) have gained increasing attention in recent years. One key issue in Bayesian networks is parameter learning. When training data is incomplete or sparse or when multiple hidden nodes exist, learning parameters in Bayesian networks becomes extremely difficult. Under these circumstances, the learning algorithms are required to operate in a high-dimensional search space and they could easily get trapped among copious local maxima. This paper presents a learning algorithm to incorporate domain knowledge into the learning to regularize the otherwise ill-posed problem, to limit the search space, and to avoid local optima. Unlike the conventional approaches that typically exploit the quantitative domain knowledge such as prior probability distribution, our method systematically incorporates qualitative constraints on some of the parameters into the learning process. Specifically, the problem is formulated as a constrained optimization problem, where an objective function is defined as a combination of the likelihood function and penalty functions constructed from the qualitative domain knowledge. Then, a gradient-descent procedure is systematically integrated with the E-step and M-step of the EM algorithm, to estimate the parameters iteratively until it converges. The experiments with both synthetic data and real data for facial action recognition show our algorithm improves the accuracy of the learned BN parameters significantly over the conventional EM algorithm.     

Learning Bayesian network classifiers from label proportions

This paper deals with a classification problem known as learning from label proportions. The provided dataset is composed of unlabeled instances and is divided into disjoint groups. General class information is given within the groups: the proportion of instances of the group that belong to each class.We have developed a method based on the Structural EM strategy that learns Bayesian network classifiers to deal with the exposed problem. Four versions of our proposal are evaluated on synthetic data, and compared with state-of-the-art approaches on real datasets from public repositories. The results obtained show a competitive behavior for the proposed algorithm.     

Learning indistinguishability from data

 In this paper we revisit the idea of interpreting fuzzy sets as representations of vague values. In this context a fuzzy set is induced by a crisp value and the membership degree of an element is understood as the similarity degree between this element and the crisp value that determines the fuzzy set. Similarity is assumed to be a notion of distance. This means that fuzzy sets are induced by crisp values and an appropriate distance function. This distance function can be described in terms of scaling the ordinary distance between real numbers. With this interpretation in mind, the task of designing a fuzzy system corresponds to determining suitable crisp values and appropriate scaling functions for the distance. When we want to generate a fuzzy model from data, the parameters have to be fitted to the data. This leads to an optimisation problem that is very similar to the optimisation task to be solved in objective function based clustering. We borrow ideas from the alternating optimisation schemes applied in fuzzy clustering in order to develop a new technique to determine our set of parameters from data, supporting the interpretability of the fuzzy system.     

Recovering networks from distance data

A fully probabilistic approach to reconstructing Gaussian graphical models from distance data is presented. The main idea is to extend the usual central Wishart model in traditional methods to using a likelihood depending only on pairwise distances, thus being independent of geometric assumptions about the underlying Euclidean space. This extension has two advantages: the model becomes invariant against potential bias terms in the measurements, and can be used in situations which on input use a kernel- or distance matrix, without requiring direct access to the underlying vectors. The latter aspect opens up a huge new application field for Gaussian graphical models, as network reconstruction is now possible from any Mercer kernel, be it on graphs, strings, probabilities or more complex objects. We combine this likelihood with a suitable prior to enable Bayesian network inference. We present an efficient MCMC sampler for this model and discuss the estimation of module networks. Experiments depict the high quality and usefulness of the inferred networks.     

Prediction of survival probabilities with Bayesian Decision Trees

Practitioners use Trauma and Injury Severity Score (TRISS) models for predicting the survival probability of an injured patient. The accuracy of TRISS predictions is acceptable for patients with up to three typical injuries, but unacceptable for patients with a larger number of injuries or with atypical injuries. Based on a regression model, the TRISS methodology does not provide the predictive density required for accurate assessment of risk. Moreover, the regression model is difficult to interpret. We therefore consider Bayesian inference for estimating the predictive distribution of survival. The inference is based on decision tree models which recursively split data along explanatory variables, and so practitioners can understand these models. We propose the Bayesian method for estimating the predictive density and show that it outperforms the TRISS method in terms of both goodness-of-fit and classification accuracy. The developed method has been made available for evaluation purposes as a stand-alone application.     

Bayesian density estimation from grouped continuous data

Grouped data occur frequently in practice, either because of limited resolution of instruments, or because data have been summarized in relatively wide bins. A combination of the composite link model with roughness penalties is proposed to estimate smooth densities from such data in a Bayesian framework. A simulation study is used to evaluate the performances of the strategy in the estimation of a density, of its quantiles and first moments. Two illustrations are presented: the first one involves grouped data of lead concentration in the blood and the second one the number of deaths due to tuberculosis in The Netherlands in wide age classes.     

Learning influence from heterogeneous social networks

Influence is a complex and subtle force that governs social dynamics and user behaviors. Understanding how users influence each other can benefit various applications, e.g., viral marketing, recommendation, information retrieval and etc. While prior work has mainly focused on qualitative aspect, in this article, we present our research in quantitatively learning influence between users in heterogeneous networks. We propose a generative graphical model which leverages both heterogeneous link information and textual content associated with each user in the network to mine topic-level influence strength. Based on the learned direct influence, we further study the influence propagation and aggregation mechanisms: conservative and non-conservative propagations to derive the indirect influence. We apply the discovered influence to user behavior prediction in four different genres of social networks: Twitter, Digg, Renren, and Citation. Qualitatively, our approach can discover some interesting influence patterns from these heterogeneous networks. Quantitatively, the learned influence strength greatly improves the accuracy of user behavior prediction.