Bayesian and non-bayesian evidential updating

Four main results are arrived at in this paper. (1) Closed convex sets of classical probability functions provide a representation of belief that includes the representations provided by Shafer probability mass functions as a special case. (2) The impact of “uncertain evidence” can be (formally) represented by Dempster conditioning, in Shafer's framework. (3) The impact of “uncertain evidence” can be (formally) represented in the framework of convex sets of classical probabilities by classical conditionalization. (4) The probability intervals that result from Dempster-Shafer updating on uncertain evidence are included in (and may be properly included in) the intervals that result from Bayesian updating on uncertain evidence.     

Learning Bayesian networks probabilities from longitudinal data

Many real applications of Bayesian networks (BN) concern problems in which several observations are collected over time on a certain number of similar plants. This situation is typical of the context of medical monitoring, in which several measurements of the relevant physiological quantities are available over time on a population of patients under treatment, and the conditional probabilities that describe the model are usually obtained from the available data through a suitable learning algorithm. In situations with small data sets for each plant, it is useful to reinforce the parameter estimation process of the BN by taking into account the observations obtained from other similar plants. On the other hand, a desirable feature to be preserved is the ability to learn individualized conditional probability tables, rather than pooling together all the available data. In this work we apply a Bayesian hierarchical model able to preserve individual parameterization, and, at the same time, to allow the conditionals of each plant to borrow strength from all the experience contained in the data-base. A testing example and an application in the context of diabetes monitoring will be shown     

Restricted Bayesian classification networks

Bayesian networks are graphical models that describe dependency relationships between variables, and are powerful tools for studying probability classifiers. At present, the causal Bayesian network learning method is used in constructing Bayesian network classifiers while the contribution of attribute to class is over-looked. In this paper, a Bayesian network specifically for classification-restricted Bayesian classification networks is proposed. Combining dependency analysis between variables, classification accuracy evaluation criteria and a search algorithm, a learning method for restricted Bayesian classification networks is presented. Experiments and analysis are done using data sets from UCI machine learning repository. The results show that the restricted Bayesian classification network is more accurate than other well-known classifiers.     

Bayesian learning of Bayesian networks with informative priors

This paper presents and evaluates an approach to Bayesian model averaging where the models are Bayesian nets (BNs). A comprehensive study of the literature on structural priors for BNs is conducted. A number of prior distributions are defined using stochastic logic programs and the MCMC Metropolis-Hastings algorithm is used to (approximately) sample from the posterior. We use proposals which are tightly coupled to the priors which give rise to cheaply computable acceptance probabilities. Experiments using data generated from known BNs have been conducted to evaluate the method. The experiments used 6 different BNs and varied: the structural prior, the parameter prior, the Metropolis-Hasting proposal and the data size. Each experiment was repeated three times with different random seeds to test the robustness of the MCMC-produced results. Our results show that with effective priors (i) robust results are produced and (ii) informative priors improve results significantly.     

Time-sensitive Arabic multiword expressions extraction from social networks

In this paper, we present a comprehensive approach for extracting and relating Arabic multiword expressions (MWE) from Social Networks. 15 million tweets were collected and processed to form our data set. Due to the complexity of processing Arabic and the lack of resources, we built an experimental system to extract and relate similar MWE using statistical methods. We introduce a new metrics for measuring valid MWE in Social Networks. We compare results obtained from our experimental system against semantic graph obtained from web knowledgebase.     

Algebraic geometry of Bayesian networks

We study the algebraic varieties defined by the conditional independence statements of Bayesian networks. A complete algebraic classification is given for Bayesian networks on at most five random variables. Hidden variables are related to the geometry of higher secant varieties.     

Method of probabilistic inference from learning data in Bayesian networks

Bayesian networks (BN) are a powerful tool for various data-mining systems. The available methods of probabilistic inference from learning data have shortcomings such as high computation complexity and cumulative error. This is due to a partial loss of information in transition from empiric information to conditional probability tables. The paper presents a new simple and exact algorithm for probabilistic inference in BN from learning data. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 93–99, May–June 2007.     

Learning tractable NAT-modeled Bayesian networks

Annals of Mathematics and Artificial Intelligence - Bayesian networks (BNs) encode conditional independence to avoid combinatorial explosion on the number of variables, but are subject to...     

Explaining inferences in Bayesian networks

While Bayesian network (BN) can achieve accurate predictions even with erroneous or incomplete evidence, explaining the inferences remains a challenge. Existing approaches fall short because they do not exploit variable interactions and cannot account for compensations during inferences. This paper proposes the Explaining BN Inferences (EBI) procedure for explaining how variables interact to reach conclusions. EBI explains the value of a target node in terms of the influential nodes in the target’s Markov blanket under specific contexts, where the Markov nodes include the target’s parents, children, and the children’s other parents. Working back from the target node, EBI shows the derivation of each intermediate variable, and finally explains how missing and erroneous evidence values are compensated. We validated EBI on a variety of problem domains, including mushroom classification, water purification and web page recommendation. The experiments show that EBI generates high quality, concise and comprehensible explanations for BN inferences, in particular the underlying compensation mechanism that enables BN to outperform alternative prediction systems, such as decision tree.     

Combating discrimination using Bayesian networks

Discrimination in decision making is prohibited on many attributes (religion, gender, etc…), but often present in historical decisions. Use of such discriminatory historical decision making as training data can perpetuate discrimination, even if the protected attributes are not directly present in the data. This work focuses on discovering discrimination in instances and preventing discrimination in classification. First, we propose a discrimination discovery method based on modeling the probability distribution of a class using Bayesian networks. This measures the effect of a protected attribute (e.g., gender) in a subset of the dataset using the estimated probability distribution (via a Bayesian network). Second, we propose a classification method that corrects for the discovered discrimination without using protected attributes in the decision process. We evaluate the discrimination discovery and discrimination prevention approaches on two different datasets. The empirical results show that a substantial amount of discrimination identified in instances is prevented in future decisions.     

贝叶斯网络扩展研究综述

贝叶斯网络是一种能够对复杂不确定系统进行推理和建模的有效工具,广泛用于不确定决策、数据分析以及智能推理等领域.由于理论和实际的需要,贝叶斯网络不断扩展,出现了各种模型和研究方法.为此,综述了贝叶斯网络在不同领域的扩展模型以及在不同理论框架下的进展,并展望了未来的几个发展方向.     

Image interpretation using Bayesian networks

The problem of image interpretation is one of inference with the help of domain knowledge. In this paper, we formulate the problem as the maximum a posteriori (MAP) estimate of a properly defined probability distribution function (PDF). We show that a Bayesian network can be used to represent this PDF as well as the domain knowledge needed for interpretation. The Bayesian network may be relaxed to obtain the set of optimum interpretations     

Reforestation planning using Bayesian networks

The aim of this research was to construct a reforestation model for woodland located in the basin of the river Liébana (NW Spain). This is essentially a pattern recognition problem: the class labels are types of woodland, and the variables for each point are environmental coordinates (referring to altitude, slope, rainfall, lithology, etc.). The model trained using data for existing wooded areas will serve as a guideline for the reforestation of deforested areas. Nonetheless, with a view to tackling reforestation from a more informed perspective, of interest is an interpretable model of relationships existing not just between woodland type and environmental variables but also between and among the environmental variables themselves. For this reason we used Bayesian networks, as a tool that is capable of constructing a causal model of the relationships existing between all the variables represented in the model. The prediction results obtained were compared with those for classical linear techniques, neural networks and support vector machines.     

Using Bayesian networks with rule extraction to infer the risk of weed infestation in a corn-crop

This paper describes the modeling of a weed infestation risk inference system that implements a collaborative inference scheme based on rules extracted from two Bayesian network classifiers. The first Bayesian classifier infers a categorical variable value for the weed–crop competitiveness using as input categorical variables for the total density of weeds and corresponding proportions of narrow and broad-leaved weeds. The inferred categorical variable values for the weed–crop competitiveness along with three other categorical variables extracted from estimated maps for the weed seed production and weed coverage are then used as input for a second Bayesian network classifier to infer categorical variables values for the risk of infestation. Weed biomass and yield loss data samples are used to learn the probability relationship among the nodes of the first and second Bayesian classifiers in a supervised fashion, respectively. For comparison purposes, two types of Bayesian network structures are considered, namely an expert-based Bayesian classifier and a naïve Bayes classifier. The inference system focused on the knowledge interpretation by translating a Bayesian classifier into a set of classification rules. The results obtained for the risk inference in a corn-crop field are presented and discussed.     

Logistic model tree extraction from artificial neural networks.

Artificial neural networks (ANNs) are a powerful and widely used pattern recognition technique. However, they remain "black boxes" giving no explanation for the decisions they make. This paper presents a new algorithm for extracting a logistic model tree (LMT) from a neural network, which gives a symbolic representation of the knowledge hidden within the ANN. Landwehr's LMTs are based on standard decision trees, but the terminal nodes are replaced with logistic regression functions. This paper reports the results of an empirical evaluation that compares the new decision tree extraction algorithm with Quinlan's C4.5 and ExTree. The evaluation used 12 standard benchmark datasets from the University of California, Irvine machine-learning repository. The results of this evaluation demonstrate that the new algorithm produces decision trees that have higher accuracy and higher fidelity than decision trees created by both C4.5 and ExTree.     

New skeleton-based approaches for Bayesian structure learning of Bayesian networks

Automatically learning the graph structure of a single Bayesian network (BN) which accurately represents the underlying multivariate probability distribution of a collection of random variables is a challenging task. But obtaining a Bayesian solution to this problem based on computing the posterior probability of the presence of any edge or any directed path between two variables or any other structural feature is a much more involved problem, since it requires averaging over all the possible graph structures. For the former problem, recent advances have shown that search + score approaches find much more accurate structures if the search is constrained by a previously inferred skeleton (i.e. a relaxed structure with undirected edges which can be inferred using local search based methods). Based on similar ideas, we propose two novel skeleton-based approaches to approximate a Bayesian solution to the BN learning problem: a new stochastic search which tries to find directed acyclic graph (DAG) structures with a non-negligible score; and a new Markov chain Monte Carlo method over the DAG space. These two approaches are based on the same idea. In a first step, both employ a previously given skeleton and build a Bayesian solution constrained by this skeleton. In a second step, using the preliminary solution, they try to obtain a new Bayesian approximation but this time in an unconstrained graph space, which is the final outcome of the methods. As shown in the experimental evaluation, this new approach strongly boosts the performance of these two standard techniques proving that the idea of employing a skeleton to constrain the model space is also a successful strategy for performing Bayesian structure learning of BNs.     

A new uncertainty measure for belief networks with applications tooptimal evidential inferencing

We are concerned with the problem of measuring the uncertainty in a broad class of belief networks, as encountered in evidential reasoning applications. In our discussion, we give an explicit account of the networks concerned, and call them the Dempster-Shafer (D-S) belief networks. We examine the essence and the requirement of such an uncertainty measure based on well-defined discrete event dynamical systems concepts. Furthermore, we extend the notion of entropy for the D-S belief networks in order to obtain an improved optimal dynamical observer. The significance and generality of the proposed dynamical observer of measuring uncertainty for the D-S belief networks lie in that it can serve as a performance estimator as well as a feedback for improving both the efficiency and the quality of the D-S belief network-based evidential inferencing. We demonstrate, with Monte Carlo simulation, the implementation and the effectiveness of the proposed dynamical observer in solving the problem of evidential inferencing with optimal evidence node selection     

Bayesian wavelet networks for nonparametric regression

Radial wavelet networks have been proposed previously as a method for nonparametric regression. We analyze their performance within a Bayesian framework. We derive probability distributions over both the dimension of the networks and the network coefficients by placing a prior on the degrees of freedom of the model. This process bypasses the need to test or select a finite number of networks during the modeling process. Predictions are formed by mixing over many models of varying dimension and parameterization. We show that the complexity of the models adapts to the complexity of the data and produces good results on a number of benchmark test series.