A Bayesian method for learning belief networks that contain hidden variables

This paper presents a Bayesian method for computing the probability of a Bayesian belief-network structure from a database. In particular, the paper focuses on computing the probability of a belief-network structure that contains a hidden (latent) variable. A hidden variable represents a postulated entity that has not been directly measured. After reviewing related techniques, which previously were reported, this paper presents a new, more efficient method for handling hidden variables in belief networks.     

Uncertainty-aware soft sensor using Bayesian recurrent neural networks

Data-driven soft sensors have been widely used to measure key variables for industrial processes. Soft sensors using deep learning models have attracted considerable attention and shown superior predictive performance. However, if a soft sensor encounters an unexpected situation in inferring data or if noisy input data is used, the estimated value derived by a standard soft sensor using deep learning may at best be untrustworthy. This problem can be mitigated by expressing a degree of uncertainty about the trustworthiness of the estimated value produced by the soft sensor. To address this issue of uncertainty, we propose using an uncertainty-aware soft sensor that uses Bayesian recurrent neural networks (RNNs). The proposed soft sensor uses a RNN model as a backbone and is then trained using Bayesian techniques. The experimental results demonstrated that such an uncertainty-aware soft sensor increases the reliability of predictive uncertainty. In comparisons with a standard soft sensor, it shows a capability to use uncertainties for interval prediction without compromising predictive performance.     

贝叶斯网络扩展研究综述

贝叶斯网络是一种能够对复杂不确定系统进行推理和建模的有效工具,广泛用于不确定决策、数据分析以及智能推理等领域.由于理论和实际的需要,贝叶斯网络不断扩展,出现了各种模型和研究方法.为此,综述了贝叶斯网络在不同领域的扩展模型以及在不同理论框架下的进展,并展望了未来的几个发展方向.     

基于预测关系的贝叶斯网络学习算法

在介绍有代表性的贝叶斯网络结构学习算法基础上,给出了变量之间预测能力的概念及估计方法,并证明了预测能力就是预测正确率,在此基础上建立了基于变量之间预测关系的贝叶斯网络结构学习方法,并使用模拟数据进行了对比实验,实验结果显示该算法能够有效地进行贝叶斯网络结构学习。     

BayesChess: A computer chess program based on Bayesian networks

In this paper, we introduce a chess program able to adapt its game strategy to its opponent, as well as to adapt the evaluation function that guides the search process according to its playing experience. The adaptive and learning abilities have been implemented through Bayesian networks. We show how the program learns through an experiment consisting on a series of games that point out that the results improve after the learning stage.     

贝叶斯学习,贝叶斯网络与数据采掘

自从50～60年代贝叶斯学派形成后,关于贝叶斯分析的研究久盛不衰。早在80年代,贝叶斯网络就成功地应用于专家系统,成为表示不确定性专家知识和推理的一种流行方法。90年代以来,贝叶斯学习一直是机器学习研究的重要方向。由于概率统计与数据采掘的     

贝叶斯网络结构学习综述

贝叶斯网络是一种有效的不确定性知识表达和推理工具,在数据挖掘等领域得到了较好的应用,而结构学习是其重要研究内容之一.经过二十多年的发展,已经出现了一些比较成熟的贝叶斯网络结构学习算法,对迄今为止的贝叶斯网络结构学习方法进行了综述.现阶段获得的用于结构学习的观测数据都比较复杂,这些数据分为完备数据和不完备数据两种类型.针对完备数据,分别从基于依赖统计分析的方法、基于评分搜索的方法和混合搜索方法三个方面对已有的算法进行分析.对于不完备数据,给出了数据不完备情况下网络结构的学习框架.在此基础上归纳总结了贝叶斯网络结构学习各个方向的研究进展,给出了贝叶斯网络结构学习未来可能的研究方向.     

SR-30 turbojet engine real-time sensor health monitoring using neural networks,and Bayesian belief networks

This paper describes the use of artificial intelligence-based techniques for detecting and isolating sensor failures in a turbojet engine. Specifically, three artificial intelligence (AI) techniques are employed: artificial neural networks (NNs), statistical expectations, and Bayesian belief networks (BBNs). These techniques are combined into an overall system that is capable of distinguishing between sensor failure and engine failure—a critical capability in the operation of turbojet engines. The turbojet engine used in this study is an SR-30 developed by Turbine Technologies. Initially, NNs were designed and trained to recognize sensor failure in the engine. The increased random noise output from failing sensors was used as the key indicator. Next, a Bayesian statistical method was used to recognize sensor failure based on the bias error occurring in the sensors. Finally, a BBN was developed to interpret the results of the NN and statistical evaluations. The BBN determines whether single or multiple sensor failures signify engine failure, or whether sensor failures represent separate, unrelated incidences. The BBN algorithm is also used to distinguish between bias and noise errors on sensors used to monitor turbojet performance. The overall system is demonstrated to work equally well during start-up and main-stage operation of the engine. Results show that the method can efficiently detect and isolate single or multiple sensor failures within this dynamic environment.     

动态贝叶斯网络在战术态势估计中的应用*

针对战术态势估计的特点和要求,分析和建立了应用于态势估计的动态贝叶斯网络模型。该模型以离散变量集为研究对象。由于该动态贝叶斯网络满足Markovian特性和平稳特性,降低了网络的复杂度。相比较于贝叶斯网络模型,该动态贝叶斯网络模型考虑了时序因素,将前时刻的态势因素作为当前时刻态势估计的证据的一部分,并能对下一时刻的态势进行预测。文中采用集树（junction tree）算法,利用相关的贝叶斯网络推理软件进行了实验,实验结果表明基于动态贝叶斯网络的估计结果较贝叶斯网络的估计结果好,验证了该模型的有效性。     

Empirical analysis of an on-line adaptive system using a mixture of Bayesian networks

An on-line reinforcement learning system that adapts to environmental changes using a mixture of Bayesian networks is described. Building intelligent systems able to adapt to dynamic environments is important for deploying real-world applications. Machine learning approaches, such as those using reinforcement learning methods and stochastic models, have been used to acquire behavior appropriate to environments characterized by uncertainty. However, efficient hybrid architectures based on these approaches have not yet been developed. The results of several experiments demonstrated that an agent using the proposed system can flexibly adapt to various kinds of environmental changes.     

面向大规模数据集的贝叶斯网络参数学习算法

贝叶斯网络的学习可以分为结构学习和参数学习。期望最大化（EM）算法通常用于不完整数据的参数学习,但是由于EM算法计算相对复杂,存在收敛速度慢和容易局部最大化等问题,传统的EM算法难于处理大规模数据集。研究了EM算法的主要问题,采用划分数据块的方法将大规模数据集划分为小的样本集来处理,降低了EM算法的计算量,同时也提高了计算精度。实验证明,该改进的EM算法具有较高的性能。     

Comparative analysis of discretization methods in Bayesian networks

A key step in implementing Bayesian networks (BNs) is the discretization of continuous variables. There are several mathematical methods for constructing discrete distributions, the implications of which on the resulting model has not been discussed in literature. Discretization invariably results in loss of information, and both the discretization method and the number of intervals determines the level of such loss. We designed an experiment to evaluate the impact of commonly used discretization methods and number of intervals on the developed BNs. The conditional probability tables, model predictions, and management recommendations were compared and shown to be different among models. However, none of the models did uniformly well in all comparison criteria. As we cannot justify using one discretization method against others, we recommend caution when discretization is used, and a verification process that includes evaluating alternative methods to ensure that the conclusions are not an artifact of the discretization approach.     

Interactive structural learning of Bayesian networks

We propose an hybrid approach for structure learning of Bayesian networks, in which a computer system and a human expert cooperate to search for the best structure. The system builds an initial tree structure which is graphically presented to the expert, and then the expert can modify this structure according to his knowledge of the domain. The system has several tools for aiding the human in this task: it allows for graphical editing (adding, deleting, inverting arcs) of the network, it shows graphically the correlation between variables, and it gives a measure of the quality and complexity for each structure. A measure which combines both quality and complexity, that we call quality, is defined. We have tested the tool in two domains: atmospheric pollution and car insurance, with good results.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

Unravelling infectious disease eco-epidemiology using Bayesian networks and scenario analysis: A case study of leptospirosis in Fiji

Regression models are the standard approaches used in infectious disease epidemiology, but have limited ability to represent causality or complexity. We explore Bayesian networks (BNs) as an alternative approach for modelling infectious disease transmission, using leptospirosis as an example. Data were obtained from a leptospirosis study in Fiji in 2013. We compared the performance of naïve versus expert-structured BNs for modelling the relative importance of animal species in disease transmission in different ethnic groups and residential settings. For BNs of animal exposures at the individual/household level, R² for predicted versus observed infection rates were 0.59 for naïve and 0.75–0.93 for structured models of ethnic groups; and 0.54 for naïve and 0.93–1.00 for structured models of residential settings. BNs provide a promising approach for modelling infectious disease transmission under complex scenarios. The relative importance of animal species varied between subgroups, with important implications for more targeted public health control strategies.     

基于因果语义定向的贝叶斯网络结构学习

基于变量之间基本依赖关系、基本结构、d-separation标准、依赖分析思想和混合定向策略,给出了一种有效实用的贝叶斯网络结构学习方法,不需要结点有序,并能避免打分-搜索方法存在的指数复杂性,以及现有依赖分析方法的大量高维条件概率计算等问题。     

Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks

In many multivariate domains, we are interested in analyzing the dependency structure of the underlying distribution, e.g., whether two variables are in direct interaction. We can represent dependency structures using Bayesian network models. To analyze a given data set, Bayesian model selection attempts to find the most likely (MAP) model, and uses its structure to answer these questions. However, when the amount of available data is modest, there might be many models that have non-negligible posterior. Thus, we want compute the Bayesian posterior of a feature, i.e., the total posterior probability of all models that contain it. In this paper, we propose a new approach for this task. We first show how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed order over network variables. This allows us to compute, for a given order, both the marginal probability of the data and the posterior of a feature. We then use this result as the basis for an algorithm that approximates the Bayesian posterior of a feature. Our approach uses a Markov Chain Monte Carlo (MCMC) method, but over orders rather than over network structures. The space of orders is smaller and more regular than the space of structures, and has much a smoother posterior landscape. We present empirical results on synthetic and real-life datasets that compare our approach to full model averaging (when possible), to MCMC over network structures, and to a non-Bayesian bootstrap approach.     

Learning bipartite Bayesian networks under monotonicity restrictions

ABSTRACT

Learning parameters of a probabilistic model is a necessary step in machine learning tasks. We present a method to improve learning from small datasets by using monotonicity conditions. Monotonicity simplifies the learning and it is often required by users. We present an algorithm for Bayesian Networks parameter learning. The algorithm and monotonicity conditions are described, and it is shown that with the monotonicity conditions we can better fit underlying data. Our algorithm is tested on artificial and empiric datasets. We use different methods satisfying monotonicity conditions: the proposed gradient descent, isotonic regression EM, and non-linear optimization. We also provide results of unrestricted EM and gradient descent methods. Learned models are compared with respect to their ability to fit data in terms of log-likelihood and their fit of parameters of the generating model. Our proposed method outperforms other methods for small sets, and provides better or comparable results for larger sets.