New skeleton-based approaches for Bayesian structure learning of Bayesian networks

Automatically learning the graph structure of a single Bayesian network (BN) which accurately represents the underlying multivariate probability distribution of a collection of random variables is a challenging task. But obtaining a Bayesian solution to this problem based on computing the posterior probability of the presence of any edge or any directed path between two variables or any other structural feature is a much more involved problem, since it requires averaging over all the possible graph structures. For the former problem, recent advances have shown that search + score approaches find much more accurate structures if the search is constrained by a previously inferred skeleton (i.e. a relaxed structure with undirected edges which can be inferred using local search based methods). Based on similar ideas, we propose two novel skeleton-based approaches to approximate a Bayesian solution to the BN learning problem: a new stochastic search which tries to find directed acyclic graph (DAG) structures with a non-negligible score; and a new Markov chain Monte Carlo method over the DAG space. These two approaches are based on the same idea. In a first step, both employ a previously given skeleton and build a Bayesian solution constrained by this skeleton. In a second step, using the preliminary solution, they try to obtain a new Bayesian approximation but this time in an unconstrained graph space, which is the final outcome of the methods. As shown in the experimental evaluation, this new approach strongly boosts the performance of these two standard techniques proving that the idea of employing a skeleton to constrain the model space is also a successful strategy for performing Bayesian structure learning of BNs.     

贝叶斯网络结构学习综述

贝叶斯网络是一种有效的不确定性知识表达和推理工具,在数据挖掘等领域得到了较好的应用,而结构学习是其重要研究内容之一.经过二十多年的发展,已经出现了一些比较成熟的贝叶斯网络结构学习算法,对迄今为止的贝叶斯网络结构学习方法进行了综述.现阶段获得的用于结构学习的观测数据都比较复杂,这些数据分为完备数据和不完备数据两种类型.针对完备数据,分别从基于依赖统计分析的方法、基于评分搜索的方法和混合搜索方法三个方面对已有的算法进行分析.对于不完备数据,给出了数据不完备情况下网络结构的学习框架.在此基础上归纳总结了贝叶斯网络结构学习各个方向的研究进展,给出了贝叶斯网络结构学习未来可能的研究方向.     

基于领域知识的贝叶斯网络结构学习算法

针对SEM算法在缺省数据学习中存在精度偏低和收敛速度缓慢的问题,通过将领域知识引入到SEM算法中,提出了KB-SEM算法,该算法首先用D-S证据理论综合领域知识,然后将采集的知识以禁忌表的方式嵌入SEM中来限制和引导算法的搜索路径,缩小算法的搜索空间。实验表明,KB-SEM算法能有效地提高算法的学习精度和时间性能,且能在一定程度上避免主观偏见和数据噪音的干扰。     

Inductive transfer for learning Bayesian networks

In several domains it is common to have data from different, but closely related problems. For instance, in manufacturing, many products follow the same industrial process but with different conditions; or in industrial diagnosis, where there is equipment with similar specifications. In these cases it is common to have plenty of data for some scenarios but very little for others. In order to learn accurate models for rare cases, it is desirable to use data and knowledge from similar cases; a technique known as transfer learning. In this paper we propose an inductive transfer learning method for Bayesian networks, that considers both structure and parameter learning. For structure learning we use conditional independence tests, by combining measures from the target task with those obtained from one or more auxiliary tasks, using a novel weighted sum of the conditional independence measures. For parameter learning, we propose two variants of the linear pool for probability aggregation, combining the probability estimates from the target task with those from the auxiliary tasks. To validate our approach, we used three Bayesian networks models that are commonly used for evaluating learning techniques, and generated variants of each model by changing the structure as well as the parameters. We then learned one of the variants with a small dataset and combined it with information from the other variants. The experimental results show a significant improvement in terms of structure and parameters when we transfer knowledge from similar tasks. We also evaluated the method with real-world data from a manufacturing process considering several products, obtaining an improvement in terms of log-likelihood between the data and the model when we do transfer learning from related products.     

Automatic real-time event detection for seismic networks

A low-power microprocessor-based seismic signal detection system has been developed for monitoring and recording earthquakes detected from multichannel seismic network data. The system operates on-line to the network and may be battery powered for field applications or as a precaution against power failures. The algorithm is designed to cope with varying noise conditions at the network outstations and detects local, regional, and teleseismic events. The sampling rate for each channel is 100 samples/sec. with 10-bit accuracy. The system is operating with the LOWNET network of BGS and is suitable for large or small aperture networks because 20 sec of data for each channel are stored in memory. The system is designed to detect seismic events for recording on either analogue or digital recorders depending on the requirements and preferences of the network management.     

Pattern-based event detection in sensor networks

Many applications of wireless sensor networks monitor the physical world and report events of interest. To facilitate event detection in these applications, in this paper we propose a pattern-based event detection approach and integrate the approach into an in-network sensor query processing framework. Different from existing threshold-based event detection, we abstract events into patterns in sensory data and convert the problem of event detection into a pattern matching problem. We focus on applying single-node temporal patterns, and define the general patterns as well as five types of basic patterns for event specification. Considering the limited storage on sensor nodes, we design an on-node cache manager to maintain the historical data required for pattern matching and develop event-driven processing techniques for queries in our framework. We have conducted experiments using patterns for events that are extracted from real-world datasets. The results demonstrate the effectiveness and efficiency of our approach.     

Bayesian learning of Bayesian networks with informative priors

This paper presents and evaluates an approach to Bayesian model averaging where the models are Bayesian nets (BNs). A comprehensive study of the literature on structural priors for BNs is conducted. A number of prior distributions are defined using stochastic logic programs and the MCMC Metropolis-Hastings algorithm is used to (approximately) sample from the posterior. We use proposals which are tightly coupled to the priors which give rise to cheaply computable acceptance probabilities. Experiments using data generated from known BNs have been conducted to evaluate the method. The experiments used 6 different BNs and varied: the structural prior, the parameter prior, the Metropolis-Hasting proposal and the data size. Each experiment was repeated three times with different random seeds to test the robustness of the MCMC-produced results. Our results show that with effective priors (i) robust results are produced and (ii) informative priors improve results significantly.     

基于字典学习的网络社团结构探测算法

复杂网络中的社团结构探测对于理解网络的拓扑结构和功能有重要的意义.本文将字典学习方法应用到社团结构探测问题中,给出一种新的字典学习方法,并将其和其他几种流行的模型与算法作了系统比较.在三种类型的人工数据和来自不同领域的实际数据上的实验结果表明,本文所提出的算法在社团结构探测问题上是非常有效的,具有算法简单、收敛速度快、计算精度高等特点.     

Community structure detection in social networks based on dictionary learning

Discovering community structures is a fundamental problem concerning how to understand the topology and the functions of complex network. In this paper, we propose how to apply dictionary learning algorithm to community structure detection. We present a new dictionary learning algorithm and systematically compare it with other state-of-the-art models/algorithms. The results show that the proposed algorithm is highly effectively at finding the community structures in both synthetic datasets, including three types of data structures, and real world networks coming from different areas.     

A new genetic approach for structure learning of Bayesian networks: Matrix genetic algorithm

In this paper, a novel method for structure learning of a Bayesian network (BN) is developed. A new genetic approach called the matrix genetic algorithm (MGA) is proposed. In this method, an individual structure is represented as a matrix chromosome and each matrix chromosome is encoded as concatenation of upper and lower triangular parts. The two triangular parts denote the connection in the BN structure. Further, new genetic operators are developed to implement the MGA. The genetic operators are closed in the set of the directed acyclic graph (DAG). Finally, the proposed scheme is applied to real world and benchmark applications, and its effectiveness is demonstrated through computer simulation.     

Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks

Many optimization problems are what can be called globally multimodal, i.e., they present several global optima. Unfortunately, this is a major source of difficulties for most estimation of distribution algorithms, making their effectiveness and efficiency degrade, due to genetic drift. With the aim of overcoming these drawbacks for discrete globally multimodal problem optimization, this paper introduces and evaluates a new estimation of distribution algorithm based on unsupervised learning of Bayesian networks. We report the satisfactory results of our experiments with symmetrical binary optimization problems.     

Interactive structural learning of Bayesian networks

We propose an hybrid approach for structure learning of Bayesian networks, in which a computer system and a human expert cooperate to search for the best structure. The system builds an initial tree structure which is graphically presented to the expert, and then the expert can modify this structure according to his knowledge of the domain. The system has several tools for aiding the human in this task: it allows for graphical editing (adding, deleting, inverting arcs) of the network, it shows graphically the correlation between variables, and it gives a measure of the quality and complexity for each structure. A measure which combines both quality and complexity, that we call quality, is defined. We have tested the tool in two domains: atmospheric pollution and car insurance, with good results.