The Sample Complexity of Learning Fixed-Structure Bayesian Networks

We consider the problem of PAC learning probabilistic networks in the case where the structure of the net is specified beforehand. We allow the conditional probabilities to be represented in any manner (as tables or specialized functions) and obtain sample complexity bounds for learning nets with and without hidden nodes.     

Data categorization using decision trellises

We introduce a probabilistic graphical model for supervised learning on databases with categorical attributes. The proposed belief network contains hidden variables that play a role similar to nodes in decision trees and each of their states either corresponds to a class label or to a single attribute test. As a major difference with respect to decision trees, the selection of the attribute to be tested is probabilistic. Thus, the model can be used to assess the probability that a tuple belongs to some class, given the predictive attributes. Unfolding the network along the hidden states dimension yields a trellis structure having a signal flow similar to second order connectionist networks. The network encodes context specific probabilistic independencies to reduce parametric complexity. We present a custom tailored inference algorithm and derive a learning procedure based on the expectation-maximization algorithm. We propose decision trellises as an alternative to decision trees in the context of tuple categorization in databases, which is an important step for building data mining systems. Preliminary experiments on standard machine learning databases are reported, comparing the classification accuracy of decision trellises and decision trees induced by C4.5. In particular, we show that the proposed model can offer significant advantages for sparse databases in which many predictive attributes are missing     

混合贝叶斯网络隐藏变量学习研究

目前，具有已知结构的隐藏变量学习主要针对具有离散变量的贝叶斯网和具有连续变量的高斯网．该文给出了具有连续和离散变量的混合贝叶斯网络隐藏变量学习方法．该方法不需要离散化连续变量，依据专业知识或贝叶斯网络道德图中Cliques的维数发现隐藏变量的位置，基于依赖结构（星形结构或先验结构）和Gibbs抽样确定隐藏变量的值，结合扩展的MDL标准和统计方法发现隐藏变量的最优维数．实验结果表明，这种方法能够有效地进行具有已知结构的混合贝叶斯网络隐藏变量学习．     

Factorial Hidden Markov Models

Hidden Markov models (HMMs) have proven to be one of the most widely used tools for learning probabilistic models of time series data. In an HMM, information about the past is conveyed through a single discrete variable—the hidden state. We discuss a generalization of HMMs in which this state is factored into multiple state variables and is therefore represented in a distributed manner. We describe an exact algorithm for inferring the posterior probabilities of the hidden state variables given the observations, and relate it to the forward–backward algorithm for HMMs and to algorithms for more general graphical models. Due to the combinatorial nature of the hidden state representation, this exact algorithm is intractable. As in other intractable systems, approximate inference can be carried out using Gibbs sampling or variational methods. Within the variational framework, we present a structured approximation in which the the state variables are decoupled, yielding a tractable algorithm for learning the parameters of the model. Empirical comparisons suggest that these approximations are efficient and provide accurate alternatives to the exact methods. Finally, we use the structured approximation to model Bach's chorales and show that factorial HMMs can capture statistical structure in this data set which an unconstrained HMM cannot.     

A Bayesian Method for the Induction of Probabilistic Networks from Data

This paper presents a Bayesian method for constructing probabilistic networks from databases. In particular, we focus on constructing Bayesian belief networks. Potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems. We extend the basic method to handle missing data and hidden (latent) variables. We show how to perform probabilistic inference by averaging over the inferences of multiple belief networks. Results are presented of a preliminary evaluation of an algorithm for constructing a belief network from a database of cases. Finally, we relate the methods in this paper to previous work, and we discuss open problems.     

Construction of a new adaptive wavelet network and its learning algorithm

A new adaptive learning algorithm for constructing and training wavelet networks is proposed based on the time-frequency localization properties of wavelet frames and the adaptive projection algorithm. The exponential convergence of the adaptive projection algorithm in finite-dimensional Hilbert spaces is constructively proved, with exponential decay ratios given with high accuracy. The learning algorithm can sufficiently utilize the time-frequency information contained in the training data, iteratively determines the number of the hidden layer nodes and the weights of wavelet networks, and solves the problem of structure optimization of wavelet networks. The algorithm is simple and efficient, as illustrated by examples of signal representation and denoising.     

基于TAN结构的启发式贝叶斯网络结构学习算法

贝叶斯网络结构学习是个NP难题。一种有效且准确性较高的学习算法是K2算法。但K2算法要确定结点次序,在无先验信息时受到很大限制。提出了一种启发式结构学习G算法,该算法以学习树扩展朴素贝叶斯TAN结构作为启发式信息,由该启发式信息生成结点次序,再用K2算法生成贝叶斯网络结构。实验结果表明,G算法可以解决无先验信息时确定结点次序的问题。所添加的弧比较简洁,网络结构比TAN结构更加合理。     

基于GPU的复杂网络社区挖掘算法并行计算

由于复杂网络的规模越来越大, 在大规模的复杂网络中快速、准确地挖掘出隐藏的社区结构是当前该领域研究的热点问题。目前社区结构挖掘常用的基于快速Newman算法的社区结构挖掘算法之一是一般概率框架方法。以规模日益增大的复杂网络为研究对象, 提出了基于GPGPU的一般概率框架并行算法, 有效地解决了在大规模的复杂网络中快速、准确地挖掘出隐藏的社区结构问题。实验证明, 随着节点数的增加, 该并行算法在不损失准确性的前提下运行效率有所提高, 为复杂网络社区结构挖掘的研究提供了一种高效的解决方案。     

Model gene network by semi-fixed Bayesian network

Gene networks describe functional pathways in a given cell or tissue, representing processes such as metabolism, gene expression regulation, and protein or RNA transport. Thus, learning gene network is a crucial problem in the post genome era. Most existing works learn gene networks by assuming one gene provokes the expression of another gene directly leading to an over-simplified model. In this paper, we show that the gene regulation is a complex problem with many hidden variables. We propose a semi-fixed model to represent the gene network as a Bayesian network with hidden variables. In addition, an effective algorithm based on semi-fixed structure learning is proposed to learn the model. Experimental results and comparison with the-state-of-the-art learning algorithms on artificial and real-life datasets confirm the effectiveness of our approach.     

基于互信息的分步式输入变量选择多元序列预测研究

针对多元序列分析中存在的输入变量选择问题,提出一种基于k-!近邻互信息估计的分步式变量选择算法. 该算法通过两步过程分别实现相关变量的选择与弱相关变量的剔除. 同时将分步变量选择算法应用于径向基函数(Radial basis function, RBF) 神经网络结构的优化中.在K均值聚类的基础上,通过分析隐含层神经元的输出权值与神经网络输出的相关性, 对隐含层节点进行选择,改进网络的结构与性能. Friedman数据的仿真实验验证了分步变量选择算法的有效性; Gas furnace多元时间序列以及Boston housing数据的仿真结果表明, 优化后的RBF网络能够在保证模型精度的基础上有效控制网络规模.