基于Bayes参数估计的垃圾邮件过滤算法研究

朴素Bayes邮件过滤算法由于简单、易于理解,已被人们广泛接受,并应用到一些商用邮件系统当中.但面对目前垃圾邮件问题依然严重的现状,人们逐渐开始认识到采用简单的朴素Bayes邮件过滤算法已不能满足现有邮件过滤的性能要求.Bayes网络一直以来作为知识发现的一个重要分支,是人们研究的热点;邮件过滤问题也可以映射到一个Bayes决策网络模型中.通过构建针对邮件过滤的Bayes决策网络模型,并经过概率学习对关键节点作Bayes参数估计,可以实现邮件的概率分类发现.邮件样本试验结果表明新算法与朴素Bayes邮件过滤算法相比具有更快的收敛速度和更高的稳定性.     

分组样本下Bayes网络条件概率的学习算法

本文应用带盘的Bayes网络作为分析模型，对于学习实例数据库为分组样本的关于各组样本私有条件概率的学习逄法进行讨论，构建出两层学习结构：第一层针对各组私有条件概率分布Θij(l)s的学习；第二层针对是各组公有条件概率分布Θij的学习，算法在综合公有后验条件概率分布和本组学习实例数据的基础上，实现对各组私有条件概率分布的学习，并可以通过调整组间差异性信度β值来改变综合值中共性和个性的比例。     

基于PCA-WBayes的WiFi室内定位算法

针对朴素贝叶斯(Bayes)室内定位算法忽略各无线接入点(AP)信号间的相关性,最终导致定位精度损失这一不足,提出一种基于主成分分析结合加权Bayes(PCA-WBayes)的WiFi室内定位算法.在离线阶段采集参考点处来自各AP的WiFi信号强度,运用PCA进行去相关性、降维处理,提取主要数据特征,结合参考点位置坐标,构建位置指纹数据库;在线定位阶段,在位置指纹数据库中匹配待测点的信号特征,通过Bayes算法估算待测点位置,获取前w个后验概率最大的参考点坐标,按后验概率分配权重,以加权结果作为待测点位置.实验结果表明:相比K最近邻(KNN)、Bayes等常用WiFi室内定位算法,PCA-WBayes算法定位误差更小,将朴素Bayes室内定位算法的精度提升了15.44％.     

基于CSI改进的WKNN室内定位方法

传统的加权K最近邻算法中以距离作为权值,随着数据维度的增加,计算距离与真实距离的误差越来越大。针对这一问题,提出了一种贝叶斯后验概率的加权K最近邻算法——贝叶斯后验概率(Bayes ian Posterior Probability-Weighted K-Nearest Neighbor,BPP-WKNN)方法。首先用支持向量机算法分类选取测试点的近邻指纹点,其次计算测试点到每个近邻指纹点的贝叶斯后验概率,最后以贝叶斯后验概率的大小作为权值进行BPPWKNN算法定位。实验果表明:与基于曼哈顿距离的加权K最近邻算法和基于欧氏距离的加权K最近邻算法相比,改进后的BPP-WKNN定位算法的定位精确度和稳定性更高;利用支持向量机算法的稀疏性定位完成时间分别缩短了49%与42%。     

基于Bayes决策的联机手写数字识别

本文介绍了贝叶斯（Bayes）决策理论和模式识别的过程,并用Bayes分类算法设计了一个图像分类器。该分类器通过对各个类别的后验概率的计算．对0-9这十类联机手写数字进行识别实验。     

集成学习算法在不平衡分类中的应用研究

提出一种基于训练集分解的不平衡分类算法,该算法使用能输出后验概率的支持向量机作为分类器,使用基于测度层次信息源合并规则实现分类器的集成.在4个不同领域的不平衡数据集上的仿真实验表明:该算法有效提高分类器对正类样本的正确率,同时尽量减少对负类样本的误判.实验结果验证集成学习算法处理不平衡分类问题的有效性.     

并行的贝叶斯网络参数学习算法

针对大样本条件下EM算法学习贝叶斯网络参数的计算问题,提出一种并行EM算法(Parallel EM,PL-EM)提高大样本条件下复杂贝叶斯网络参数学习的速度.PL-EM算法在E步并行计算隐变量的后验概率和期望充分统计因子;在M步,利用贝叶斯网络的条件独立性和完整数据集下的似然函数可分解性,并行计算各个局部似然函数.实验结果表明PL-EM为解决大样本条件下贝叶斯网络参数学习提供了一种有效的方法.     

基于样本条件价值改进的 Co-training 算法

Co-training是一种主流的半监督学习算法. 该算法中两视图下的分类器通过迭代的方式, 互为对方从无标记样本集中挑选新增样本, 以更新对方训练集. Co-training以分类器的后验概率输出作为新增样本的挑选策略, 该策略忽略了样本对于当前分类器的价值. 针对该问题, 本文提出一种改进的Co-training式算法—CVCOT (Conditional value-based co-training), 即采用基于样本条件价值的挑选策略来优化Co-training. 通过定义无标记样本的条件价值, 各视图下的分类器以样本条件价值为依据来挑选新增样本, 以此更新训练集. 该策略既可保证新增样本的标记可靠性, 又能优先将价值较高的富信息样本补充到训练集中, 可以有效地优化分类器. 在UCI数据集和网页分类应用上的实验结果表明: CVCOT具有较好的分类性能和学习效率.     

后验概率估计及其应用:基于核Logistic回归的方法

提出一种基于特征矢量集的核Logistic回归方法,解决核Logistic回归的解的稀疏性问题,降低后验概率估计的计算复杂度.该方法与Markov随机场方法相结合,应用到图像分割中.在Bayes公式中,对样本条件概率的估计转换为对核Logistic回归方法的后验概率的估计,从而提出一种新的Markov随机场模型的实现方法,在对纹理图像的分割实验中得到良好效果.     

动态神经网络分类器主动学习算法及其智能控制应用

针对动态神经网络分类器训练时采样时间长、计算量大的问题,提出一种动态神经网络分类器的主动学习算法。根据主动学习AL(Active Learning)算法中一种改进型不确定性采样策略,综合考虑样本的后验概率及其与已标记样本间的相似性,标注综合评价得分值较小的样本,将其用于对网络分类器的训练。通过Sobol’敏感度分析法,神经网络适时地增加敏感度值较大或删减敏感度值较小的隐层神经元,以提高其学习速率,减小输出误差。分类器训练仿真实验结果表明,与被动学习算法相比,该算法能够大大缩短网络分类器训练时间,降低其输出误差。将该算法用于液压AGC系统中,实验结果表明,该算法可实现系统中PID控制器参数的在线调节,提高了厚度控制精度,以此验证了该算法的适用性。     

Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

We describe a Bayesian approach for learning Bayesian networks from a combination of prior knowledge and statistical data. First and foremost, we develop a methodology for assessing informative priors needed for learning. Our approach is derived from a set of assumptions made previously as well as the assumption of likelihood equivalence, which says that data should not help to discriminate network structures that represent the same assertions of conditional independence. We show that likelihood equivalence when combined with previously made assumptions implies that the user's priors for network parameters can be encoded in a single Bayesian network for the next case to be seen—a prior network—and a single measure of confidence for that network. Second, using these priors, we show how to compute the relative posterior probabilities of network structures given data. Third, we describe search methods for identifying network structures with high posterior probabilities. We describe polynomial algorithms for finding the highest-scoring network structures in the special case where every node has at most k = 1 parent. For the general case (k > 1), which is NP-hard, we review heuristic search algorithms including local search, iterative local search, and simulated annealing. Finally, we describe a methodology for evaluating Bayesian-network learning algorithms, and apply this approach to a comparison of various approaches.     

Restricted Bayesian classification networks

Bayesian networks are graphical models that describe dependency relationships between variables, and are powerful tools for studying probability classifiers. At present, the causal Bayesian network learning method is used in constructing Bayesian network classifiers while the contribution of attribute to class is over-looked. In this paper, a Bayesian network specifically for classification-restricted Bayesian classification networks is proposed. Combining dependency analysis between variables, classification accuracy evaluation criteria and a search algorithm, a learning method for restricted Bayesian classification networks is presented. Experiments and analysis are done using data sets from UCI machine learning repository. The results show that the restricted Bayesian classification network is more accurate than other well-known classifiers.     

An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach

Given the explosive growth of data collected from current business environment, data mining can potentially discover new knowledge to improve managerial decision making. This paper proposes a novel data mining approach that employs an evolutionary algorithm to discover knowledge represented in Bayesian networks. The approach is applied successfully to handle the business problem of finding response models from direct marketing data. Learning Bayesian networks from data is a difficult problem. There are two different approaches to the network learning problem. The first one uses dependency analysis, while the second one searches good network structures according to a metric. Unfortunately, both approaches have their own drawbacks. Thus, we propose a novel hybrid algorithm of the two approaches, which consists of two phases, namely, the conditional independence (CI) test and the search phases. In the CI test phase, dependency analysis is conducted to reduce the size of the search space. In the search phase, good Bayesian network models are generated by using an evolutionary algorithm. A new operator is introduced to further enhance the search effectiveness and efficiency. In a number of experiments and comparisons, the hybrid algorithm outperforms MDLEP, our previous algorithm which uses evolutionary programming (EP) for network learning, and other network learning algorithms. We then apply the approach to two data sets of direct marketing and compare the performance of the evolved Bayesian networks obtained by the new algorithm with those by MDLEP, the logistic regression models, the na/spl inodot//spl uml/ve Bayesian classifiers, and the tree-augmented na/spl inodot//spl uml/ve Bayesian network classifiers (TAN). In the comparison, the new algorithm outperforms the others.     

New skeleton-based approaches for Bayesian structure learning of Bayesian networks

Automatically learning the graph structure of a single Bayesian network (BN) which accurately represents the underlying multivariate probability distribution of a collection of random variables is a challenging task. But obtaining a Bayesian solution to this problem based on computing the posterior probability of the presence of any edge or any directed path between two variables or any other structural feature is a much more involved problem, since it requires averaging over all the possible graph structures. For the former problem, recent advances have shown that search + score approaches find much more accurate structures if the search is constrained by a previously inferred skeleton (i.e. a relaxed structure with undirected edges which can be inferred using local search based methods). Based on similar ideas, we propose two novel skeleton-based approaches to approximate a Bayesian solution to the BN learning problem: a new stochastic search which tries to find directed acyclic graph (DAG) structures with a non-negligible score; and a new Markov chain Monte Carlo method over the DAG space. These two approaches are based on the same idea. In a first step, both employ a previously given skeleton and build a Bayesian solution constrained by this skeleton. In a second step, using the preliminary solution, they try to obtain a new Bayesian approximation but this time in an unconstrained graph space, which is the final outcome of the methods. As shown in the experimental evaluation, this new approach strongly boosts the performance of these two standard techniques proving that the idea of employing a skeleton to constrain the model space is also a successful strategy for performing Bayesian structure learning of BNs.     

A heuristic method for learning Bayesian networks using discrete particle swarm optimization

Bayesian networks are a powerful approach for representing and reasoning under conditions of uncertainty. Many researchers aim to find good algorithms for learning Bayesian networks from data. And the heuristic search algorithm is one of the most effective algorithms. Because the number of possible structures grows exponentially with the number of variables, learning the model structure from data by considering all possible structures exhaustively is infeasible. PSO (particle swarm optimization), a powerful optimal heuristic search algorithm, has been applied in various fields. Unfortunately, the classical PSO algorithm only operates in continuous and real-valued space, and the problem of Bayesian networks learning is in discrete space. In this paper, two modifications of updating rules for velocity and position are introduced and a Bayesian networks learning based on binary PSO is proposed. Experimental results show that it is more efficient because only fewer generations are needed to obtain optimal Bayesian networks structures. In the comparison, this method outperforms other heuristic methods such as GA (genetic algorithm) and classical binary PSO.     

Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm

Structure learning of Bayesian networks is a well-researched but computationally hard task. We present an algorithm that integrates an information-theory-based approach and a scoring-function-based approach for learning structures of Bayesian networks. Our algorithm also makes use of basic Bayesian network concepts like d-separation and condition independence. We show that the proposed algorithm is capable of handling networks with a large number of variables. We present the applicability of the proposed algorithm on four standard network data sets and also compare its performance and computational efficiency with other standard structure-learning methods. The experimental results show that our method can efficiently and accurately identify complex network structures from data.     

Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks

In many multivariate domains, we are interested in analyzing the dependency structure of the underlying distribution, e.g., whether two variables are in direct interaction. We can represent dependency structures using Bayesian network models. To analyze a given data set, Bayesian model selection attempts to find the most likely (MAP) model, and uses its structure to answer these questions. However, when the amount of available data is modest, there might be many models that have non-negligible posterior. Thus, we want compute the Bayesian posterior of a feature, i.e., the total posterior probability of all models that contain it. In this paper, we propose a new approach for this task. We first show how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed order over network variables. This allows us to compute, for a given order, both the marginal probability of the data and the posterior of a feature. We then use this result as the basis for an algorithm that approximates the Bayesian posterior of a feature. Our approach uses a Markov Chain Monte Carlo (MCMC) method, but over orders rather than over network structures. The space of orders is smaller and more regular than the space of structures, and has much a smoother posterior landscape. We present empirical results on synthetic and real-life datasets that compare our approach to full model averaging (when possible), to MCMC over network structures, and to a non-Bayesian bootstrap approach.     

A novel discrete particle swarm optimization algorithm for solving bayesian network structures learning problem

ABSTRACT

Bayesian network is an effective representation tool to describe the uncertainty of the knowledge in artificial intelligence. One important method to learning Bayesian network from data is to employ a search procedure to explore the space of networks and a scoring metric to evaluate each candidate structure. In this paper, a novel discrete particle swarm optimization algorithm has been designed to solve the problem of Bayesian network structures learning. The proposed algorithm not only maintains the search advantages of the classical particle swarm optimization but also matches the characteristics of Bayesian networks. Meanwhile, mutation and neighbor searching operators have been used to overcome the drawback of premature convergence and balance the exploration and exploitation abilities of the particle swarm optimization. The experimental results on benchmark networks illustrate the feasibility and effectiveness of the proposed algorithm, and the comparative experiments indicate that our algorithm is highly competitive compared to other algorithms.     

Learning the structure of dynamic Bayesian networks from time series and steady state measurements

Dynamic Bayesian networks (DBN) are a class of graphical models that has become a standard tool for modeling various stochastic time-varying phenomena. In many applications, the primary goal is to infer the network structure from measurement data. Several efficient learning methods have been introduced for the inference of DBNs from time series measurements. Sometimes, however, it is either impossible or impractical to collect time series data, in which case, a common practice is to model the non-time series observations using static Bayesian networks (BN). Such an approach is obviously sub-optimal if the goal is to gain insight into the underlying dynamical model. Here, we introduce Bayesian methods for the inference of DBNs from steady state measurements. We also consider learning the structure of DBNs from a combination of time series and steady state measurements. We introduce two different methods: one that is based on an approximation and another one that provides exact computation. Simulation results demonstrate that dynamic network structures can be learned to an extent from steady state measurements alone and that inference from a combination of steady state and time series data has the potential to improve learning performance relative to the inference from time series data alone.     

Structure learning of Bayesian networks by genetic algorithms: aperformance analysis of control parameters

We present a new approach to structure learning in the field of Bayesian networks. We tackle the problem of the search for the best Bayesian network structure, given a database of cases, using the genetic algorithm philosophy for searching among alternative structures. We start by assuming an ordering between the nodes of the network structures. This assumption is necessary to guarantee that the networks that are created by the genetic algorithms are legal Bayesian network structures. Next, we release the ordering assumption by using a “repair operator” which converts illegal structures into legal ones. We present empirical results and analyze them statistically. The best results are obtained with an elitist genetic algorithm that contains a local optimizer