判别贝叶斯网络的CEM学习算法

当数据存在缺值时,通常应用EM算法学习贝叶斯网络.然而,EM算法以联合似然作为目标函数,与判别预测问题的目标相偏离.与EM算法不同,CEM(Conditional Expectation Maximum)算法直接以条件似然作为目标函数.研究了判别贝叶斯网络学习的CEM算法,提出一种使得CEM算法具有单调性和收敛性的Q函数.为了简化计算,在CEM算法的E步,应用Q函数的一种简化形式;在CEM算法的M步,应用梯度下降法的一次搜索结果作为最优值的近似.最后,在UCI数据集上的实验结果表明了CEM算法在判别贝叶斯网络学习中的有效性.     

Complex Probabilistic Modeling with Recursive Relational Bayesian Networks

A number of representation systems have been proposed that extend the purely propositional Bayesian network paradigm with representation tools for some types of first-order probabilistic dependencies. Examples of such systems are dynamic Bayesian networks and systems for knowledge based model construction. We can identify the representation of probabilistic relational models as a common well-defined semantic core of such systems.Recursive relational Bayesian networks (RRBNs) are a framework for the representation of probabilistic relational models. A main design goal for RRBNs is to achieve greatest possible expressiveness with as few elementary syntactic constructs as possible. The advantage of such an approach is that a system based on a small number of elementary constructs will be much more amenable to a thorough mathematical investigation of its semantic and algorithmic properties than a system based on a larger number of high-level constructs. In this paper we show that with RRBNs we have achieved our goal, by showing, first, how to solve within that framework a number of non-trivial representation problems. In the second part of the paper we show how to construct from a RRBN and a specific query, a standard Bayesian network in which the answer to the query can be computed with standard inference algorithms. Here the simplicity of the underlying representation framework greatly facilitates the development of simple algorithms and correctness proofs. As a result we obtain a construction algorithm that even for RRBNs that represent models for complex first-order and statistical dependencies generates standard Bayesian networks of size polynomial in the size of the domain given in a specific application instance.     

Representing Probabilistic Rules with Networks of Gaussian Basis Functions

There is great interest in understanding the intrinsic knowledge neural networks have acquired during training. Most work in this direction is focussed on the multi-layer perceptron architecture. The topic of this paper is networks of Gaussian basis functions which are used extensively as learning systems in neural computation. We show that networks of Gaussian basis functions can be generated from simple probabilistic rules. Also, if appropriate learning rules are used, probabilistic rules can be extracted from trained networks. We present methods for the reduction of network complexity with the goal of obtaining concise and meaningful rules. We show how prior knowledge can be refined or supplemented using data by employing either a Bayesian approach, by a weighted combination of knowledge bases, or by generating artificial training data representing the prior knowledge. We validate our approach using a standard statistical data set.     

混合贝叶斯网络隐藏变量学习研究

目前,具有已知结构的隐藏变量学习主要针对具有离散变量的贝叶斯网和具有连续变量的高斯网．该文给出了具有连续和离散变量的混合贝叶斯网络隐藏变量学习方法．该方法不需要离散化连续变量,依据专业知识或贝叶斯网络道德图中Cliques的维数发现隐藏变量的位置,基于依赖结构（星形结构或先验结构）和Gibbs抽样确定隐藏变量的值,结合扩展的MDL标准和统计方法发现隐藏变量的最优维数．实验结果表明,这种方法能够有效地进行具有已知结构的混合贝叶斯网络隐藏变量学习．     

具有隐藏变量的贝叶斯网络结构学习

目前,学习具有隐藏变量的贝叶斯网络结构主要采用结合EM算法的打分-搜索方法,其效率和可靠性低.本文针对此问题建立一种新的具有隐藏变量贝叶斯网络结构学习方法.该方法首先依据变量之间基本依赖关系、基本结构和依赖分析思想进行不考虑隐藏变量的贝叶斯网络结构学习,然后利用贝叶斯网络道德图中的Cliques发现隐藏变量的位置,最后基于依赖结构、Gibbs sampling和MDL标准确定隐藏变量的取值、维数和局部结构.该方法能够避免标准Gibbs sampling的指数复杂性问题和现有学习方法存在的主要问题.实验结果表明,该方法能够有效进行具有隐藏变量的贝叶斯网络结构学习.     

A Bayesian Method for the Induction of Probabilistic Networks from Data

This paper presents a Bayesian method for constructing probabilistic networks from databases. In particular, we focus on constructing Bayesian belief networks. Potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems. We extend the basic method to handle missing data and hidden (latent) variables. We show how to perform probabilistic inference by averaging over the inferences of multiple belief networks. Results are presented of a preliminary evaluation of an algorithm for constructing a belief network from a database of cases. Finally, we relate the methods in this paper to previous work, and we discuss open problems.     

贝叶斯网络概率推理中的空间优化方法

递归调节算法作为一种贝叶斯网络精确推理算法由于存储结果的个数是任意的,存储所占用的内存大小是可变的,所以该算法在空间上是自由的。然而当内存空间有限时,又希望节省计算时间,存储哪些计算结果就成了关键问题。针对此问题本文提出了应用深度优先分支定界法寻找存储占用空间的所有可能性,然后通过任意空间下推理时间的求解公式得到相应的推理时间,进而构造贝叶斯网络推理的时间—空间曲线,通过所构造的曲线找出最优离散存储策略,得到时间和空间之间的最佳的结合点,以最小时间代价换取了最大的存储空间。     

用含动态隐层的前馈网辨识非线性系统

用含动态隐层的前馈网对非线性系统建立全局成立的输入输出模型,证明了这种辨识结构的可行性,网络学习算法为动态BP法.     

Incorporating Prior Knowledge in the Form of Production Rules into Neural Networks Using Boolean-Like Neurons

At present, nearly all neural networks are formulated by learning only from examples or patterns. For a real-word problem, some forms of prior knowledge in a non-example form always exist. Incorporation of prior knowledge will benefit the formulation of neural networks. Prior knowledge could be in several forms. Production rule is one form in which the prior knowledge is frequently represented. This paper proposes an approach to incorporate production rules into neural networks. A newly defined neuron architecture, Boolean-like neuron, is proposed. With this Boolean-like neuron, production rules can be encoded into the neural network during the network initialization period. Experiments are described in this paper. The results show that the incorporation of this prior knowledge can not only increase the training speed, but also the explainability of the neural networks.     

基于定性动态概率网络的网络拥塞控制

网络拥塞已经成为了人们普遍关注的问题。目前,针对网络拥塞控制的研究,提出了一些具体的方法和建议,但其都不能很好地解决网络拥塞。定性动态概率网络(QDPNs)是目前进行动态地推理不确定知识领域最有效的模型之一。提出了一种基于定性动态概率网络的网络拥塞控制,该方法能够有效地对网络拥塞进行预测,在预测的基础上给出了相应的拥塞控制策略。     

Intelligent Human-Machine Interaction Based on Dynamic Bayesian Networks Probabilistic Intention Recognition

In this article, a novel human–machine interaction based on the machine intention recognition of the human is presented. This work is motivated by the desire that intelligent machines as robots imitate human–human interaction, that is to minimize the need for classical direct human–machine interface and communication. A philosophical and technical background for intention recognition is discussed. Here, the intention–action–state scenario is modified and modeled by Dynamic Bayesian Networks to facilitate for probabilistic intention inference. The recognized intention, then, drives the interactive behavior of the machine such that it complies with the human intention in light of the real state of the world. An illustrative example of a human commanding a mobile robot remotely is given and discussed in details.     

Application of Cascade Correlation Networks for Structures to Chemistry

We present the application of Cascade Correlation for structures to QSPR (quantitative structure-property relationships) and QSAR (quantitative structure-activity relationships) analysis. Cascade Correlation for structures is a neural network model recently proposed for the processing of structured data. This allows the direct treatment of chemical compounds as labeled trees, which constitutes a novel approach to QSPR/QSAR. We report the results obtained for QSPR on Alkanes (predicting the boiling point) and QSAR of a class of Benzodiazepines. Our approach compares favorably versus the traditional QSAR treatment based on equations and it is competitive with ad hoc MLPs for the QSPR problem.     

基于遗传算法的Bayesian网中连续变量离散化的研究

文中如何从含有离散变量和连续变量的混合数据中学习Bayesian网进行了研究，提出了一种基于遗传算法的连续变量散化算法，在该处中给出了兼顾离散模型准确度和复杂度的适应度函数；并基于对离散化的实质性分析，定义了离散策略等价的概念，由此制定了离散策略的编码方案；进一步设计了变换离散策略的遗传算法。算法不存在局部极值问题，且不需要事先给定变量序关系，模拟实验结果表明，该算法能有效地对连续变量散化，从而使得从混合数据中学到的Bayesian网具有较好性能。     

无线传感器网络的系统化自适应建模

无线传感器网络的系统能耗制约着全网络的综合应用能力,其中节点有限的能量从根本上影响着传感器网络效能。针对无线传感器网络的全局能耗问题,提出了基于径向基函数神经网络以及状态空间表达的系统化建模方法。考虑到无线传感器网络的拓扑结构与分级关系, 采用径向基函数神经网络自适应实时规划系统。鉴于各传感器节点对数据的不同处理方式与能耗密切相关, 对全系统能耗建立系统化矩阵模型。仿真分析表明该模型可根据实际应用背景调整设置完成全局优化。     

态势估计贝叶斯网络的面向对象知识表示方法

知识表示是大规模贝叶斯网络的一个难题。论文以Koller提出的面向对象贝叶斯网络为基础,讨论态势估计贝叶斯网络的面向对象知识表示方法,并以关系数据库实现网络的存储与访问。     

基于混合因子分析的隐马尔可夫模型

经典隐马尔可夫模型用于语音识别存在的两个主要缺陷是“离散状态假设”和“独立分布假设”。前者忽略了语音信号的非平稳性,后者忽略了语音信号的相关性。文章将混合因子分析方法用于语音建模,提出了基于混合因子分析的隐马尔可夫模型框架,并用动态贝叶斯网络形象地表示。该模型框架不仅从理论上解决了上述问题,而且给出许多语音建模的选择。目前广泛使用的统计声学模型均可视为该模型的特例。     

基于ROI-KNN卷积神经网络的面部表情识别

深度神经网络已经被证明在图像、语音、文本领域具有挖掘数据深层潜在的分布式表达特征的能力. 通过在多个面部情感数据集上训练深度卷积神经网络和深度稀疏校正神经网络两种深度学习模型, 对深度神经网络在面部情感分类领域的应用作了对比评估. 进而, 引入了面部结构先验知识, 结合感兴趣区域(Region of interest, ROI)和K最近邻算法(K-nearest neighbors, KNN), 提出一种快速、简易的针对面部表情分类的深度学习训练改进方案——ROI-KNN, 该训练方案降低了由于面部表情训练数据过少而导致深度神经网络模型泛化能力不佳的问题, 提高了深度学习在面部表情分类中的鲁棒性, 同时, 显著地降低了测试错误率.     

带有微分项改进的自适应梯度下降优化算法

梯度下降算法作为卷积神经网络训练常用优化算法,其性能的优劣直接影响网络训练收敛性.本文主要分析了目前梯度优化算法中存在超调而影响收敛性问题以及学习率自适应性问题,提出了一种带微分项的自适应梯度优化算法,旨在改善网络优化过程收敛性的同时提高收敛速率.首先,针对优化过程存在较大超调量的问题,通过对迭代算法的重整合以及结合传统控制学原理引入微分项等方式来克服权重更新滞后于实际梯度改变的问题;然后,引入自适应机制来应对因学习率的不适应性导致的收敛率差和收敛速率慢等问题;紧接着,基于柯西-施瓦茨和杨氏不等式等证明了新算法的最差性能上界(悔界)为■.最后,通过在包括MNIST数据集以及CIFAR-10基准数据集上的仿真实验来验证新算法的有效性,结果表明新算法引入的微分项和自适应机制的联合模式能够有效地改善梯度下降算算法的收敛性能,从而实现算法性能的明显改善.     

基于LabVIEW的神经网络自适应逆控制系统

目前基于人工神经网络的非线性自适应逆控制研究主要集中在Matlab仿真研究方面,无法直接推广为实际应用。为此,采用基于LabVIEW的动态神经网络非线性自适应逆控制方法,首先在LabVIEW中建立动态神经网络结构及在线学习算法,并依此建立非线性对象的辨识器和逆控制器等模型;然后构建完整的非线性对象自适应逆控制系统,并在LabVIEW环境中通过仿真验证了系统性能。通过配置相应的数据采集设备,该系统可以直接推广为实际应用。