Structured neural network modelling of multi-valued functions for wind vector retrieval from satellite scatterometer measurements

A conventional neural network approach to regression problems approximates the conditional mean of the output vector. For mappings which are multi-valued this approach breaks down, since the average of two solutions is not necessarily a valid solution. In this article mixture density networks, a principled method for modelling conditional probability density functions, are applied to retrieving Cartesian wind vector components from satellite scatterometer data. A hybrid mixture density network is implemented to incorporate prior knowledge of the predominantly bimodal function branches. An advantage of a fully probabilistic model is that more sophisticated and principled methods can be used to resolve ambiguities.     

Neural network models for conditional distribution under bayesian analysis

We use neural networks (NN) as a tool for a nonlinear autoregression to predict the second moment of the conditional density of return series. The NN models are compared to the popular econometric GARCH(1,1) model. We estimate the models in a Bayesian framework using Markov chain Monte Carlo posterior simulations. The interlinked aspects of the proposed Bayesian methodology are identification of NN hidden units and treatment of NN complexity based on model evidence. The empirical study includes the application of the designed strategy to market data, where we found a strong support for a nonlinear multilayer perceptron model with two hidden units.     

Combining expert knowledge and local data for improved service life modeling of water supply networks

The presented approach aims to overcome the scarce data problem in service life modeling of water networks by combining subjective expert knowledge and local replacement data. A procedure to elicit imprecise quantile estimates of survival functions from experts, considering common cognitive biases, was developed and applied. The individual expert priors of the parameters of the service life distribution are obtained by regression over the stated distribution quantiles and aggregated into a single prior distribution. Furthermore, a likelihood function for the commonly encountered censored and truncated pipe replacement data is formulated. The suitability of the suggested Bayesian approach based on elicitation data from eight experts and real network data is demonstrated. Robust parameter estimates could be derived in data situations where frequentist maximum likelihood estimation is unsatisfactory, and to show how the consideration of imprecision and in-between-variance of experts improves posterior inference.     

An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach

Given the explosive growth of data collected from current business environment, data mining can potentially discover new knowledge to improve managerial decision making. This paper proposes a novel data mining approach that employs an evolutionary algorithm to discover knowledge represented in Bayesian networks. The approach is applied successfully to handle the business problem of finding response models from direct marketing data. Learning Bayesian networks from data is a difficult problem. There are two different approaches to the network learning problem. The first one uses dependency analysis, while the second one searches good network structures according to a metric. Unfortunately, both approaches have their own drawbacks. Thus, we propose a novel hybrid algorithm of the two approaches, which consists of two phases, namely, the conditional independence (CI) test and the search phases. In the CI test phase, dependency analysis is conducted to reduce the size of the search space. In the search phase, good Bayesian network models are generated by using an evolutionary algorithm. A new operator is introduced to further enhance the search effectiveness and efficiency. In a number of experiments and comparisons, the hybrid algorithm outperforms MDLEP, our previous algorithm which uses evolutionary programming (EP) for network learning, and other network learning algorithms. We then apply the approach to two data sets of direct marketing and compare the performance of the evolved Bayesian networks obtained by the new algorithm with those by MDLEP, the logistic regression models, the na/spl inodot//spl uml/ve Bayesian classifiers, and the tree-augmented na/spl inodot//spl uml/ve Bayesian network classifiers (TAN). In the comparison, the new algorithm outperforms the others.     

贝叶斯学习,贝叶斯网络与数据采掘

自从50～60年代贝叶斯学派形成后,关于贝叶斯分析的研究久盛不衰。早在80年代,贝叶斯网络就成功地应用于专家系统,成为表示不确定性专家知识和推理的一种流行方法。90年代以来,贝叶斯学习一直是机器学习研究的重要方向。由于概率统计与数据采掘的     

混合贝叶斯网络隐藏变量学习研究

目前,具有已知结构的隐藏变量学习主要针对具有离散变量的贝叶斯网和具有连续变量的高斯网．该文给出了具有连续和离散变量的混合贝叶斯网络隐藏变量学习方法．该方法不需要离散化连续变量,依据专业知识或贝叶斯网络道德图中Cliques的维数发现隐藏变量的位置,基于依赖结构（星形结构或先验结构）和Gibbs抽样确定隐藏变量的值,结合扩展的MDL标准和统计方法发现隐藏变量的最优维数．实验结果表明,这种方法能够有效地进行具有已知结构的混合贝叶斯网络隐藏变量学习．     

基于频繁项挖掘的贝叶斯网络结构学习算法BNSL-FIM

贝叶斯网络能够表示不确定知识并进行推理计算表达,但由于实际样本数据存在噪声和大小限制以及网络空间搜索的复杂性,贝叶斯网络结构学习始终会存在一定的误差。为了提高贝叶斯网络结构学习的准确度,提出了以最大频繁项集和关联规则分析结果为先验知识的贝叶斯网络结构学习算法BNSL-FIM 。首先从数据中挖掘出最大频繁项集并对该项集进行结构学习,之后使用关联规则分析结果对其进行校正,从而确定基于频繁项挖掘和关联规则分析的先验知识。然后提出一种融合先验知识的BDeu评分算法进行贝叶斯网络结构学习。最后在6个公开标准的数据集上开展了实验,并对比引入先验/不引入先验的结构与原始网络结构的汉明距离,结果表明所提算法与未引入先验的BDeu评分算法相比显著提高了贝叶斯网络结构学习的准确度。     

柴油机燃油系统多故障的解耦与诊断技术

柴油机燃油系统多故障的强关联耦合给其诊断过程带来严重的不确定性,同时导致建立诊断模型也往往依赖大量的先验知识,多故障的解耦与诊断已成为柴油机燃油系统故障诊断研究中的一大技术难题.针对该问题,提出一种基于简化模型结构和定量参数的贝叶斯网络诊断方法.在模型结构方面,利用粗糙集理论中的属性约简方法评估故障信息的等价关系,去除冗余故障特征,简化贝叶斯网络诊断模型的拓扑结构;在定量参数方面,采用因果机制独立模型分析故障事件的因果关联强度,将多故障对同种征兆的耦合影响解耦为单故障下的因果机制,模型所需的条件概率数量减化为故障数的线性形式.应用该诊断方法,燃油系统贝叶斯网络诊断模型所需的先验知识大幅减少,显著降低了该模型建立和应用的复杂程度.     

A novel Bayesian classification for uncertain data

Data uncertainty can be caused by numerous factors such as measurement precision limitations, network latency, data staleness and sampling errors. When mining knowledge from emerging applications such as sensor networks or location based services, data uncertainty should be handled cautiously to avoid erroneous results. In this paper, we apply probabilistic and statistical theory on uncertain data and develop a novel method to calculate conditional probabilities of Bayes theorem. Based on that, we propose a novel Bayesian classification algorithm for uncertain data. The experimental results show that the proposed method classifies uncertain data with potentially higher accuracies than the Naive Bayesian approach. It also has a more stable performance than the existing extended Naive Bayesian method.     

遥感波段与样本组合及贝叶斯网络结构变化分析

贝叶斯网络表达了输入数据与分类结果之间的依赖关系,网络结构则表达了节点之间的条件概率状态。遥感数据的贝叶斯网络结构训练涉及先验知识和样本数量两个方面,是贝叶斯网络结构分类的重要环节。该文以应用目标和遥感数据波段的物理意义为先验知识指导,进行了贝叶斯网络结构建立中的遥感数据波段数和样本数的优化组合实验,为贝叶斯网络在遥感数据分类方面提供了基础性实验结果。     

Modeling Incomplete Knowledge of Semantic Web Using Bayesian Networks

ABSTRACT

Interoperable ontologies already exist in the biomedical field, enabling scientists to communicate with minimum ambiguity. Unfortunately, ontology languages, in the semantic web, such as OWL and RDF(S), are based on crisp logic and thus they cannot handle uncertain knowledge about an application field, which is unsuitable for the medical domain. In this paper, we focus on modeling incomplete knowledge in the classical OWL ontologies, using Bayesian networks, all keeping the semantic of the first ontology, and applying algorithms dedicated to learn parameters of Bayesian networks in order to generate the Bayesian networks. We use EM algorithm for learning conditional probability tables of different nodes of Bayesian network automatically, contrary to different tools of Bayesian networks where probabilities are inserted manually. To validate our work, we have applied our model on the diagnosis of liver cancer using classical ontology containing incomplete instances, in order to handle medical uncertain knowledge, for predicting a liver cancer.     

Hybrid learning of Bayesian multinets for binary classification

We propose a scoring criterion, named mixture-based factorized conditional log-likelihood (mfCLL), which allows for efficient hybrid learning of mixtures of Bayesian networks in binary classification tasks. The learning procedure is decoupled in foreground and background learning, being the foreground the single concept of interest that we want to distinguish from a highly complex background. The overall procedure is hybrid as the foreground is discriminatively learned, whereas the background is generatively learned. The learning algorithm is shown to run in polynomial time for network structures such as trees and consistent κ-graphs. To gauge the performance of the mfCLL scoring criterion, we carry out a comparison with state-of-the-art classifiers. Results obtained with a large suite of benchmark datasets show that mfCLL-trained classifiers are a competitive alternative and should be taken into consideration.     

Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

We describe a Bayesian approach for learning Bayesian networks from a combination of prior knowledge and statistical data. First and foremost, we develop a methodology for assessing informative priors needed for learning. Our approach is derived from a set of assumptions made previously as well as the assumption of likelihood equivalence, which says that data should not help to discriminate network structures that represent the same assertions of conditional independence. We show that likelihood equivalence when combined with previously made assumptions implies that the user's priors for network parameters can be encoded in a single Bayesian network for the next case to be seen—a prior network—and a single measure of confidence for that network. Second, using these priors, we show how to compute the relative posterior probabilities of network structures given data. Third, we describe search methods for identifying network structures with high posterior probabilities. We describe polynomial algorithms for finding the highest-scoring network structures in the special case where every node has at most k = 1 parent. For the general case (k > 1), which is NP-hard, we review heuristic search algorithms including local search, iterative local search, and simulated annealing. Finally, we describe a methodology for evaluating Bayesian-network learning algorithms, and apply this approach to a comparison of various approaches.     

基于先验节点序学习贝叶斯网络结构的优化方法

针对小样本数据集下学习贝叶斯网络 (Bayesian networks, BN)结构的不足, 以及随着条件集的增大, 利用统计方法进行条件独立 (Conditional independence, CI) 测试不稳定等问题, 提出了一种基于先验节点序学习网络结构的优化方法. 新方法通过定义优化目标函数和可行域空间, 首次将贝叶斯网络结构学习问题转化为求解目标函数极值的数学规划问题, 并给出最优解的存在性及唯一性证明, 为贝叶斯网络的不断扩展研究提出了新的方案. 理论证明以及实验结果显示了新方法的正确性和有效性.     

Learning Bayesian network parameters under incomplete data with domain knowledge

Bayesian networks (BNs) have gained increasing attention in recent years. One key issue in Bayesian networks is parameter learning. When training data is incomplete or sparse or when multiple hidden nodes exist, learning parameters in Bayesian networks becomes extremely difficult. Under these circumstances, the learning algorithms are required to operate in a high-dimensional search space and they could easily get trapped among copious local maxima. This paper presents a learning algorithm to incorporate domain knowledge into the learning to regularize the otherwise ill-posed problem, to limit the search space, and to avoid local optima. Unlike the conventional approaches that typically exploit the quantitative domain knowledge such as prior probability distribution, our method systematically incorporates qualitative constraints on some of the parameters into the learning process. Specifically, the problem is formulated as a constrained optimization problem, where an objective function is defined as a combination of the likelihood function and penalty functions constructed from the qualitative domain knowledge. Then, a gradient-descent procedure is systematically integrated with the E-step and M-step of the EM algorithm, to estimate the parameters iteratively until it converges. The experiments with both synthetic data and real data for facial action recognition show our algorithm improves the accuracy of the learned BN parameters significantly over the conventional EM algorithm.     

On classification with incomplete data

We address the incomplete-data problem in which feature vectors to be classified are missing data (features). A (supervised) logistic regression algorithm for the classification of incomplete data is developed. Single or multiple imputation for the missing data is avoided by performing analytic integration with an estimated conditional density function (conditioned on the observed data). Conditional density functions are estimated using a Gaussian mixture model (GMM), with parameter estimation performed using both expectation-maximization (EM) and variational Bayesian EM (VB-EM). The proposed supervised algorithm is then extended to the semisupervised case by incorporating graph-based regularization. The semisupervised algorithm utilizes all available data-both incomplete and complete, as well as labeled and unlabeled. Experimental results of the proposed classification algorithms are shown     

Hybrid modeling of microwave devices using multi‐kernel support vector regression with prior knowledge

This article proposes a support‐vector hybrid modeling method of microwave devices when only a small number of measurements are available. In this method, a hybrid model of microwave device has been obtained by combining a coarse model and a support‐vector model, where the coarse model is complemented by a support‐vector model capable of correcting the difference between the measurements and the coarse model. The support‐vector model was developed using a novel algorithm. In the algorithm, multi‐kernel and prior knowledge from a calibrated simulator were incorporated into the framework of the linear programming support vector regression by utilizing multiple feature spaces and modifying the optimization formulation. The experimental results from two microwave devices show that the hybrid modeling can enhance the physical meaning of the support‐vector model and improve the modeling accuracy for a small dataset, and that the proposed algorithm shows great potential in some applications where sufficient experimental data is difficult and costly to obtain, but the prior knowledge from a simulation model is available. The hybrid modeling is suited to a microwave computer‐aided design tool or an automatic tuning robot. © 2014 Wiley Periodicals, Inc. Int J RF and Microwave CAE 25:219–228, 2015.     

Constructing Bayesian networks for medical diagnosis fromincomplete and partially correct statistics

The paper discusses several knowledge engineering techniques for the construction of Bayesian networks for medical diagnostics when the available numerical probabilistic information is incomplete or partially correct. This situation occurs often when epidemiological studies publish only indirect statistics and when significant unmodeled conditional dependence exists in the problem domain. While nothing can replace precise and complete probabilistic information, still a useful diagnostic system can be built with imperfect data by introducing domain-dependent constraints. We propose a solution to the problem of determining the combined influences of several diseases on a single test result from specificity and sensitivity data for individual diseases. We also demonstrate two techniques for dealing with unmodeled conditional dependencies in a diagnostic network. These techniques are discussed in the context of an effort to design a portable device for cardiac diagnosis and monitoring from multimodal signals     

Efficient inferencing for sigmoid Bayesian networks by reducing sampling space

A sigmoid Bayesian network is a Bayesian network in which a conditional probability is a sigmoid function of the weights of relevant arcs. Its application domain includes that of Boltzmann machine as well as traditional decision problems. In this paper we show that the node reduction method that is an inferencing algorithm for general Bayesian networks can also be used on sigmoid Bayesian networks, and we propose a hybrid inferencing method combining the node reduction and Gibbs sampling. The time efficiency of sampling after node reduction is demonstrated through experiments. The results of this paper bring sigmoid Bayesian networks closer to large scale applications.