Representing Probabilistic Rules with Networks of Gaussian Basis Functions

There is great interest in understanding the intrinsic knowledge neural networks have acquired during training. Most work in this direction is focussed on the multi-layer perceptron architecture. The topic of this paper is networks of Gaussian basis functions which are used extensively as learning systems in neural computation. We show that networks of Gaussian basis functions can be generated from simple probabilistic rules. Also, if appropriate learning rules are used, probabilistic rules can be extracted from trained networks. We present methods for the reduction of network complexity with the goal of obtaining concise and meaningful rules. We show how prior knowledge can be refined or supplemented using data by employing either a Bayesian approach, by a weighted combination of knowledge bases, or by generating artificial training data representing the prior knowledge. We validate our approach using a standard statistical data set.     

PREPROCESSING RULES FOR TRIANGULATION OF PROBABILISTIC NETWORKS*

Currently, the most efficient algorithm for inference with a probabilistic network builds upon a triangulation of a network's graph. In this paper, we show that pre‐processing can help in finding good triangulations for probabilistic networks, that is, triangulations with a maximum clique size as small as possible. We provide a set of rules for stepwise reducing a graph, without losing optimality. This reduction allows us to solve the triangulation problem on a smaller graph. From the smaller graph's triangulation, a triangulation of the original graph is obtained by reversing the reduction steps. Our experimental results show that the graphs of some well‐known real‐life probabilistic networks can be triangulated optimally just by preprocessing; for other networks, huge reductions in their graph's size are obtained.     

AND/OR search spaces for graphical models

The paper introduces an AND/OR search space perspective for graphical models that include probabilistic networks (directed or undirected) and constraint networks. In contrast to the traditional (OR) search space view, the AND/OR search tree displays some of the independencies present in the graphical model explicitly and may sometimes reduce the search space exponentially. Indeed, most algorithmic advances in search-based constraint processing and probabilistic inference can be viewed as searching an AND/OR search tree or graph. Familiar parameters such as the depth of a spanning tree, treewidth and pathwidth are shown to play a key role in characterizing the effect of AND/OR search graphs vs. the traditional OR search graphs. We compare memory intensive AND/OR graph search with inference methods, and place various existing algorithms within the AND/OR search space.     

Approximation Algorithms for Treewidth

This paper presents algorithms whose input is an undirected graph, and whose output is a tree decomposition of width that approximates the optimal, the treewidth of that graph. The algorithms differ in their computation time and their approximation guarantees. The first algorithm works in polynomial-time and finds a factor-O(log OPT) approximation, where OPT is the treewidth of the graph. This is the first polynomial-time algorithm that approximates the optimal by a factor that does not depend on n, the number of nodes in the input graph. As a result, we get an algorithm for finding pathwidth within a factor of O(log OPT⋅log n) from the optimal. We also present algorithms that approximate the treewidth of a graph by constant factors of 3.66, 4, and 4.5, respectively and take time that is exponential in the treewidth. These are more efficient than previously known algorithms by an exponential factor, and are of practical interest. Finding triangulations of minimum treewidth for graphs is central to many problems in computer science. Real-world problems in artificial intelligence, VLSI design and databases are efficiently solvable if we have an efficient approximation algorithm for them. Many of those applications rely on weighted graphs. We extend our results to weighted graphs and weighted treewidth, showing similar approximation results for this more general notion. We report on experimental results confirming the effectiveness of our algorithms for large graphs associated with real-world problems.     

Ising图模型概率推理的参数化复杂性

Ising图模型概率推理的主要工作是通过变量求和来计算配分函数和边缘概率分布。传统计算复杂性理论证明Ising图模型精确概率推理是NP难的,并且Ising图模型近似概率推理是NP难的。研究了Ising图模型精确概率推理和Ising均值场近似概率推理的参数化复杂性。首先证明了不同参数的Ising图模型概率推理的参数化复杂性定理,指出基于变量个数或图模型树宽的参数化概率推理问题是固定参数可处理的。然后证明了Ising均值场的参数化复杂性定理,指出基于自由分布树宽、迭代次数和变量个数的参数化Icing均值场是固定参数可处理的;进一步,当Ising图模型参数满足Ising均值场迭代式压缩条件时,基于自由分布树宽和迭代次数的参数化Ising均值场是固定参数可处理的。     

A taxonomy of weight learning methods for statistical relational learning

Machine Learning - Statistical relational learning (SRL) frameworks are effective at defining probabilistic models over complex relational data. They often use weighted first-order logical rules...     

Probabilistic knowledge bases

We define a new fixpoint semantics for rule based reasoning in the presence of weighted information. The semantics is illustrated on a real world application requiring such reasoning. Optimizations and approximations of the semantics are shown so as to make the semantics amenable to very large scale real world applications. We finally prove that the semantics is probabilistic and reduces to the usual fixpoint semantics of stratified Datalog if all information is certain. We implemented various knowledge discovery systems which automatically generate such probabilistic decision rules. In collaboration with a bank in Hong Kong we use one such system to forecast currency exchange rates     

加权3-Set Packing问题的核心化

Packing和Matching问题是一类重要的NP难解问题,该类问题的参数算法和核心化研究受到了人们广泛的关注.主要研究了加权3-SetPacking的核心化算法.对于加权3-SetPacking问题,基于对问题结构的深入分析,提出并证明了2个简化规则.首先限定加权3-SetPacking问题实例中包含给定2个元素的集合的个数,然后在限定问题实例中包含1个给定元素的集合的个数.基于对集合个数的限定,得到问题实例中总的集合个数的上界.并基于上述性质得到2个简化规则,可得到加权3-SetPacking问题大小为27k3-36k2+12k的核,该核心化结果是加权3-SetPacking问题的首个核心化结果.得到的加权3-SetPacking的核心化过程同样适用于加权3D-Matching问题的核化,可得到与加权3-SetPacking问题同样大小的问题核.     

Fast training of recurrent networks based on the EM algorithm

In this work, a probabilistic model is established for recurrent networks. The expectation-maximization (EM) algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean-field approximation. This new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons. These neurons are then trained via a linear weighted regression algorithm. The training time has been improved by five to 15 times on benchmark problems.     

Functional abilities of a stochastic logic neural network

The authors have studied the information processing ability of stochastic logic neural networks, which constitute one of the pulse-coded artificial neural network families. These networks realize pseudoanalog performance with local learning rules using digital circuits, and therefore suit silicon technology. The synaptic weights and the outputs of neurons in stochastic logic are represented by stochastic pulse sequences. The limited range of the synaptic weights reduces the coding noise and suppresses the degradation of memory storage capacity. To study the effect of the coding noise on an optimization problem, the authors simulate a probabilistic Hopfield model (Gaussian machine) which has a continuous neuron output function and probabilistic behavior. A proper choice of the coding noise amplitude and scheduling improves the network's solutions of the traveling salesman problem (TSP). These results suggest that stochastic logic may be useful for implementing probabilistic dynamics as well as deterministic dynamics.     

On the Expressive Power of Permanents and Perfect Matchings of Matrices of Bounded Pathwidth/Cliquewidth

Some 25 years ago Valiant introduced an algebraic model of computation in order to study the complexity of evaluating families of polynomials. The theory was introduced along with the complexity classes VP and VNP which are analogues of the classical classes P and NP. Families of polynomials that are difficult to evaluate (that is, VNP-complete) include the permanent and hamiltonian polynomials. In a previous paper the authors together with P. Koiran studied the expressive power of permanent and hamiltonian polynomials of matrices of bounded treewidth, as well as the expressive power of perfect matchings of planar graphs. It was established that the permanent and hamiltonian polynomials of matrices of bounded treewidth are equivalent to arithmetic formulas. Also, the sum of weights of perfect matchings of planar graphs was shown to be equivalent to (weakly) skew circuits. In this paper we continue the research in the direction described above, and study the expressive power of permanents, hamiltonians and perfect matchings of matrices that have bounded pathwidth or bounded cliquewidth. In particular, we prove that permanents, hamiltonians and perfect matchings of matrices that have bounded pathwidth express exactly arithmetic formulas. This is an improvement of our previous result for matrices of bounded treewidth. Also, for matrices of bounded weighted cliquewidth we show membership in VP for these polynomials.     

A stock selection strategy using fuzzy neural networks

This paper describes, from a general system-design perspective, an artificial neural network (ANN) approach to a stock selection strategy. The paper suggests a concept of neural gates which are similar to the processing elements in ANN, but generalized into handling various types of information such as fuzzy logic, probabilistic and Boolean information together. Forecasting of stock market returns, assessing of country risk and rating of stocks based on fuzzy rules, probabilistic and Boolean data can be done using the proposed neural gates. Fuzzy logic is known to be useful for decision-making where there is a great deal of uncertainty as well as vague phenomena, but lacks the learning capability; on the other hand, neural networks are useful in constructing an adaptive system which can learn from historical data, but are not able to process ambiguous rules and probabilistic data sets. This paper describes how these problems can be solved using the proposed neural gates.     

Algebraic reasoning for probabilistic action systems and while-loops

Back and von Wright have developed algebraic laws for reasoning about loops in a total correctness framework using the refinement calculus. We extend their work to reasoning about probabilistic loops in the probabilistic refinement calculus. We apply our algebraic reasoning to derive transformation rules for probabilistic action systems and probabilistic while-loops. In particular we focus on developing data refinement rules for these two constructs. Our extension is interesting since some well known transformation rules that are applicable to standard programs are not applicable to probabilistic ones: we identify some of these important differences and we develop alternative rules where possible.     

基于模糊规则自动生成算法的茶味觉信号识别

在充分研究了模糊加权神经网络和微粒群算法的基础上，给出一种能够自动生成模糊规则的剪枝算法,并以此建立了新的网络模型。通过茶味觉信号识别的仿真实验验证了该算法的有效性。     

基于变精度粗糙直觉模糊集的决策规则获取

以直觉模糊目标信息系统为研究对象,以粗糙集和直觉模糊集为工具,以知识发现为目的,给出了从直觉模糊决策表中获取决策规则的一种有效方法。即通过对Pawlak粗糙隶属函数的定义进行推广,给出粗糙直觉模糊隶属函数,利用新的粗糙隶属函数,建立了变精度粗糙直觉模糊集模型。在此模型基础上定义了变精度粗糙直觉模糊集的近似质量和近似约简,由近似约简导出概率决策规则集,从而给出了直觉模糊决策表的概率决策规则获取方法。最后,以实例说明了这一方法的有效性。关键词：     

基于概率覆盖决策粗糙集模型的中医菜谱分析

针对营养决策表规则提取中规则矛盾多、覆盖样例冗余多,导致有效规则遗漏的问题,提出概率覆盖决策粗糙集模型.首先,对决策粗糙集相关理论进行简要介绍,给出对应的属性约简和值约简理论和算法.然后,在决策粗糙集基础上,提出概率覆盖模型,根据值约简需求提出一、二、三度覆盖矩阵,以解决规则矛盾和冗余问题.最后,通过中医菜谱数据提取营养学规则实验,证明所提模型可有效解决规则矛盾问题,相比其他常用规则提取模型,概率覆盖模型所得规则约简力度较高,矛盾个数较少.     

Attribute reduction in decision-theoretic rough set models

Rough set theory can be applied to rule induction. There are two different types of classification rules, positive and boundary rules, leading to different decisions and consequences. They can be distinguished not only from the syntax measures such as confidence, coverage and generality, but also the semantic measures such as decision-monotocity, cost and risk. The classification rules can be evaluated locally for each individual rule, or globally for a set of rules. Both the two types of classification rules can be generated from, and interpreted by, a decision-theoretic model, which is a probabilistic extension of the Pawlak rough set model.As an important concept of rough set theory, an attribute reduct is a subset of attributes that are jointly sufficient and individually necessary for preserving a particular property of the given information table. This paper addresses attribute reduction in decision-theoretic rough set models regarding different classification properties, such as: decision-monotocity, confidence, coverage, generality and cost. It is important to note that many of these properties can be truthfully reflected by a single measure γ in the Pawlak rough set model. On the other hand, they need to be considered separately in probabilistic models. A straightforward extension of the γ measure is unable to evaluate these properties. This study provides a new insight into the problem of attribute reduction.     

Parallel Algorithms for Series Parallel Graphs and Graphs with Treewidth Two 1

In this paper a parallel algorithm is given that, given a graph G=(V,E) , decides whether G is a series parallel graph, and, if so, builds a decomposition tree for G of series and parallel composition rules. The algorithm uses O(log \kern -1pt |E|log ^\ast \kern -1pt |E|) time and O(|E|) operations on an EREW PRAM, and O(log \kern -1pt |E|) time and O(|E|) operations on a CRCW PRAM. The results hold for undirected as well as for directed graphs. Algorithms with the same resource bounds are described for the recognition of graphs of treewidth two, and for constructing tree decompositions of treewidth two. Hence efficient parallel algorithms can be found for a large number of graph problems on series parallel graphs and graphs with treewidth two. These include many well-known problems like all problems that can be stated in monadic second-order logic. Received July 15, 1997; revised January 29, 1999, and June 23, 1999.     

基于模糊加权神经网络的模糊规则自动获取

如何实现模糊规则的自动获取一直是模糊系统中的一个难题，文中在充分研究了模糊加权神经网络和遗传算法的基础上，给出了一种能够自动获取模糊规则的剪枝算法，并以此建立了新的网络模型。模拟结果验证了该模型的有效性。