ALLPAD: approximate learning of logic programs with annotated disjunctions

Logic Programs with Annotated Disjunctions (LPADs) provide a simple and elegant framework for representing probabilistic knowledge in logic programming. In this paper we consider the problem of learning ground LPADs starting from a set of interpretations annotated with their probability. We present the system ALLPAD for solving this problem. ALLPAD modifies the previous system LLPAD in order to tackle real world learning problems more effectively. This is achieved by looking for an approximate solution rather than a perfect one. A number of experiments have been performed on real and artificial data for evaluating ALLPAD, showing the feasibility of the approach. Editors: Stephen Muggleton, Ramon Otero, Simon Colton.     

Structured machine learning: the next ten years

The field of inductive logic programming (ILP) has made steady progress, since the first ILP workshop in 1991, based on a balance of developments in theory, implementations and applications. More recently there has been an increased emphasis on Probabilistic ILP and the related fields of Statistical Relational Learning (SRL) and Structured Prediction. The goal of the current paper is to consider these emerging trends and chart out the strategic directions and open problems for the broader area of structured machine learning for the next 10 years.     

Learning probabilistic logic models from probabilistic examples

We revisit an application developed originally using abductive Inductive Logic Programming (ILP) for modeling inhibition in metabolic networks. The example data was derived from studies of the effects of toxins on rats using Nuclear Magnetic Resonance (NMR) time-trace analysis of their biofluids together with background knowledge representing a subset of the Kyoto Encyclopedia of Genes and Genomes (KEGG). We now apply two Probabilistic ILP (PILP) approaches—abductive Stochastic Logic Programs (SLPs) and PRogramming In Statistical modeling (PRISM) to the application. Both approaches support abductive learning and probability predictions. Abductive SLPs are a PILP framework that provides possible worlds semantics to SLPs through abduction. Instead of learning logic models from non-probabilistic examples as done in ILP, the PILP approach applied in this paper is based on a general technique for introducing probability labels within a standard scientific experimental setting involving control and treated data. Our results demonstrate that the PILP approach provides a way of learning probabilistic logic models from probabilistic examples, and the PILP models learned from probabilistic examples lead to a significant decrease in error accompanied by improved insight from the learned results compared with the PILP models learned from non-probabilistic examples.     

PRL: A probabilistic relational language

In this paper, we describe the syntax and semantics for a probabilistic relational language (PRL). PRL is a recasting of recent work in Probabilistic Relational Models (PRMs) into a logic programming framework. We show how to represent varying degrees of complexity in the semantics including attribute uncertainty, structural uncertainty and identity uncertainty. Our approach is similar in spirit to the work in Bayesian Logic Programs (BLPs), and Logical Bayesian Networks (LBNs). However, surprisingly, there are still some important differences in the resulting formalism; for example, we introduce a general notion of aggregates based on the PRM approaches. One of our contributions is that we show how to support richer forms of structural uncertainty in a probabilistic logical language than have been previously described. Our goal in this work is to present a unifying framework that supports all of the types of relational uncertainty yet is based on logic programming formalisms. We also believe that it facilitates understanding the relationship between the frame-based approaches and alternate logic programming approaches, and allows greater transfer of ideas between them. Editors: Hendrik Blockeel, David Jensen and Stefan Kramer An erratum to this article is available at .     

基于归结的最大一般理论特化

提出一种基于归结的并有关于背景适应吸示例的一致特化理论,该理论给出了最大一般特化假设的归结构造方法,可将其作为一种蕴涵意义下的一般理论特化框架。基于该理论,进一步提出ｋ一般特化概念以解决特化的可计算性问题,并相应地给出特化算法。有关实验表明,该理论与算法能够正确并有效地进行一阶理论特化。     

A new approach to hybrid probabilistic logic programs

This paper presents a novel revision of the framework of Hybrid Probabilistic Logic Programming, along with a complete semantics characterization, to enable the encoding of and reasoning about real-world applications. The language of Hybrid Probabilistic Logic Programs framework is extended to allow the use of non-monotonic negation, and two alternative semantical characterizations are defined: stable probabilistic model semantics and probabilistic well-founded semantics. These semantics generalize the stable model semantics and well-founded semantics of traditional normal logic programs, and they reduce to the semantics of Hybrid Probabilistic Logic programs for programs without negation. It is the first time that two different semantics for Hybrid Probabilistic Programs with non-monotonic negation as well as their relationships are described. This proposal provides the foundational grounds for developing computational methods for implementing the proposed semantics. Furthermore, it makes it clearer how to characterize non-monotonic negation in probabilistic logic programming frameworks for commonsense reasoning. An erratum to this article can be found at     

A three-dimensional probabilistic fuzzy control system for network queue management

A novel probabilistic fuzzy control system is proposed to treat the congestion avoidance problem in transmission control protocol （TCP） networks. Studies on traffic measurement of TCP networks have shown that the packet traffic exhibits long range dependent properties called self-similarity, which degrades the network performance greatly. The probabilistic fuzzy control （PFC） system is used to handle the complex stochastic features of self-similar traffic and the modeling uncertainties in the network system. A three-dimensional （3-D） membership function （MF） is embedded in the PFC to express and describe the stochastic feature of network traffic. The 3-D MF has extended the traditional fuzzy planar mapping and further provides a spatial mapping among ＂fuzziness-randomness-state＂. The additional stochastic expression of 3-D MF provides the PFC an additional freedom to handle the stochastic features of self-similar traffic. Simulation experiments show that the proposed control method achieves superior performance compared to traditional control schemes in a stochastic environment.     

基于高阶逻辑的复杂结构归纳学习研究

归纳学习的目的在于发现样例与离散的类之间的映射关系,样例及归纳的映射都需用某个形式化语言描述.归纳学习器采用的形式化语言经历了属性-值语言、一阶逻辑、类型化的高阶逻辑三个阶段,后者能克服前二者在知识表达及学习过程中的很多缺点.本文首先阐述了基于高阶逻辑的复杂结构归纳学习产生的历史背景;其次介绍了基于高阶逻辑的编程语言--Escher的知识描述形式及目前已提出的三种学习方法;复杂结构的归纳学习在机器学习领域的应用及如何解决一些现实问题的讨论随后给出; 最后分析了复杂结构归纳学习的研究所面临的挑战性问题.     

Explanation-based interpretation of open-textured concepts in logical models of legislation

In this paper we discuss a view of the Machine Learning technique called Explanation-Based Learning (EBL) or Explanation-Based Generalization (EBG) as a process for the interpretation of vague concepts in logic-based models of law.The open-textured nature of legal terms is a well-known open problem in the building of knowledge-based legal systems. EBG is a technique which creates generalizations of given examples on the basis of background domain knowledge. We relate these two topics by considering EBG's domain knowledge as corresponding to statute law rules, and EBG's training example as corresponding to a precedent case.By making the interpretation of vague predicates as guided by precedent cases, we use EBG as an effective process capable of creating a link between predicates appearing as open-textured concepts in law rules, and predicates appearing as ordinary language wording for stating the facts of a case.Standard EBG algorithms do not change the deductive closure of the domain theory. In the legal context, this is only adequate when concepts vaguely defined in some law rules can be reformulated in terms of other concepts more precisely defined in other rules. We call theory reformulation the process adopted in this situation of complete knowledge.In many cases, however, statutory law leaves some concepts completely undefined. We then propose extensions to the EBG standard that deal with this situation of incomplete knowledge, and call theory revision the extended process. In order to fill in knowledge gaps we consider precedent cases supplemented by additional heuristic information. The extensions proposed treat heuristics represented by abstraction hierarchies with constraints and exceptions.In the paper we also precisely characterize the distinction between theory reformulation and theory revision by stating formal definitions and results, in the context of the Logic Programming theory.We offer this proposal as a possible contribution to cross fertilization between machine learning and legal reasoning methods.     

Generalizing predicates with string arguments

The least general generalization (LGG) of strings may cause an over-generalization in the generalization process of the clauses of predicates with string arguments. We propose a specific generalization (SG) for strings to reduce over-generalization. SGs of strings are used in the generalization of a set of strings representing the arguments of a set of positive examples of a predicate with string arguments. In order to create a SG of two strings, first, a unique match sequence between these strings is found. A unique match sequence of two strings consists of similarities and differences to represent similar parts and differing parts between those strings. The differences in the unique match sequence are replaced to create a SG of those strings. In the generalization process, a coverage algorithm based on SGs of strings or learning heuristics based on match sequences are used. Ilyas Cicekli received a Ph.D. in computer science from Syracuse University in 1991. He is currently a professor of the Department of Computer Engineering at Bilkent University. From 2001 till 2003, he was a visiting faculty at University of Central Florida. His current research interests include example-based machine translation, machine learning, natural language processing, and inductive logic programming. Nihan Kesim Cicekli is an Associate Professor of the Department of Computer Engineering at the Middle East Technical University (METU). She graduated in computer engineering at the Middle East Technical University in 1986. She received the MS degree in computer engineering at Bilkent University in 1988; and the PhD degree in computer science at Imperial College in 1993. She was a visiting faculty at University of Central Florida from 2001 till 2003. Her current research interests include multimedia databases, semantic web, web services, data mining, and machine learning.     

Relational Instance-Based Learning with Lists and Terms

The similarity measures used in first-order IBL so far have been limited to the function-free case. In this paper we show that a lot of power can be gained by allowing lists and other terms in the input representation and designing similarity measures that work directly on these structures. We present an improved similarity measure for the first-order instance-based learner ribl that employs the concept of edit distances to efficiently compute distances between lists and terms, discuss its computational and formal properties, and empirically demonstrate its additional power on a problem from the domain of biochemistry. The paper also includes a thorough reconstruction of ribl's overall algorithm.     

Computing most probable worlds of action probabilistic logic programs: scalable estimation for 1030,000 worlds

The semantics of probabilistic logic programs (PLPs) is usually given through a possible worlds semantics. We propose a variant of PLPs called action probabilistic logic programs or -programs that use a two-sorted alphabet to describe the conditions under which certain real-world entities take certain actions. In such applications, worlds correspond to sets of actions these entities might take. Thus, there is a need to find the most probable world (MPW) for -programs. In contrast, past work on PLPs has primarily focused on the problem of entailment. This paper quickly presents the syntax and semantics of -programs and then shows a naive algorithm to solve the MPW problem using the linear program formulation commonly used for PLPs. As such linear programs have an exponential number of variables, we present two important new algorithms, called and to solve the MPW problem exactly. Both these algorithms can significantly reduce the number of variables in the linear programs. Subsequently, we present a “binary” algorithm that applies a binary search style heuristic in conjunction with the Naive, and algorithms to quickly find worlds that may not be “most probable.” We experimentally evaluate these algorithms both for accuracy (how much worse is the solution found by these heuristics in comparison to the exact solution) and for scalability (how long does it take to compute). We show that the results of are very accurate and also very fast: more than 10^30,000 worlds can be handled in a few minutes. Subsequently, we develop parallel versions of these algorithms and show that they provide further speedups.      

束搜索算法的候选选取方法研究

在假设空间中进行爬山搜索是机器学习算法中常用的策略，爬山算法不能保证得到全局最优解，为了减少收敛到局部最优解的风险，束搜索应用而生。宽度为k的束搜索，在每一步以k个最佳候选为入口进行搜索(产生分支)，并从结果集中再次选取k个候选作为下一步的搜索入口。但目前多数算法只是在结果集中简单选取具有最大启发式性能量度值的k个成员。该文讨论了束搜索算法，提出了几种合理的候选选取方法，并在UCI数据库上进行对比实验测试，给出了实验结果。     

Complexity parameters for first order classes

We study several complexity parameters for first order formulas and their suitability for first order learning models. We show that the standard notion of size is not captured by sets of parameters that are used in the literature and thus they cannot give a complete characterization in terms of learnability with polynomial resources. We then identify an alternative notion of size and a simple set of parameters that are useful for first order Horn Expressions. These parameters are the number of clauses in the expression, the maximum number of distinct terms in a clause, and the maximum number of literals in a clause. Matching lower bounds derived using the Vapnik Chervonenkis dimension complete the picture showing that these parameters are indeed crucial. This work has been partly supported by NSF Grant IIS-0099446. A preliminary version of this paper appeared in the proceeding of the conference on Inductive Logic Programming 2003. Most of this work was done while M.A. was at Tufts University. Editors: Tamás Horváth and Akihiro Yamamoto     

归纳逻辑程序设计中的优化问题研究

归纳逻辑程序设计的核心问题是如何从背景知识中优选谓词构造满足约束的归纳假设,按Ｏｃｃａｍ准则,满足约束的最精简归纳假设为优,但迄今归纳逻辑程序设计中精简归纳假设构造的计算复杂性尚未解决。     

粗集和多关系学习综述

文中给出了多关系学习的产生、实质以及任务,指出多关系学习具有狭义和广义两个层面。对粗集和多关系学习给出了简单综述,表明粗集理论在多关系学习中占有重要地位。分析研究了粗集和归纳逻辑程序设计方法用于多关系学习的几种结合途径,尤其重点介绍了RSILP系列模型,并在文中给出其一般模型。文中对用于多关系学习中其它粗集方法也作了简单介绍。RSILP模型的完善扩展以及粗集方法在多关系学习中的进一步应用是今后的工作方向。     

On the non-efficient PAC learnability of conjunctive queries

This note serves three purposes: (i) we provide a self-contained exposition of the fact that conjunctive queries are not efficiently learnable in the Probably-Approximately-Correct (PAC) model, paying clear attention to the complicating fact that this concept class lacks the polynomial-size fitting property, a property that is tacitly assumed in much of the computational learning theory literature; (ii) we establish a strong negative PAC learnability result that applies to many restricted classes of conjunctive queries (CQs), including acyclic CQs for a wide range of notions of acyclicity; (iii) we show that CQs (and UCQs) are efficiently PAC learnable with membership queries.     

统计关系学习研究进展

统计关系学习是人工智能领域的一个新研究热点,它将关系表示、似然性理论和机器学习相结合,能更好地解决现实世界中复杂的关系数据问题,在生物信息学、web导航、社会网、地理信息系统和自然语言理解等领域有着重要的应用.首先对统计关系学习的研究内容以及研究任务进行了介绍和总结,然后根据概率表示和推理机制的不同,对当前的统计关系学习方法进行了分类,并对各类方法进行了详细介绍,最后讨论了当前统计关系学习存在的问题,并指出了今后研究和发展的方向.     

一种基于ILP和ASP的学习B语言描述的动作模型方法

动作模型学习可以使Agent主动适应动态环境中的变化,从而提高Agent的自治性,同时也可为动态域建模提供一个初步模型,为后期的模型完善和修改提供了基础.通过结合归纳逻辑程序设计(Inductive Logic Program-ming,ILP)和回答集程序设计(Answer Set Programming,ASP),设计了一个学习B语言描述的动作模型算法,该算法可以在混合规模的动态域中进行学习,并采用经典规划实例验证了该学习算法的有效性.     

概率逻辑模型与学习研究进展

近年来,概率逻辑学习研究取得了很大进展,已经提出各种不同的形式化方法和学习方法,包括概率关系模（PRMs）、贝叶斯逻辑程序（BLPs）、逻辑贝叶斯网络（LBNs）和随机逻辑程序（SLPs）等。文章重点介绍了贝叶斯网络与一阶逻辑的结合,并以PRMs、BLPs和LBNs为例,描述了基于贝叶斯网络的概率逻辑模型（PLMs）的知识表示方法,给出了此类PLMs一般使用的参数估计方法和结构学习方法,并给出了建议的研究方向。