归纳逻辑程序设计方法综述

归纳逻辑程序设计是机器学习与逻辑程序设计交叉所形成的一个研究领域,克服了传统机器学习方法的两个主要限制：即知识表示的限制和背景知识利用的限制,成为机器学习的前沿研究课题。首先从归纳逻辑程序设计的产生背景、定义、应用领域及问题背景介绍了归纳逻辑程序设计系统的概貌,对归纳逻辑程序设计方法的研究现状进行了总结和分析,最后探讨了该领域的进一步的研究方向。     

Inductive logic programming for relational knowledge discovery

Inductive logic programming (ILP) is concerned with the induction of logic programs from examples and background knowledge. In ILP, the shift of attention from program synthesis to knowledge discovery resulted in advanced techniques that are practically applicable for discovering knowledge in relational databases. This paper gives a brief introduction to ILP, presents selected ILP techniques for relational knowledge discovery and reviews selected ILP applications. Nada Lavrač, Ph.D.: She is a senior research associate at the Department of Intelligent Systems, J. Stefan Institute, Ljubljana, Slovenia (since 1978) and a visiting professor at the Klagenfurt University, Austria (since 1987). Her main research interest is in machine learning, in particular inductive logic programming and intelligent data analysis in medicine. She received a BSc in Technical Mathematics and MSc in Computer Science from Ljubljana University, and a PhD in Technical Sciences from Maribor University, Slovenia. She is coauthor of KARDIO: A Study in Deep and Qualitative Knowledge for Expert Systems, The MIT Press 1989, and Inductive Logic Programming: Techniques and Applications, Ellis Horwood 1994, and coeditor of Intelligent Data Analysis in Medicine and Pharmacology, Kluwer 1997. She was the coordinator of the European Scientific Network in Inductive Logic Programming ILPNET (1993–1996) and program cochair of the 8th European Machine Learning Conference ECML’95, and 7th International Workshop on Inductive Logic Programming ILP’97. Sašo Džeroski, Ph.D.: He is a research associate at the Department of Intelligent Systems, J. Stefan Institute, Ljubljana, Slovenia (since 1989). He has held visiting researcher positions at the Turing Institute, Glasgow (UK), Katholieke Universiteit Leuven (Belgium), German National Research Center for Computer Science (GMD), Sankt Augustin (Germany) and the Foundation for Research and Technology-Hellas (FORTH), Heraklion (Greece). His research interest is in machine learning and knowledge discovery in databases, in particular inductive logic programming and its applications and knowledge discovery in environmental databases. He is co-author of Inductive Logic Programming: Techniques and Applications, Ellis Horwood 1994. He is the scientific coordinator of ILPnet2, The Network of Excellence in Inductive Logic Programming. He was program co-chair of the 7th International Workshop on Inductive Logic Programming ILP’97 and will be program co-chair of the 16th International Conference on Machine Learning ICML’99. Masayuki Numao, Ph.D.: He is an associate professor at the Department of Computer Science, Tokyo Institute of Technology. He received a bachelor of engineering in electrical and electronics engineering in 1982 and his Ph.D. in computer science in 1987 from Tokyo Institute of Technology. He was a visiting scholar at CSLI, Stanford University from 1989 to 1990. His research interests include Artificial Intelligence, Global Intelligence and Machine Learning. Numao is a member of Information Processing Society of Japan, Japanese Society for Artificial Intelligence, Japanese Cognitive Science Society, Japan Society for Software Science and Technology and AAAI.     

基于归结的最大一般理论特化

提出一种基于归结的并有关于背景适应吸示例的一致特化理论,该理论给出了最大一般特化假设的归结构造方法,可将其作为一种蕴涵意义下的一般理论特化框架。基于该理论,进一步提出ｋ一般特化概念以解决特化的可计算性问题,并相应地给出特化算法。有关实验表明,该理论与算法能够正确并有效地进行一阶理论特化。     

The Connectionist Inductive Learning and Logic Programming System

This paper presents the Connectionist Inductive Learning and Logic Programming System (C-IL²P). C-IL²P is a new massively parallel computational model based on a feedforward Artificial Neural Network that integrates inductive learning from examples and background knowledge, with deductive learning from Logic Programming. Starting with the background knowledge represented by a propositional logic program, a translation algorithm is applied generating a neural network that can be trained with examples. The results obtained with this refined network can be explained by extracting a revised logic program from it. Moreover, the neural network computes the stable model of the logic program inserted in it as background knowledge, or learned with the examples, thus functioning as a parallel system for Logic Programming. We have successfully applied C-IL²P to two real-world problems of computational biology, specifically DNA sequence analyses. Comparisons with the results obtained by some of the main neural, symbolic, and hybrid inductive learning systems, using the same domain knowledge, show the effectiveness of C-IL²P.     

Rough Problem Settings for ILP Dealing With Imperfect Data

This paper applies rough set theory to Inductive Logic Programming (ILP, a relatively new method in machine learning) to deal with imperfect data occurring in large real-world applications. We investigate various kinds of imperfect data in ILP and propose rough problem settings to deal with incomplete background knowledge (where essential predicates/clauses are missing), indiscernible data (where some examples belong to both sets of positive and negative training examples), missing classification (where some examples are unclassified) and too strong declarative bias (hence the failure in searching for solutions). The rough problem settings relax the strict requirements in the standard normal problem setting for ILP, so that rough but useful hypotheses can be induced from imperfect data. We give simple measures of learning quality for the rough problem settings. For other kinds of imperfect data (noise data, too sparse data, missing values, real-valued data, etc.), while referring to their traditional handling techniques, we also point out the possibility of new methods based on rough set theory.     

一种自顶向下和自底向上相结合的ILP算法

归纳逻辑程序设计(ILP)是机器学习的一个重要分支,给定一个样例集和相关背景知识,ILP研究如何构建与其相一致的逻辑程序,这些逻辑程序由有限一阶子句组成。文章描述了一种综合当前一些ILP方法多方面优势的算法ICCR,ICCR溶合了以FOIL为代表的自顶向下搜索策略和以GOLEM为代表的自底向上搜索策略,并能根据需要发明新谓词、学习递归逻辑程序,对比实验表明,对相同的样例及背景知识,ICCR比FOIL和GOLEM能学到精度更高的目标逻辑程序。     

Engineering applications of ILP

Several applications of Inductive Logic Programming (ILP) are presented. These belong to various areas of engineering, including mechanical, environmental, software, and dynamical systems engineering. The particular applications are finite element mesh design, biological classification of river water quality, data reification, inducing program invariants, learning qualitative models of dynamic systems, and learning control rules for dynamic systems. A number of other applications are briefly mentioned. Finally, a discussion of the advantages and disadvantages of ILP as compared to other approaches to machine learning is given.     

Learning probabilistic logic models from probabilistic examples

We revisit an application developed originally using abductive Inductive Logic Programming (ILP) for modeling inhibition in metabolic networks. The example data was derived from studies of the effects of toxins on rats using Nuclear Magnetic Resonance (NMR) time-trace analysis of their biofluids together with background knowledge representing a subset of the Kyoto Encyclopedia of Genes and Genomes (KEGG). We now apply two Probabilistic ILP (PILP) approaches—abductive Stochastic Logic Programs (SLPs) and PRogramming In Statistical modeling (PRISM) to the application. Both approaches support abductive learning and probability predictions. Abductive SLPs are a PILP framework that provides possible worlds semantics to SLPs through abduction. Instead of learning logic models from non-probabilistic examples as done in ILP, the PILP approach applied in this paper is based on a general technique for introducing probability labels within a standard scientific experimental setting involving control and treated data. Our results demonstrate that the PILP approach provides a way of learning probabilistic logic models from probabilistic examples, and the PILP models learned from probabilistic examples lead to a significant decrease in error accompanied by improved insight from the learned results compared with the PILP models learned from non-probabilistic examples.     

归纳逻辑程序设计综述

归纳逻辑程序设计是由机器学习与逻辑程序设计交叉所形成的一个研究领域,是机器学习的前沿研究课题。该文首先从归纳逻辑程序设计的问题背景、类型划分和搜索程序子句三个方面介绍了归纳逻辑程序设计系统的概貌;然后结合实验室的相关研究工作,回顾了归纳逻辑程序设计研究的发展;之后介绍了归纳逻辑程序设计领域中需要深入研究的若干问题,并提出了新的解决思路;最后是总结,以引起读者对归纳逻辑程序设计领域研究的进一步关注。     

Parallel ILP for distributed-memory architectures

The growth of machine-generated relational databases, both in the sciences and in industry, is rapidly outpacing our ability to extract useful information from them by manual means. This has brought into focus machine learning techniques like Inductive Logic Programming (ILP) that are able to extract human-comprehensible models for complex relational data. The price to pay is that ILP techniques are not efficient: they can be seen as performing a form of discrete optimisation, which is known to be computationally hard; and the complexity is usually some super-linear function of the number of examples. While little can be done to alter the theoretical bounds on the worst-case complexity of ILP systems, some practical gains may follow from the use of multiple processors. In this paper we survey the state-of-the-art on parallel ILP. We implement several parallel algorithms and study their performance using some standard benchmarks. The principal findings of interest are these: (1) of the techniques investigated, one that simply constructs models in parallel on each processor using a subset of data and then combines the models into a single one, yields the best results; and (2) sequential (approximate) ILP algorithms based on randomized searches have lower execution times than (exact) parallel algorithms, without sacrificing the quality of the solutions found. This is an extended version of the paper entitled Strategies to Parallelize ILP Systems, published in the Proceedings of the 15th International Conference on Inductive Logic Programming (ILP 2005), vol. 3625 of LNAI, pp. 136–153, Springer-Verlag.     

The Variable Precision Rough Set Inductive Logic Programming Model and Strings

The Variable Precision Rough Set Inductive Logic Programming model (VPRSILP model) extends the Variable Precision Rough Set (VPRS) model to Inductive Logic Programming (ILP). The generic Rough Set Inductive Logic Programming (gRS-ILP) model provides a framework for ILP when the setting is imprecise and any induced logic program will not be able to distinguish between certain positive and negative examples. The gRS-ILP model is extended in this paper to the VPRSILP model by including features of the VPRS model. The VPRSILP model is applied to strings and an illustrative experiment on transmembrane domains in amino acid sequences is presented.     

Inductive logic programming

A new research area, Inductive Logic Programming, is presently emerging. While inheriting various positive characteristics of the parent subjects of Logic Programming and Machine Learning, it is hoped that the new area will overcome many of the limitations of its forebears. The background to present developments within this area is discussed and various goals and aspirations for the increasing body of researchers are identified. Inductive Logic Programming needs to be based on sound principles from both Logic and Statistics. On the side of statistical justification of hypotheses we discuss the possible relationship between Algorithmic Complexity theory and Probably-Approximately-Correct (PAC) Learning. In terms of logic we provide a unifying framework for Muggleton and Buntine’s Inverse Resolution (IR) and Plotkin’s Relative Least General Generalisation (RLGG) by rederiving RLGG in terms of IR. This leads to a discussion of the feasibility of extending the RLGG framework to allow for the invention of new predicates, previously discussed only within the context of IR.     

Machine learning in artificial intelligence

Among several forms of learning, learning concepts from examples is the most common and best understood. In this paper some approaches to learning concepts from examples are reviewed. In particular those approaches that are currently most important with respect to practical applications (learning decision trees and if-then rules), or likely to become very important in the near future (Inductive Logic Programming as a form of relational learning) are discussed.     

Inference of abduction theories for handling incompleteness in first-order learning

In real-life domains, learning systems often have to deal with various kinds of imperfections in data such as noise, incompleteness and inexactness. This problem seriously affects the knowledge discovery process, specifically in the case of traditional Machine Learning approaches that exploit simple or constrained knowledge representations and are based on single inference mechanisms. Indeed, this limits their capability of discovering fundamental knowledge in those situations. In order to broaden the investigation and the applicability of machine learning schemes in such particular situations, it is necessary to move on to more expressive representations which require more complex inference mechanisms. However, the applicability of such new and complex inference mechanisms, such as abductive reasoning, strongly relies on a deep background knowledge about the specific application domain. This work aims at automatically discovering the meta-knowledge needed to abduction inference strategy to complete the incoming information in order to handle cases of missing knowledge. Floriana Esposito received the Laurea degree in electronic Physics from the University of Bari, Italy, in 1970. Since 1994 is Full Professor of Computer Science at the University of Bari and Dean of the Faculty of Computer Science from 1997 to 2002. She founded and chairs the Laboratory for Knowledge Acquisition and Machine Learning of the Department of Computer Science. Her research activity started in the field of numerical models and statistical pattern recognition. Then her interests moved to the field of Artificial Intelligence and Machine Learning. The current research concerns the logical and algebraic foundations of numerical and symbolic methods in machine learning with the aim of the integration, the computational models of incremental and multistrategy learning, the revision of logical theories, the knowledge discovery in data bases. Application include document classification and understanding, content based document retrieval, map interpretation and Semantic Web. She is author of more than 270 scientific papers and is in the scientific committees of many international scientific Conferences in the field of Artificial Intelligence and Machine Learning. She co-chaired ICML96, MSL98, ECML-PKDD 2003, IEA-AIE 2005, ISMIS 2006. Stefano Ferilli was born in 1972. After receiving his Laurea degree in Information Science in 1996, he got a Ph.D. in Computer Science at the University of Bari in 2001. Since 2002 he is an Assistant Professor at the Department of Computer Science of the University of Bari. His research interests are centered on Logic and Algebraic Foundations of Machine Learning, Inductive Logic Programming, Theory Revision, Multi-Strategy Learning, Knowledge Representation, Electronic Document Processing and Digital Libraries. He participated in various National and European (ESPRIT and IST) projects concerning these topics, and is a (co-)author of more than 80 papers published on National and International journals, books and conferences/workshops proceedings. Teresa M.A. Basile got the Laurea degree in Computer Science at the University of Bari, Italy (2001). In March 2005 she discussed a Ph.D. thesis in Computer Science at the University of Bari titled “A Multistrategy Framework for First-Order Rules Learning.” Since April 2005, she is a research at the Computer Science Department of the University of Bari working on methods and techniques of machine learning for the Semantic Web. Her research interests concern the investigation of symbolic machine learning techniques, in particular of the cooperation of different inferences strategies in an incremental learning framework, and their application to document classification and understanding based on their semantic. She is author of about 40 papers published on National and International journals and conferences/workshops proceedings and was/is involved in various National and European projects. Nicola Di Mauro got the Laurea degree in Computer Science at the University of Bari, Italy. From 2001 he went on making research on machine learning in the Knowledge Acquisition and Machine Learning Laboratory (LACAM) at the Department of Computer Science, University of Bari. In March 2005 he discussed a Ph.D. thesis in Computer Science at the University of Bari titled “First Order Incremental Theory Refinement” which faces the problem of Incremental Learning in ILP. Since January 2005, he is an assistant professor at the Department of Computer Science, University of Bari. His research activities concern Inductive Logic Programming (ILP), Theory Revision and Incremental Learning, Multistrategy Learning, with application to Automatic Document Processing. On such topics HE is author of about 40 scientific papers accepted for presentation and publication on international and national journals and conference proceedings. He took part to the European projects 6th FP IP-507173 VIKEF (Virtual Information and Knowledge Environment Framework) and IST-1999-20882 COLLATE (Collaboratory for Annotation, Indexing and Retrieval of Digitized Historical Archive Materials), and to various national projects co-funded by the Italian Ministry for the University and Scientific Research.     

The Appropriateness of Predicate Invention as Bias Shift Operation in ILP

The task of predicate invention in Inductive Logic Programming is to extend the hypothesis language with new predicates if the vocabulary given initially is insufficient for the learning task. However, whether predicate invention really helps to make learning succeed in the extended language depends on the language bias currently employed.In this paper, we investigate for which commonly employed language biases predicate invention is an appropriate shift operation. We prove that for some restricted languages predicate invention does not help when the learning task fails and we characterize the languages for which predicate invention is useful. We investigate the decidability of the bias shift problem for these languages and discuss the capabilities of predicate invention as a bias shift operation.     

Improving scalability of inductive logic programming via pruning and best-effort optimisation

Inductive Logic Programming (ILP) combines rule-based and statistical artificial intelligence methods, by learning a hypothesis comprising a set of rules given background knowledge and constraints for the search space. We focus on extending the XHAIL algorithm for ILP which is based on Answer Set Programming and we evaluate our extensions using the Natural Language Processing application of sentence chunking. With respect to processing natural language, ILP can cater for the constant change in how we use language on a daily basis. At the same time, ILP does not require huge amounts of training examples such as other statistical methods and produces interpretable results, that means a set of rules, which can be analysed and tweaked if necessary. As contributions we extend XHAIL with (i) a pruning mechanism within the hypothesis generalisation algorithm which enables learning from larger datasets, (ii) a better usage of modern solver technology using recently developed optimisation methods, and (iii) a time budget that permits the usage of suboptimal results. We evaluate these improvements on the task of sentence chunking using three datasets from a recent SemEval competition. Results show that our improvements allow for learning on bigger datasets with results that are of similar quality to state-of-the-art systems on the same task. Moreover, we compare the hypotheses obtained on datasets to gain insights on the structure of each dataset.     

Constrained relative least general generalization for Inducing Constraint Logic Programs

Relative least general generalization proposed by Plotkin, is widely used for generalizing first-order clauses in Inductive Logic Programming, and this paper describes an extension of Plotkin’s work to allow various computation domains: Herbrand Universe, sets, numerical data, ect. The ?-subsumption in Plotkin’s framework is replaced by a more general constraint-based subsumption. Since this replacement is analogous to that of unification by constraint solving in Constraint Logic Programming, the resultant method can be viewed as a Constraint Logic Programming version of relative least general generalization. Constraint-based subsumption, however, leads to a search on an intractably large hypothesis space. We therefore providemeta-level constraints that are used as semantic bias on the hypothesis language. The constraintsfunctional dependency andmonotonicity are introduced by analyzing clausal relationships. Finally, the advantage of the proposed method is demonstrated through a simple layout problem, where geometric constraints used in space planning tasks are produced automatically.