Learning From Noisy Examples

Machine Learning - The basic question addressed in this paper is: how can a learning algorithm cope with incorrect training examples? Specifically, how can algorithms that produce an...     

PAC Analogues of Perceptron and Winnow Via Boosting the Margin

We describe a novel family of PAC model algorithms for learning linear threshold functions. The new algorithms work by boosting a simple weak learner and exhibit sample complexity bounds remarkably similar to those of known online algorithms such as Perceptron and Winnow, thus suggesting that these well-studied online algorithms in some sense correspond to instances of boosting. We show that the new algorithms can be viewed as natural PAC analogues of the online p-norm algorithms which have recently been studied by Grove, Littlestone, and Schuurmans (1997, Proceedings of the Tenth Annual Conference on Computational Learning Theory (pp. 171–183) and Gentile and Littlestone (1999, Proceedings of the Twelfth Annual Conference on Computational Learning Theory (pp. 1–11). As special cases of the algorithm, by taking p = 2 and p = we obtain natural boosting-based PAC analogues of Perceptron and Winnow respectively. The p = case of our algorithm can also be viewed as a generalization (with an improved sample complexity bound) of Jackson and Craven's PAC-model boosting-based algorithm for learning sparse perceptrons (Jackson & Craven, 1996, Advances in neural information processing systems 8, MIT Press). The analysis of the generalization error of the new algorithms relies on techniques from the theory of large margin classification.     

LT Revisited: Explanation-Based Learning and the Logic of Principia Mathematica

This paper descibes an explanation-based learning (EBL) system based on a version of Newell, Shaw, and Simon's LOGIC-THEORIST (LT). Results of applying this system to propositional calculus problems from Principia Mathematica are compared with results of applying several other versions of the same performance element to these problems. The primary goal of this study is to characterize and analyze differences between non-learning, rote learning (LT's original learning method), and EBL. Another aim is to provide a characterization of the performance of a simple problem solver in the context of the Principia problems, in the hope that these problems can be used as a benchmark for testing improved learning methods, just as problems like chess and the eight puzzle have been used as benchmarks in research on search methods.     

Noise-Tolerant Occam Algorithms and Their Applications to Learning Decision Trees

In the distribution-independent model of concept learning of Valiant, Angluin and Laird have introduced a formal model of noise process, called classification noise process, to study how to compensate for randomly introduced errors, or noise, in classifying the example data. In this article, we investigate the problem of designing efficient learning algorithms in the presence of classification noise. First, we develop a technique of building efficient robust learning algorithms, called noise-tolerant Occam algorithms, and show that using them, one can construct a polynomial-time algorithm for learning a class of Boolean functions in the presence of classification noise. Next, as an instance of such problems of learning in the presence of classification noise, we focus on the learning problem of Boolean functions represented by decision trees. We present a noise-tolerant Occam algorithm for k-DL (the class of decision lists with conjunctive clauses of size at most k at each decision introduced by Rivest) and hence conclude that k-DL is polynomially learnable in the presence of classification noise. Further, we extend the noise-tolerant Occam algorithm for k-DL to one for r-DT (the class of decision trees of rank at most r introduced by Ehrenfeucht and Haussler) and conclude that r-DT is polynomially learnable in the presence of classification noise.     

Explanation-Based Learning: A survey

The paper provides an introductory survey of Explanation-Based Learning (EBL). It attempts to define EBL's position in AI by exploring its relationship to other AI techniques, including other sub-fields of machine learning. Further issues discussed include the form of learning exhibited by EBL and potential applications of the method.     

Efficient PAC Learning for Episodic Tasks with Acyclic State Spaces

This paper considers the problem of computing an optimal policy for a Markov decision process, under lack of complete a priori knowledge of (1) the branching probability distributions determining the evolution of the process state upon the execution of the different actions, and (2) the probability distributions characterizing the immediate rewards returned by the environment as a result of the execution of these actions at different states of the process. In addition, it is assumed that the underlying process evolves in a repetitive, episodic manner, with each episode starting from a well-defined initial state and evolving over an acyclic state space. A novel efficient algorithm for this problem is proposed, and its convergence properties and computational complexity are rigorously characterized in the formal framework of computational learning theory. Furthermore, in the process of deriving the aforementioned results, the presented work generalizes Bechhofer’s “indifference-zone” approach for the ranking & selection problem, that arises in statistical inference theory, so that it applies to populations with bounded general distributions.

关系tri-training:利用无标记数据学习一阶规则

针对目前归纳逻辑程序设计(inductive logic programming,ILP)系统要求训练数据充分且无法利用无标记数据的不足,提出了一种利用无标记数据学习一阶规则的算法——关系tri-training(relational-tri-training,R-tri-training)算法。该算法将基于命题逻辑表示的半监督学习算法tri-training的思想引入到基于一阶逻辑表示的ILP系统,在ILP框架下研究如何利用无标记样例信息辅助分类器训练。R-tri-training算法首先根据标记数据和背景知识初始化三个不同的ILP系统,然后迭代地用无标记样例对三个分类器进行精化,即如果两个分类器对一个无标记样例的标记结果一致,则在一定条件下该样例将被标记给另一个分类器作为新的训练样例。标准数据集上实验结果表明:R-tri-training能有效地利用无标记数据提高学习性能,且R-tri-training算法性能优于GILP(genetic inductive logic programming)、NFOIL、KFOIL和ALEPH。     

Induction Over the Unexplained: Using Overly-General Domain Theories to Aid Concept Learning

This paper describes and evaluates an approach to combining empirical and explanation-based learning called Induction Over the Unexplained (IOU). IOU is intended for learning concepts that can be partially explained by an overly-general domain theory. An eclectic evaluation of the method is presented which includes results from all three major approaches: empirical, theoretical, and psychological. Empirical results show that IOU is effective at refining overly-general domain theories and that it learns more accurate concepts from fewer examples than a purely empirical approach. The application of theoretical results from PAC learnability theory explains why IOU requires fewer examples. IOU is also shown to be able to model psychological data demonstrating the effect of background knowledge on human learning.     

Plausible Justification Trees: A Framework for Deep and Dynamic Integration of Learning Strategies

This article describes a framework for the deep and dynamic integration of learning strategies. The framework is based on the idea that each single-strategy learning method is ultimately the result of certain elementary inferences (like deduction, analogy, abduction, generalization, specialization, abstraction, concretion, etc.). Consequently, instead of integrating learning strategies at a macro level, we propose to integrate the different inference types that generate individual learning strategies. The article presents a concept-learning and theory-revision method that was developed in this framework. It allows the system to learn from one or from several (positive and/or negative) examples, and to both generalize and specialize its knowledge base. The method integrates deeply and dynamically different learning strategies, depending on the relationship between the input information and the knowledge base. It also behaves as a single-strategy learning method whenever the applicability conditions of such a method are satisfied.     

A Market-based Solution to the Multiple Traveling Salesmen Problem

This paper describes a market-based solution to the problem of assigning mobile agents to tasks. The problem is formulated as the multiple depots, multiple traveling salesmen problem (MTSP), where agents and tasks operate in a market to achieve near-optimal solutions. We consider both the classical MTSP, in which the sum of all tour lengths is minimized, and the Min-Max MTSP, in which the longest tour is minimized. We compare the market-based solution with direct enumeration in small scenarios, and show that the results are nearly optimal. For the classical MTSP, we compare our results to linear programming, and show that the results are within 1 % of the best cost found by linear programming in more than 90 % of the runs, with a significant reduction in runtime. For the Min-Max case, we compare our method with Carlsson’s algorithm and show an improvement of 5 % to 40 % in cost, albeit at an increase in runtime. Finally, we demonstrate the ability of the market-based solution to deal with changes in the scenario, e.g., agents leaving and entering the market. We show that the market paradigm is ideal for dealing with these changes during runtime, without the need to restart the algorithm, and that the solution reacts to the new scenarios in a quick and near-optimal way.     

A Computational Approach to the Collective Action Problem: Assessment of Alternative Learning Rules

We sketch here the basis of a behavioral theory of non-market decision making or collective action. Departing from the basic social problem, the coordination of individual actions when individual rationality is opposed to collective rationality, we model a population of agents choosing their level of individual cooperation. The social dilemma that emerges may be solved in a bounded rationality evolutionary context. We find that the efficiency embodied in the solutions is dependent on the type of learning individuals adopt. Additional returns to the individual from collective contributions and discounting the future play key roles in the determination of the solution. We conclude that the emergent properties of the social cooperation agree with the findings in the experimental literature: cooperation, although not optimal, is a fact, and institutional settings affect the outcomes in a significant way.     

基于再励学习蚁群算法的多约束QoS路由方法

本文研究了多约束QoS路由问题,给出基于模糊评判的路由模型,实现了多QoS约束的综合优化;同时提出一种再励学习蚁群路由算法对该问题进行求解,算法通过对蚂蚁搜索路径进行评价产生再励信号,并根据再励信号采取了不同的信息素更新策略,提高了算法的寻优能力和收敛速度。仿真实验表明,该算法能快速得到较大程度满足业务QoS要求的路径。     

改进遗传算法求解走班制下的排课问题

针对新高考政策背景下课程安排的问题，分析了走班制制度下排课的特点，结合现实中学校的资源条件以及学生和教师对课程安排的偏好，设计了有针对性的生成可行解方法和适应度函数，克服了传统遗传算法无法适用于走班制下排课的问题。使用改进后的遗传算法分别对走班课程和非走班课程进行排课，其中对于走班课程，将每个年级的班级分为若干个组，同一个组中的所有相同课程安排在相同的时间。实验结果证明了该算法可以得到合理的排课方案。     

基于蓝牙技术的移动远程教育系统实现方案

提出了利用蓝牙技术实现远程教育系统的移动终端的方案，并阐述了应用总体方案及软硬件实现途径，特别对蓝牙设备间互连与通信的具体建立进行了详细的介绍。     

工程类学科网络课程中的三维动画实现方法研究

分析了网络课程中三维动画素材研制中的主要问题,提出了一种适合于工程类学科网络课程的三维动画素材制作方法。     

DSmT相关证据模型及反问题近似解求法

介绍了一种新的信息融合理论——DSmT（Dezert-Smarandache Theory）。在DSmT下,鉴于实际处理的证据经常是相关证据,提出了一种新的模型表示相关证据。其中两个相关证据各自由一个独立源证据和一个相关源证据正交和合成,相关证据的合成就归结为这两个独立源证据和这个相关源证据的正交和合成。辨识独立源证据是一个反问题,该反问题可能不存在唯一精确解,此时采用了粒子群优化算法求其近似解。     

Rademacher复杂度在统计学习理论中的研究

假设空间复杂性是统计学习理论中用于分析学习模型泛化能力的关键因素.与数据无关的复杂度不同,Rademacher复杂度是与数据分布相关的,因而通常能得到比传统复杂度更紧致的泛化界表达.近年来,Rademacher复杂度在统计学习理论泛化能力分析的应用发展中起到了重要的作用.鉴于其重要性,本文梳理了各种形式的Rademacher复杂度及其与传统复杂度之间的关联性,并探讨了基于Rademacher复杂度进行学习模型泛化能力分析的基本技巧.考虑样本数据的独立同分布和非独立同分布两种产生环境,总结并分析了Rademacher复杂度在泛化能力分析方面的研究现状.展望了当前Rademacher复杂度在非监督框架与非序列环境等方面研究的不足,及其进一步应用与发展.     

分层边缘联邦学习：一种基于契约论的双层激励机制

联邦学习(Federated Learning,FL)是一种分布式神经网络训练的方法,它使多个终端协作训练一个共享的神经网络模型,同时保证训练数据存在于终端上。然而,FL涉及数千个异构分布式终端设备。在这种情况下,低效的通信是FL的关键瓶颈。为了解决上述问题,一种融合边缘计算的分层联邦学习(Hierarchical FL,HFL)框架被提出来了。尽管HFL有上述的巨大好处,但是激励机制设计的挑战在HFL框架中尚未得到解决。因此,提出了一种基于契约论的双层激励机制方法;然后,基于CIFAR10数据集的实验结果验证了该激励机制的有效性。