Symbolic and Neural Learning Algorithms: An Experimental Comparison

Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a distributed output encoding.     

结合网络表示学习和文本卷积网络的类案发现

作为“智慧法院”的核心应用之一,相似裁判文书的发现有助于解决司法过程中裁判尺度不统一、类案不同、量刑不规范等问题。目前,一部分方法侧重于从裁判文书中总结领域特征,并将这些特征融入到语言模型中来提升相似文书发现的效果。另一部分工作将其转化为分类任务,利用有监督学习模型来进行建模与预测。然而,已有的方法没有考虑将语言模型与分类模型各自的优势进行结合。为此,提出一种基于网络表示学习（network representation learning）和文本卷积网络（convolutional neural network for texts）的类案发现方法。方法分别从无监督学习与有监督学习的视角来建模裁判文书中的信息,并根据法律知识体系对原有模型的负采样方法（negative sampling）进行改进。最终,方法设计了一种较为合理的投票机制将两类模型的结果进行融合。实验结果表明,提出的联合方法较已有方法能在类案发现任务中取得更高的推送准确率。     

Combining Symbolic Constraint Solvers on Algebraic Domains

In the context of constraint logic programming and theorem proving, the development of constraint solvers on algebraic domains and their combination is of prime interest. As an example, a constraint solver in finite algebras is presented for a constraint language including for instance equations, disequations and inequations. By extending techniques used for the combination of unification in disjoint equational theories, we show how to combine constraint solvers on different algebraic domains that may share some constant symbols. We illustrate this technique by combining the constraint solver in finite algebras with other unification algorithms, and with another constraint solver on a different finite algebra.     

Combining Symmetry Reduction and Under-Approximation for Symbolic Model Checking

This work presents a collection of methods that integrate symmetry reduction and under-approximation with symbolic model checking in order to reduce space and time. The main objective of these methods is falsification. However, under certain conditions, they can provide verification as well.We first present algorithms that use symmetry reduction to perform on-the-fly model checking for temporal safety properties. These algorithms avoid building the orbit relation and choose representatives on-the-fly while computing the reachable states. We then extend these algorithms to check liveness properties as well. In addition, we introduce an iterative on-the-fly algorithm that builds subsets of the orbit relation rather than the full relation.Our methods are fully automatic once the user supplies some basic information about the symmetry in the verified system. Moreover, the methods are robust and work correctly even if the information supplied by the user is incorrect. Furthermore, the methods return correct results even when the computation of the symmetry reduction has not been completed due to memory or time explosion.We implemented our methods within the IBM model checker Rule-Base and compared their performance to that of RuleBase. In most cases, our algorithms outperformed RuleBase in both time and space.     

基于神经网络的符号知识获取方法

本文基于神经网络的知识获取研究进行了综述,介绍了几种有效的模型和方法,并通过比较分析,提出了进一步开展研究工作的意见和看法。     

Combining Committee-Based Semi-Supervised Learning and Active Learning

Many data mining applications have a large amount of data but labeling data is usually difficult, expensive, or time consuming, as it requires human experts for annotation. Semi-supervised learning addresses this problem by using unlabeled data together with labeled data in the training process. Co-Training is a popular semi-supervised learning algorithm that has the assumptions that each example is represented by multiple sets of features (views) and these views are sufficient for learning and independent given the class. However, these assumptions are strong and are not satisfied in many real-world domains. In this paper, a single-view variant of Co-Training, called Co-Training by Committee (CoBC) is proposed, in which an ensemble of diverse classifiers is used instead of redundant and independent views. We introduce a new labeling confidence measure for unlabeled examples based on estimating the local accuracy of the committee members on its neighborhood. Then we introduce two new learning algorithms, QBC-then-CoBC and QBC-with-CoBC, which combine the merits of committee-based semi-supervised learning and active learning. The random subspace method is applied on both C4.5 decision trees and 1-nearest neighbor classifiers to construct the diverse ensembles used for semi-supervised learning and active learning. Experiments show that these two combinations can outperform other non committee-based ones.     

人工神经网络的符号解释

提出了人工神经网络符号解释的基本过程，并详细阐述了该过程中的三个重要步骤：网络的构建、规则的提取以及规则的评估。     

代数符号计算的神经网络模型

文中给出一种p-adic数制式非对称连接神经网络模型,该网络在整个矢量空间只有唯一平衡点,因而可获得问题的最优解,且在存在计算误差,这种神经网络保持高度并行结构,可用了代数符号计算,本文重点分析了实现神经网络的方法,给代数符号计算提供了一个新的计算模型。     

Analytica – An Experiment in Combining Theorem Proving and Symbolic Computation

Analytica is an automatic theorem prover for theorems in elementary analysis. The prover is written in the Mathematica language and runs in the Mathematica environment. The goal of the project is to use a powerful symbolic computation system to prove theorems that are beyond the scope of previous automatic theorem provers. The theorem prover is also able to deduce the correctness of certain simplification steps that would otherwise not be performed. We describe the structure of Analytica and explain the main techniques that it uses to construct proofs. Analytica has been able to prove several nontrivial theorems. In this paper, we show how it can prove a series of lemmas that lead to the Bernstein approximation theorem.     

面向事件抽取的深度与主动联合学习方法

事件抽取旨在从非结构化的文本中抽取出事件的信息,并以结构化的形式予以呈现。监督学习作为基础的事件抽取方法往往受制于训练语料规模小、类别分布不平衡和质量参差不齐的问题。同时,传统基于特征工程的事件抽取方法往往会产生错误传递的问题,且特征工程较为复杂。为此,该文提出了一种联合深度学习和主动学习的事件抽取方法。该方法将RNN模型对触发词分类的置信度融入在主动学习的查询函数中,以此在主动学习过程中提高语料标注效率,进而提高实验的最终性能。实验结果显示,这一联合学习方法能够辅助事件抽取性能的提升,但也显示,联合模式仍有较高的提升空间,有待进一步思考和探索。     

进化算法与符号执行结合的程序复杂度分析方法

程序的最坏执行路径是计算程序复杂度的一项重要指标,有助于发现系统可能存在的复杂性漏洞.近年来将符号执行应用于程序复杂度分析的研究取得了不小的进展,但现有方法存在通用性较差、分析时间较长的问题.文中提出一种面向最坏路径探测的进化算法——EvoWca,其核心思想是利用程序在较小输入规模下的已知最坏路径特征指导较大输入规模下初始路径集合的构建,然后模拟进化算法,对路径进行组合、突变和选择迭代,使得在搜索范围内探测到的最坏路径逼近于最坏时间复杂度对应的路径.基于该算法实现了一个用于程序复杂度分析的原型工具EvoWca2j,使用该工具和已有技术对一组Java程序进行最坏路径探索和执行效率评估,实验结果表明,相比现有方法,EvoWca2j的通用性和探索效率都有明显提高.     

结合表示学习与嵌入子空间学习的降维方法

在对样本数据进行降维时,子空间学习模型无法揭示数据结构和处理训练样本外的新样本.提出一种融合表示学习和嵌入子空间学习的降维方法.将低秩表示、加权稀疏表示和低维子空间学习构建到一个统一的框架中,并采用交替优化策略,实现数据表示系数矩阵和数据投影矩阵的同时学习和相互优化,最终达到重建效果最优的降维精度.在3个数据库上的实验...     

半监督学习和主动学习相结合的浅层语义分析

语义分析是基于内容的文本挖掘领域的重要技术和研究难点。有监督机器学习方法受限于标注语料的规模,在小规模标注样本中难以获取较高性能。本文面向浅层语义分析任务,采用一种新颖的半监督学习方法——直推式支持向量机,并结合其训练特点提出了基于主动学习的样本优化策略。实验表明,本文提出的浅层语义分析方法通过整合主动学习与半监督学习,在小规模标注样本环境中取得了良好的学习效果。     

Octopus: Combining Learning and Parallel Search

This paper presents Octopus, an automated theorem-proving system that combines learning and parallel search. The learning technique involves proving a simpler version of a given theorem and then using what it has learned to prove the given theorem. As of January 2004 Octopus had successfully proved 43 of the 1.0-rated theorems of the TPTP Problem Library.     

基于符号神经网络的数据溶合系统的研究

本文介绍一种把多个神经网络和符号推理结合在一起用于移动式机器人多传感器数据溶合的新技术。用该技术实现的系统不仅能够准确定位，而且可以增强机器人的应变能力。     

一种自动机学习和符号化执行的软件自动测试方法

针对高可信软件提出一种软件脆弱性自动测试方法。与传统测试方法不同,该方法对待测试程序进行预处理,使用自动机学习算法构造软件与环境交互的抽象机模型,在符号化执行迭代过程中利用抽象机模型指导符号化执行,并动态生成测试数据,同时精化交互抽象机用于后继的符号化迭代测试。解决了传统符号化执行测试技术中缺乏指引、具有较高盲目性的问题,同时也提高了符号化执行测试的效率和代码覆盖率。     

融合语义角色的神经机器翻译

近年来,深度学习取得了重大突破,融合深度学习技术的神经机器翻译逐渐取代统计机器翻译,成为学术界主流的机器翻译方法。然而,传统的神经机器翻译将源端句子看作一个词序列,没有考虑句子的隐含语义信息,使得翻译结果与源端语义不一致。为了解决这个问题,一些语言学知识如句法、语义等被相继应用于神经机器翻译,并取得了不错的实验效果。语义角色也可用于表达句子语义信息,在神经机器翻译中具有一定的应用价值。文中提出了两种融合句子语义角色信息的神经机器翻译编码模型,一方面,在句子词序列中添加语义角色标签,标记每段词序列在句子中担当的语义角色,语义角色标签与源端词汇共同构成句子词序列;另一方面,通过构建源端句子的语义角色树,获取每个词在该语义角色树中的位置信息,将其作为特征向量与词向量进行拼接,构成含语义角色信息的词向量。在大规模中-英翻译任务上的实验结果表明,相较基准系统,文中提出的两种方法分别在所有测试集上平均提高了0.9和0.72个BLEU点,在其他评测指标如TER(Translation Edit Rate)和RIBES(Rank-based Intuitive Bilingual Evaluation Score)上也有不同程度的性能提升。进一步的实验分析显示,相较基准系统,文中提出的融合语义角色的神经机器翻译编码模型具有更佳的长句翻译效果和翻译充分性。     

融合TuckER嵌入和强化学习的知识推理

知识推理是补全知识图谱的重要方法,旨在根据图谱中已有的知识,推断出未知的事实或关系.针对多数推理方法仍存在没有充分考虑实体对之间的路径信息,且推理效率偏低、可解释性差的问题,提出了将TuckER嵌入和强化学习相结合的知识推理方法 TuckRL (TuckER embedding with reinforcement learning).首先,通过TuckER嵌入将实体和关系映射到低维向量空间,在知识图谱环境中采用策略引导的强化学习算法对路径推理过程进行建模,然后在路径游走进行动作选择时引入动作修剪机制减少无效动作的干扰,并将LSTM作为记忆组件保存智能体历史动作轨迹,促使智能体更准确地选择有效动作,通过与知识图谱的交互完成知识推理.在3个主流大规模数据集上进行了实验,结果表明TuckRL优于现有的大多数推理方法,说明将嵌入和强化学习相结合的方法用于知识推理的有效性.     

融合互利共生和透镜成像学习的HHO算法

针对哈里斯鹰优化算法收敛速度慢、易陷入局部最优的问题,提出一种融合互利共生和透镜成像学习的哈里斯鹰优化算法(improved Harris hawks optimization,IHHO).利用Tent混沌映射初始化种群,增加种群多样性,提高算法寻优性能;在探索阶段融入一种互利共生思想,并引入非线性惯性因子,以增强种群...     

融合深度学习和语义树的草图识别方法

现有的草图识别框架利用整幅图像作为网络输入,草图识别过程可解释性较差.文中融合深度学习和语义树,提出草图语义网(Sketch-Semantic Net).首先对草图进行部件分割,将单幅完整的草图分割为多个具有语义概念的部件图.然后利用深度迁移学习识别草图部件.最后通过语义树的语义概念关联部件同部件所属草图对象类别,较好地弥补sketch图像从底层语义到高层语义之间的语义鸿沟.在广泛应用的草图分割数据集上的实验验证文中方法的有效性.