基于自然语言处理的弱监督知识获取系统的实现

知识获取多年来一直被认为是阻碍智能系统开发的瓶颈问题，尤其是互联网时代，大量的信息都以非结构化的文本形式存在。本文运用分布式计算思想设计了一个基于互联网大规模语料库的知识自动获取系统。采用弱监督条件下机器学习的方法对信息自动挖掘和获取，实现机器对知识的自动学习和挖掘、新词词典发现、实体关系模板提取、命名实体识别等功能。利用该系统分别对未登录新词发现和地名识别两种应用进行了实验，运用N gram和互信息(PMI)方法分别取得了72.1%和87.28%的准确率。

Implementation of weakly supervised learning knowledge acquisition system based on natural language processing

Knowledge acquisition has been considered as a bottleneck problem in the development of intelligent systems for many years. Especially in the Internet era, a large number of information exists in the form of unstructured text. This paper introduces a knowledge acquisition system for a large Web page corpus based on distributed computing. This system is designed for automatic information mining and acquisition by the weakly supervised learning method. Computers can realize the automatic learning and mining of knowledge, the discovery of new words dictionary, the extraction of entity relation template, the entity recognition and so on. We represent the N gram model and pairwise mutual information methods for new words recognition and location name entity detection, and the experimental results show the precision are 72.1% and 87.28% respectively.