A Review of Statistical Language Processing Techniques

We present a review of some recently developed techniques in the field of natural language processing. This area has witnessed a confluence of approaches which are inspired by theories from linguistics and those which are inspired by theories from information theory: statistical language models are becoming more linguistically sophisticated and the models of language used by linguists are incorporating stochastic techniques to help resolve ambiguities. We include a discussion about the underlying similarities between some of these systems and mention two approaches to the evaluation of statistical language processing systems.     

Prological Language Processing

We describe a Prolog-based approach to the development of language processors (such as preprocessors, frontends, evaluators, tools for software modification and analysis). The design of the corresponding environment Laptob for prological language processing is outlined. Language processor definitions in Laptob are basically Prolog programs. The programs might contain grammars, that is, we consider logic grammars. The programs can be typed, and they can be higher-order. The adaptation and composition of the logic programs themselves is supported by meta-programming. The environment offers tool support for efficient scanning, testing, and application development based on a make-system. We report on recent and ongoing applications of the Prolog-based approach.     

Unnatural Language Processing

Computer-based logic proofs are a form of unnatural language in which the process and structure of proof generation can be observed in considerable detail. We have been studying how students respond to multimodal logic teaching, and performance measures have already indicated that students' pre-existing cognitive styles have a significant impact on teaching outcome. Furthermore, a large corpus of proofs has been gathered via automatic logging of proof development. This paper applies a series of techniques, including corpus statistical methods, to the proof logs. The results indicate that students' cognitive styles influence the structure of their logical discourse, via their differing methods of handling abstract information in diagrams, and transferring information between modalities.     

Micro-Cobol: A Data Processing Language for Microprocessor Systems

New constructs and the essentials of Cobol are combined to Create Micro-Cobol: a new, simple language for small business systems.     

A Compiler for an Array and Vector Processing Language

A compiler for a Pascal-based language Actus is described. The language is suitable for the expression of the type of parallelism offered by both array and vector processors. The implementation described is for the Cray-1 computer. An objective of the implementation has been to construct an optimizing compiler which can be readily adapted for a range of array and vector processors. As a result the machine-dependent sections of the compiler have been clearly identified.     

Natural Language Processing for Industry

Recently, natural language processing applications have become very popular in the industry. Examples of such applications include “semantic” enterprise search engines, document categorizers, speech recognizers and – last but not least – conversational agents, also known as virtual assistants or “chatbots”. The latter in particular are very sought-after in the customer care domain, where the aim is to complement the live agent experience with an artificial intelligence able to help users fulfil a task. In this paper, we discuss the challenges and limitations of industrial chatbot applications, with a particular focus on the “human-in-the-loop” aspect, whereby a cooperation between human and machine takes place in mutual interest. Furthermore, we analyse how the same aspect intervenes in other industrial natural language processing applications.     

Natural Language Processing and the Web

This special issue focuses on applications that innovatively use the Web and Web-scale document collections to create useful resources or applications that let end users navigate the Web more easily. This article is part of a special issue on Natural Language Processing and the Web.     

Paradigm Shift in Natural Language Processing

Machine Intelligence Research - In the era of deep learning, modeling for most natural language processing (NLP) tasks has converged into several mainstream paradigms. For example, we usually adopt...     

自然语言处理中的模型

本文简单介绍了自然语言处理发展的现状,讨论了自然语言处理模型,将其分为三大类：分析模型、统计模型及混合模型。具体介绍了分析模型原理及存在的问题,重点讨论了各种统计模型的特点及局限性,最后简单介绍了混合模型,并指出目前自然语言处理技术中存在的问题。     

COMLEX Syntax – A Large Syntactic Dictionary for Natural Language Processing

This article is a detailed account of COMLEX Syntax, an on-line syntactic dictionary of English, developed by the Proteus Project at New York University under the auspices of the Linguistics Data Consortium. This lexicon was intended to be used for a variety of tasks in natural language processing by computer and as such has very detailed classes with a large number of syntactic features and complements for the major parts of speech and is, as far as possible, theory neutral. The dictionary was entered by hand with reference to hard copy dictionaries, an on-line concordance and native speakers‘intuition. Thus it is without prior encumbrances and can be used for both pure research and commercial purposes. This revised version was published online in August 2006 with corrections to the Cover Date.     

面向车联网的时空事件处理语言STEP

车联网是物联网技术应用于智能交通领域所形成的重要研究领域。复杂事件处理技术是车联网系统数据流处理的重要方法。有别于经典的物联网系统,车联网中数据流包含大量的时间和空间信息。在复杂事件处理技术中,如何有效地表达和处理车联网的时空数据流成为亟待解决的问题。针对该问题,提出了一种时空事件处理语言(spatial-temporal event processing language,STEP)。STEP分别采用时间段和栅格地图作为时间和空间模型。基于该时空模型,首先给出STEP语言的相关时空算子和完整语法,从而有效地表达车联网中时空事件流的时空信息。然后,分别从形式语义学角度引入STEP语言的操作语义,并且从实现角度给出了基于Petri网模型的时空事件流处理算法,从而建立车联网时空事件流处理机制。最后,通过实验说明了基于STEP语言的车联网时空事件流处理机制的有效性。     

巧用ACTION SCRIPTS脚本语言制作海浪效果

Action Scripts是FLASH中一种面向对象的编程语言。运用Action编写一些程序,经过一系列语句的控制和位置及比例的调整,并结合FLASH的强大造型工具就可以用一根很细很小的线条,制作出文件量少,不同于传统手绘的比较适合网页上应用的波浪效果。     

Collaborative Development of Interactive Language Processing Tools

Many research groups are developing software tools as part of their research. In many cases, it is hard to integrate tools developed by different groups into new, interactive, applications.This talk focuses on the question how tools in the area of language definition and language processing could be developed in a collaborative fashion. In the first part of the talk, the basic infra-structural needs for such an approach are described: how to represent data, how to coordinate the operation of tools, how to identify the necessary common services?In the second part of the talk, I will sketch our own experience with collaborative development of language processing tools in the ASF+SDF Meta-Environment [2]. Key ingredients are Annotated Terms (ATerms) [3] for exchanging data and the ToolBus [1] for coordinating tools.The presentation ends with open questions: is it desirable and possible to create an infrastructure that really enables the collaborative development of language processing tools across different research groups? If so, what needs to be done to achieve this?     

An Image Processing Language with Icon-Assisted Navigation

This paper describes a generic image processing language IPL, and a programming environment supporting the language primitives for an image information system. The central notion of IPL is that it allows the user to navigate through the image database and manipulate images using generalized icons. The image processing language IPL consists of three subsets: the logical image processing language LIPL, the interactive image processing language IIPL, and the physical image processing language PIPL. This paper presents the main concepts of this generic language, some examples, and a scenario.     

面向自然语言处理的预训练技术研究综述

近年来,随着深度学习的快速发展,面向自然语言处理领域的预训练技术获得了长足的进步。早期的自然语言处理领域长期使用Word2Vec等词向量方法对文本进行编码,这些词向量方法也可看作静态的预训练技术。然而,这种上下文无关的文本表示给其后的自然语言处理任务带来的提升非常有限,并且无法解决一词多义问题。ELMo提出了一种上下文相关的文本表示方法,可有效处理多义词问题。其后,GPT和BERT等预训练语言模型相继被提出,其中BERT模型在多个典型下游任务上有了显著的效果提升,极大地推动了自然语言处理领域的技术发展,自此便进入了动态预训练技术的时代。此后,基于BERT的改进模型、XLNet等大量预训练语言模型不断涌现,预训练技术已成为自然语言处理领域不可或缺的主流技术。文中首先概述预训练技术及其发展历史,并详细介绍自然语言处理领域的经典预训练技术,包括早期的静态预训练技术和经典的动态预训练技术;然后简要梳理一系列新式的有启发意义的预训练技术,包括基于BERT的改进模型和XLNet;在此基础上,分析目前预训练技术研究所面临的问题;最后对预训练技术的未来发展趋势进行展望。