Computational models for integrating linguistic and visual information: A survey

This paper surveys research in developing computational models for integrating linguistic and visual information. It begins with a discussion of systems which have been actually implemented and continues with computationally motivated theories of human cognition. Since existing research spans several disciplines (e.g., natural language understanding, computer vision, knowledge representation), as well as several application areas, an important contribution of this paper is to categorize existing research based on inputs and objectives. Finally, some key issues related to integrating information from two such diverse sources are outlined and related to existing research. Throughout, the key issue addressed is the correspondence problem, namely how to associate visual events with words and vice versa.     

国内外隐喻知识库建设综述

就国外的几个隐喻知识库,包括Master Metaphor List、Senseframe、MetaBank、Metalude、Hamburg Metaphor Database、ATTMeta以及国内的隐喻句库的建设进行综述,并对上述隐喻知识库作了简单评述。最后提出了一种建立面向计算的大规模汉语隐喻知识库构建的设想。     

Natural language understanding and Montague grammar1, 2

This paper reevaluates some of the contributions of Montague grammar in view of the increasing importance of computational considerations in linguistic theory and the demand for linguistic theories that can provide support in the design of natural language systems. It also considers Montague grammar in relation to work on lexical semantics and semantic nets. In this perspective the techniques of Montague grammar for systematically linking syntactic form and a model-theoretic semantics emerge as the most significant feature of the theory, while a number of the specific semantic assumptions recede in importance. Yet, with different ways of thinking about the structure mapping between levels of linguistic form and interpretations (e.g., constraint systems), we can also implement this connection using different techniques from what Montague had at his disposal.     

On the referential competence of some machines

The main reason why systems of natural language understanding are often said not to really understand natural language is their lack of referential competence. A traditional system, even an ideal one, cannot relate language to the perceived world, whereas — obviously-a human speaker can. The paper argues that the recognition abilities underlying the application of language to the world are indeed a prerequisite of semantic competence.If a system of the traditional kind were appropriately endowed with the analytic abilities of a system of artificial vision, it would display (partial) referential competence: e.g. it would be able to verify sentences. In response to Searle's objections to the so-called robot reply, the paper argues that such an integrated system could not be considered as essentially on a par with a purely inferential system of the traditional kind, unless one were prepared to regard even the human understanding system as purely syntactic (and therefore incapable of genuine understanding).     

一个实验性的汉语篇章理解系统

本系统从世界现象的组成和人类的记忆结构特点出发, 结合汉语的具体情况, 从意义分析的角度将汉语的词汇分为描述性的词、过程性的词、辅助性的词三类。这三类词分别描述了世界现象中的事实、事件以及语言本身所具有的特性。在此基础上形成了汉语的篇章理解所依赖的知识表示和知识组织形式, 即以事实一事件网络为基本结构的记忆模型通过这个模型建立了汉语篇章理解系统的知识库, 以及与之相应的加工和管理机制系统对汉语篇章的分析是以词为引导进行的。汉语的词直接对应于事实一事件网络中的节点和辅助词表中的词项这些节点和词项综合了语法的、语义的、语用的知识,并且能根据处理的需要及时地为分析过程提供预期本系统通过阅读, 对自己的知识库进行动态的自我管理。在阅读了有关七种鸟类的汉语故事之后, 系统能够学习到有关鸟类的一些新概念, 并能回答相应的问题。汉语篇章理解需要依赖各种知识。这些知识来自语法、语义和语用三个方面为了使计算机能够更好地处理汉语的篇章, 必须对各方面的知识进行合理的组织和管理。由于语言是人们用来描述世界现象, 传递信息的工具, 对于自然语言理解的研究工作有必要从意义分析的角度进行。意义分析就是找出语言是如何对世界现象进行模拟, 进而发现特定的言语活动所描述的有关世界现象特征及其相互关系的过程。本文从汉语的词与世界现象的对应关系出发, 就汉语理解系统的建造进行了初步尝试。     

自然语言处理中逻辑词的知识图分析

知识图是一种新的知识表示方法。本文从本体论的角度出发,将知识图的本体论分别与Aristotle、Kant和Peirce的三种知识表示的本体论进行了比较,表明知识图方法的有效性以及本原性,说明知识图是一种更为一般的知识表示方法。从知识图本体论的观点,研究了各类逻辑词的知识图表示。本文结合汉语的特点,从结构的角度,研究并揭示了逻辑词的共性和规律性。进一步阐明知识图“结构就是含义”的思想。逻辑词的知识图分析将为自然语言分析中词典的建立奠定基础。     

一种智能辅导系统中建模的实现方法

针对自然语言中数学领域的特点,提出了一种智能辅导系统中建立模型的方法。该方法通过对语义理解结果的名词聚类分析,调用静态知识库中相应的内涵模型,根据静态知识库中对内涵模型的描述在语义理解结果中搜集相应的信息使抽象概念具体化。在此基础上依据领域知识对题目进行信息挖掘,并根据题目类型对信息进行约简,从而建立模型。这种建模方法在智能辅导系统中得到了较好的应用。     

从故事到动画片--全过程计算机辅助动画自动生成

论述了作者提出并实现的全过程计算机辅助动画自动生成技术,即从用自然语言书写的故事开始,直至最终生成动画,每一步都是在计算机辅助下完成的,这是一种新的基于人工智能的动画生成技术.文中还分别从自然语言理解、故事理解、动画设计和生成、规划、动画知识库、常识处理几个方面对该技术进行阐述.     

MODELS OF HUMAN COMMUNICATION: DIALOGUE WITH COMPUTER

A model of understanding texts of a natural language is presented, which consists of three levels. On the first level, the system uses only information in the given text. On the second level, it employs additional information from a knowledge base. On the third level, understanding is connected with some activity of the system stimulated by the available information.     

REASONING WITH BACKGROUND KNOWLEDGE–A THREE-LEVEL THEORY

We address the problem of reasoning in cases when knowledge bases containing background knowledge are understood not as sets of formulas (rules and facts) but as collections of partially ordered theories. In our system, the usual, two-part logical structures, consisting of a metalevel and an object level, are augmented by a third level–a referential level. The referential level is a partially ordered collection of theories; it encodes background knowledge. As usual, current situations are described on the object level, and the metalevel is a place for rules that can eliminate some of the models permitted by the object level and the referential level. As a logic of reasoning the system generalizes the standard model of a rational agent: deducing actions and deriving new information about the world from a logical theory–its knowledge base. It is a natural logical system in which priorities on the possible readings of predicates, not special rules of inference, are the main source of nonmonotonicity. We introduce a theory forming operator PT(x) to exploit the priorities, and we investigate its basic logical properties. Then we show how such a system can be augmented by metarules. Although this paper concentrates on basic logical properties of the new theory, this formalism has already been applied to model a number of natural language phenomena such as the notion of text coherence, Gricean maxims, vagueness, and a few others. The paper also briefly compares it with the model of background knowledge of CYC, as proposed by Lenat and Guha.     

PAU: PARSING AND UNDERSTANDING WITH UNIFORM SYNTACTIC, SEMANTIC, AND IDIOMATIC REPRESENTATIONS

PAU is an all-paths chart-based unification parser that uses the same uniform representation for regular syntax, irregular syntax such as idioms, and semantics. PAU's representation has very little redundancy, simplifying the task of adding new semantics and syntax fo PAU's knowledge base. PAU uses relations between the syntax and semantics to avoid the proliferation of rules found in semantic grammars. By encoding semantics at the same level of representation as syntax, PAU is able to use semantic constraints early in the parse to eliminate semantically anomalous syntactic interpretations. Examples are given to show how PAU can handle the many eccentricities of different idioms using the same mechanisms as are used to handle regular syntax and semantics. These include the ability of some idioms, but not other idioms of the same syntactic form to undergo passivization, particle movement, action nominalization, indirect object movement, modification by adjectives, gerundive nominalization, prepositional phrase preposing, and topicalization. PAU's representation is bidirectional and is also used by a companion generator. PAU is designed to be efficient, runs in real time on typical workstations, and is being used in a number of natural language systems.     

面向范畴语法分析的汉语词库的构造及实现

在蒙太鸠语法理论的基础上,利用范畴语法对汉语进行句法分析,并针对汉语范畴动态标注的不确定性进行跨层次松弛关联的计算研究,需要相应地构造范畴化机器词库。本文采用基本词库加扩展生成的思想构建生成的面向范畴语法分析的汉语词库,除具有一般词库的特点外,还对词语的范畴归属、词谓、词用等相关信息给出说明,以供范畴句法分析时选用。实验结果表明,在假设完备的前提下,测试该词库取得了较好的效果。     

基于BERT的任务导向对话系统自然语言理解的改进模型与调优方法

任务导向对话系统的自然语言理解,其目的就是解析用户以自然语言形式输入的语句,并提取出可以被计算机所理解的结构化信息,其包含意图识别和槽填充两个子任务。BERT是近期提出来的一种自然语言处理预训练模型,已有研究者提出基于BERT的任务导向对话系统自然语言理解模型。在此基础上,该文提出一种改进的自然语言理解模型,其编码器使用BERT,而解码器基于LSTM与注意力机制构建。同时,该文提出了该模型的两种调优方法: 锁定模型参数的训练方法、使用区分大小写的预训练模型版本。在基线模型与改进模型上,这些调优方法均能够显著改进模型的性能。实验结果显示,利用改进后的模型与调优方法,可以分别在ATIS和Snips两个数据集上得到0.883 3和0.925 1的句子级准确率。     

Visible speech improves human language understanding: Implications for speech processing systems

Evidence from the study of human language understanding is presented suggesting that our ability to perceive visible speech can greatly influence our ability to understand and remember spoken language. A view of the speaker's face can greatly aid in the perception of ambiguous or noisy speech and can aid cognitive processing of speech leading to better understanding and recall. Some of these effects have been replicated using computer synthesized visual and auditory speech. Thus, it appears that when giving an interface a voice, it may be best to give it a face too.     

基于动态词遮掩的句子匹配预训练模型

BERT通过遮掩语言模型、下一句预测等自监督学习任务学习通用语言规律,在自然语言理解任务中取得了良好效果。但BERT的下一句预测任务不能直接建模句子的语义匹配关系,且随机遮掩策略也不能高效处理句子的关键内容。针对上述问题,该文提出基于动态词遮掩的预训练模型: 基于预训练模型获得句子的向量表示,并通过近似语义计算获取大规模“句子对”预训练数据,最后遮掩重要字词训练遮掩语言模型。在4个句子匹配数据集上的实验表明,使用该文提出的预训练方法,RBT3和BERT base的效果都有一定提升,平均准确率分别提升1.03%和0.61%。     

面向法律文书的自然语言理解

法律人工智能因其高效、便捷的特点,近年来受到社会各界的广泛关注。法律文书是法律在社会生活中最常见的表现形式,应用自然语言理解方法智能地处理法律文书内容是一个重要的研究和应用方向。该文梳理与总结面向法律文书的自然语言理解技术,首先介绍了五类面向法律文书的自然语言理解任务形式: 法律文书信息提取、类案检索、司法问答、法律文书摘要和判决预测。然后,该文探讨了运用现有自然语言理解技术应对法律文书理解的主要挑战,指出需要解决好法律文书与日常生活语言之间的表述差异性、建模好法律文书中特有的推理与论辩结构,并且需要将法条、推理模式等法律知识融入自然语言理解模型。     

Natural Language Processing and User Modeling: Synergies and Limitations

The fields of user modeling and natural language processing have been closely linked since the early days of user modeling. Natural language systems consult user models in order to improve their understanding of users' requirements and to generate appropriate and relevant responses. At the same time, the information natural language systems obtain from their users is expected to increase the accuracy of their user models. In this paper, we review natural language systems for generation, understanding and dialogue, focusing on the requirements and limitations these systems and user models place on each other. We then propose avenues for future research.     

Computational Mechanisms for Metaphor in Languages: A Survey

Metaphor computation has attracted more and more attention because metaphor, to some extent, is the focus of mind and language mechanism. However, it encounters problems not only due to the rich expressive power of natural language but also due to cognitive nature of human being. Therefore machine-understanding of metaphor is now becoming a bottle-neck in natural language processing and machine translation. This paper first suggests how a metaphor is understood and then presents a survey of current computational approaches, in terms of their linguistic historical roots, underlying foundations, methods and techniques currently used, advantages, limitations, and future trends. A comparison between metaphors in English and Chinese languages is also introduced because compared with development in English language Chinese metaphor computation is just at its starting stage. So a separate summarization of current progress made in Chinese metaphor computation is presented. As a conclusion, a few suggestions are proposed for further research on metaphor computation especially on Chinese metaphor computation.     

人工智能的应用及困惑

知识获取、知识表示和知识利用成为人工智能的中心问题.但在当前的智能系统中,他们都极不完善.本文通过对人工智能的四大应用领域:专家系统、智能教学系统、自然语言理解及自动程序设计进行分析,说明了在人工智能的应用领域中困难重重.     

An Intelligent Lexicon for Contextual Word Sense Discrimination

This paper presents a lexical model dedicated to the semanticrepresentation and interpretation of individual words inunrestricted text, where sense discrimination is difficult toassess. We discuss the need of a lexicon including local inferencemechanisms and cooperating with as many other knowledge sources(about syntax, semantics and pragmatics) as possible. We suggest aminimal representation (that is, the smallest representationpossible) acting as a bridge between a conceptual representation andthe microscopic sense variations of lexical semantics. We describean interpretation method providing one or many alternativecandidate(s) to the word, as representatives of its meaning in thesentence (and text).