Combining natural language with direct manipulation: the conceptual framework for a hybrid human-computer interface

Abstract

This paper describes the development of a conceptual model of an interaction system for future human-computer interaction. It is suggested that an integrated system is the most promising solution for diverse users and tasks. A ‘ hybrid interaction system’ is considered, which tries to efficiently direct manipulation, menu selection, and natural language. The mechanism of the natural language interface module is described in more detail. Finally, the allocation of functionalities for generalized tasks on the conceptual model is considered from the taxonomic approach.     

Conceptual feature generation for textual information using a conceptual network constructed from Wikipedia

A proper semantic representation of textual information underlies many natural language processing tasks. In this paper, a novel semantic annotator is presented to generate conceptual features for text documents. A comprehensive conceptual network is automatically constructed with the aid of Wikipedia that has been represented as a Markov chain. Furthermore, semantic annotator gets a fragment of natural language text and initiates a random walk to generate conceptual features that represent topical semantic of the input text. The generated conceptual features are applicable to many natural language processing tasks where the input is textual information and the output is a decision based on its context. Consequently, the effectiveness of the generated features is evaluated in the task of document clustering and classification. Empirical results demonstrate that representing text using conceptual features and considering the relations between concepts can significantly improve not only the bag of words representation but also other state‐of‐the‐art approaches.     

Modelling Route Instructions for Robust Human-Robot Interaction on Navigation Tasks

In this paper, we demonstrate the use of qualitative spatial modelling as the foundation for the conceptual representation of route instructions, to enable robust humanrobot interaction on navigation tasks. Our conceptual model is motivated by empirical studies on route navigation, and combines Qualitative Orientation Calculi for spatial reasoning using directional orientation information and topological maps for structuring route segments and routes. Moreover, we present a formal de nition of the conceptual model using the algebraic speci cation language CASL for syntactic and semantic checking, consistency checking and veri cation. Finally, we introduce a generic route graph concept and its formalization. The instantiation of the generic route graph at di erent abstraction levels provides a formal foundation for linking the conceptual model to a global environment map used by an intelligent robot, e.g., a semi-autonomous wheelchair, to carry out human navigation tasks.     

Inference and the computer understanding of natural language

The notion of computer understanding of natural language is examined relative to inference mechanisms designed to function in a language-free deep conceptual base (Conceptual Dependency). The conceptual analysis of a natural language sentence into this conceptual base, and the nature of the memory which stores and operates upon these conceptual structures are described from both theoretical and practical standpoints. The various types of inferences which can be made during and after the conceptual analysis of a sentence are defined, and a functioning program which performs these inference tasks is described. Actual computer output is included.     

A form-based natural language front-end to a CIM database

The paper presents a methodology for developing a user interface that combines fourth generation interface tools (SQL forms) with a natural language processor for a database management system. The natural language processor consists of an index, a lexicon and a parser. The index is used to uniquely identify each form in the system through a conceptual representation of its purpose. The form fields specify database or nondatabase fields whose values are either entered by the user (user-defined) or are derived by the form (system-defined) in response to user input. A set of grammar rules are associated with each form. The lexicon consists of all words recognized by the system, their grammatical categories, roots, their associations (if any) with database objects and forms. The parser scans, a natural language query to identify a form in a bottom-up fashion. The information requested in the user query is determined in a top-down manner by parsing, through the grammar rules associated with the identified form. Extragrammatical inputs with limited deviations from the grammar rules are supported. Combining a natural language processor with SQL forms allows processing data modification tasks without violating any database integrity constraint, having duplicate records, or entering invalid data. A prototype natural language interface is described as a front-end to an ORACLE database for a computer integrated manufacturing system     

KDA: a knowledge-based database assistant with a query guidingfacility

A knowledge-based database assistant (KDA) which integrates a natural language query system with a skeleton-based query guiding facility is provided. When a user works with the KDA natural language query system, the query guiding facility can supply several kinds of skeletons to guide users in performing database retrieval tasks. A semantic network model, S-Net, is introduced to represent the knowledge for natural language query processing and skeleton generation. Methods for implementing the system are discussed     

基于深度学习的语言模型研究进展

语言模型旨在对语言的内隐知识进行表示,作为自然语言处理的基本问题,一直广受关注．基于深度学习的语言模型是目前自然语言处理领域的研究热点,通过预训练-微调技术展现了内在强大的表示能力,并能够大幅提升下游任务性能．本文围绕语言模型基本原理和不同应用方向,以神经概率语言模型与预训练语言模型作为深度学习与自然语言处理结合的切入点,从语言模型的基本概念和理论出发,介绍了神经概率与预训练模型的应用情况和当前面临的挑战,对现有神经概率、预训练语言模型及方法进行对比和分析．我们又从新型训练任务和改进网络结构两方面对预训练语言模型训练方法进行详细阐述,并对目前预训练模型在规模压缩、知识融合、多模态和跨语言等研究方向进行概述和评价．最后总结语言模型在当前自然语言处理应用中的瓶颈,对未来可能的研究重点做出展望．     

基于XML的ER模型的建立

XML已经成为Web上表示结构化和半结构化数据的标准格式,为了描述XML数据的结构和内容,业界已经提出了多个XML模式语言。虽然XML模式对ValidatingXML文档非常有用,但它不适用于要求表示数据有关语义知识的任务,对这样的任务最好使用概念模式。针对XML模式的概念建模,介绍了一种扩展实体关系模型及将用XML模式语言定义的模式转换成扩展实体模式的过程。     

QBD*: a graphical query language with recursion

A system to query databases using diagrams as a standard user interface is proposed. The system, called Query by Diagram* (QBD*), makes use of a conceptual data model, a query language on this model, and a graphical user interface. The conceptual model is the entity-relationship model. The query language, whose expressive power allows recursive queries, supports visual interaction. The main characteristics of the interface are ease of use and the availability of a rich set of primitives for schema selection and query formulation. The expressive power of QBD* and G⁺, which are the only languages allowing recursive queries to be expressed graphically are compared     

基于强化学习的对抗预训练语言建模方法

在大规模无监督语料上的BERT、XLNet等预训练语言模型,通常采用基于交叉熵损失函数的语言建模任务进行训练。模型的评价标准则采用困惑度或者模型在其他下游自然语言处理任务中的性能指标,存在损失函数和评测指标不匹配等问题。为解决这些问题,该文提出一种结合强化学习的对抗预训练语言模型RL-XLNet(Reinforcement Learning-XLNet)。RL-XLNet采用对抗训练方式训练一个生成器,基于上下文预测选定词,并训练一个判别器判断生成器预测的词是否正确。通过对抗网络生成器和判别器的相互促进作用,强化生成器对语义的理解,提高模型的学习能力。由于在文本生成过程中存在采样过程,导致最终的损失无法直接进行回传,故提出采用强化学习的方式对生成器进行训练。基于通用语言理解评估基准(GLUE Benchmark)和斯坦福问答任务(SQuAD 1.1)的实验,结果表明,与现有BERT、XLNet方法相比,RL-XLNet模型在多项任务中的性能上表现出较明显的优势: 在GLUE的六个任务中排名第1,一个任务排名第2,一个任务排名第3。在SQuAD 1.1任务中F₁值排名第1。考虑到运算资源有限,基于小语料集的模型性能也达到了领域先进水平。     

Deriving static and dynamic concepts from software requirements using sophisticated tagging

Natural language requirements specifications form the basis for the subsequent phase of the information system development process, namely the development of conceptual schemata. Both, the textual as well as the conceptual representations are not really appropriate for being thoroughly captured and validated by the ‘requirement holders’, i.e. the end users. Therefore, in our approach the textual specifications are firstly linguistically analyzed and translated into a so-called conceptual predesign schema. That schema is formulated using an interlingua which is based on a lean semantic model, thus allowing users to participate more efficiently in the design and validation process. After validation, the predesign schema is mapped to a conceptual representation (e.g. UML). The sequence of these translation and transformation steps is described by the “NIBA workflow”. This paper focuses on the information supporting a step by step mapping of natural language requirements specifications to a conceptual model, and on how that information is gained. On particular, we present a four-level interpretation of tagging-output.     

基于语义Web的受控自然语言系统推理模型

虽然相关研究组织提供了语义Web的一些简化工具,但是对不具备相关背景知识的领域专家来说,语义Web的可用性较低。提出了基于语义Web的受控自然语言系统推理模型,以解决这个问题。首先给出受控自然语言系统推理模型框架;然后分析受控自然语言的语言处理部分,提出基于WordNct的受控自然语言系统的本体词库模型和基于本体词库的受控自然语言解释器,把受控自然语言转换成中间表达语言篇章表述结构;最后通过推理部分把篇章表述结构转换成语义Web的本体和规则,通过模板工具映射成Jess的事实和规则,根据预定义的语义Web的公理和定理对受控自然语言进行推理。试验证明此模型大大提高了知识表示建模的效率,也基本满足简单推理任务,具有实用价值。     

Automatic classification of documents in a natural language: A conceptual model

A conceptual model is proposed for a system whose function is to solve the problem of automatic classification of text documents in a natural language, i.e., to determine whether a new text document belongs to a predefined class. The functional requirements of the future system are given. Various representations of natural language texts, as well as statistical and logical-combinatorial methods of text analysis, are discussed. This work may be of interest to specialists in natural-language processing, data mining, and computational linguistics.     

Caesar: an intelligent domestic service robot

In this paper we present Caesar, an intelligent domestic service robot. In domestic settings for service robots complex tasks have to be accomplished. Those tasks benefit from deliberation, from robust action execution and from flexible methods for human?Crobot interaction that account for qualitative notions used in natural language as well as human fallibility. Our robot Caesar deploys AI techniques on several levels of its system architecture. On the low-level side, system modules for localization or navigation make, for instance, use of path-planning methods, heuristic search, and Bayesian filters. For face recognition and human?Cmachine interaction, random trees and well-known methods from natural language processing are deployed. For deliberation, we use the robot programming and plan language Readylog, which was developed for the high-level control of agents and robots; it allows combining programming the behaviour using planning to find a course of action. Readylog is a variant of the robot programming language Golog. We extended Readylog to be able to cope with qualitative notions of space frequently used by humans, such as ??near?? and ??far??. This facilitates human?Crobot interaction by bridging the gap between human natural language and the numerical values needed by the robot. Further, we use Readylog to increase the flexible interpretation of human commands with decision-theoretic planning. We give an overview of the different methods deployed in Caesar and show the applicability of a system equipped with these AI techniques in domestic service robotics.     

A simulation study with a combined network and production systems model of pilot behavior on an ILS-approach

A simulation study was conducted for determining information flow requirements of a pilot-cockpit interface during an ILS-approach. Study steps for modeling, simulating, and analyzing system processes are discussed. Starting with the problem definition which includes relevant flight processes, the network of pilot tasks, and performance measures, the conceptual model of processes is mathematically described by means of algebraic and difference equations as well as production systems. To transform the model into a simulation program, the simulation language SLAM is used. SLAM elements utilized for modeling flight processes and pilot tasks as well as the procedure of model validation are discussed. The analysis of simulation output data, i.e. state trajectories and task timelines, results in the determination of important information requirements useful in early stages of a design project.     

面向事实的概念建模方法研究综述

领域概念知识建模是构建信息系统分析的关键技术和任务之一,同时也是知识工程的瓶颈问题,其难点在于如何正确完整地捕捉和验证领域专家的知识。ER和UML等有属性特性的建模方法能够很好地描述领域知识,但却难以让领域专家确认知识的正确性和完整性。面向事实的信息建模(FOM)是一种完全面向自然语言交流的领域概念知识建模方法,是一种理想的概念建模和本体工程辅助工具。简要分析了概念建模过程,比较了不同概念建模方法,介绍了FOM的技术演化过程,从业务规则、动态建模、模型抽象机制、模型转换和工程应用等方面评述了FOM的研究现状和待研究问题。     

面向机器阅读理解的多任务层次微调模型

机器阅读理解与问答一直以来被认为是自然语言理解的核心问题之一, 要求模型通过给定的文章与问题去挑选出最佳答案. 随着BERT等预训练模型的兴起, 众多的自然语言处理任务取得了重大突破, 然而在复杂的阅读理解任务方面仍然存在一些不足, 针对该任务, 提出了一个基于回顾式阅读器的机器阅读理解模型. 模型使用RoBERTa预...     

Modeling the user's conceptual knowledge in BGP-MS,a user modeling shell system1

The paper describes the modeling of a user's conceptual knowledge in the general user modeling shell system BGP-MS. On the one hand, BGP-MS is a workbench for the develment of a user model in a particular application domain. It supports the definition of the architecture of the individual user model, and of the architecture and the contents of user stereotypes. A rich representation language for conceptual knowledge, a partition mechanism, and flexible graphics-based interfaces are at the disposal of the user model developer. On the other hand, BGP-MS is a runtime user modeling tool aimed at fulfilling central tasks of a user modeling component in an application system. In this mode, the system offers a functional interface for accessing and updating the model of the current user, as well as basic domain-independent inference mechanisms, support for inferences defined by the developer, and a customizable stereotype management utility. The usefulness of BGP-MS will be demonstrated by illustrating how it can support the user modeling of various recently developed application systems.