Building syntax-aware editors for visual languages

Syntax-aware editors are a class of editors prompting users into writing syntactically correct programs by exploiting visual language syntax. They are particularly useful in those application domains where the way a visual symbol spatially relates to others depends from the context. This does not mean constraining users to enter only correct syntactic states in a visual sentence, rather it means detecting both syntax and potential semantic errors as early as possible, and providing error feedbacks in a non-intrusive way during editing. As a consequence, error handling strategies are an essential part of this editing style.In this work, we present a strategy for the automatic generation of syntax-aware visual language editors integrating incremental subsentence parsers into freehand editors. The proposed parsing strategy has turned out to be useful in many application domains involving spatial information systems, thanks to the possibility of interactively prompting feasible visual sentence extensions, and to the presence of a non-correcting error recovery strategy. A first experimental prototype implementing the whole approach has been embedded into the VLDesk system, and empirical studies have been performed in order to verify the performance and the effectiveness of the proposed approach.     

Towards Symbolic Analysis of Visual Modeling Languages

Graph transformation has recently become more and more popular as a general, rule-based visual specification paradigm to formally capture the operational semantics of modeling languages based on metamodeling techniques as demonstrated by benchmark applications focusing on the formal treatment of the Unified Modeling Language (UML). In the paper, we enable model checking-based symbolic verification for such modeling languages by providing a meta-level transformation of well-formed model instances into SAL specifications [4]. We also discuss several optimizations in the translation process that makes our approach efficient and independent of the SAL framework.     

融入句法感知表示进行句法增强的语义解析

在自然语言处理(NLP)中,句法信息是完整句子中词汇与词汇之间的句法结构关系或者依存关系,是一种重要且有效的参考信息.语义解析任务是将自然语言语句直接转化成语义完整的、计算机可执行的语言.在以往的语义解析研究中,少有采用输入源的句法信息来提高端到端语义解析效率的工作.为了进一步提高端到端语义解析模型的准确率和效率,提出...     

人机交互中视觉语言的灰度轮廓权向量差分唇形特征模型

该文结合函数变形模型和灰度轮廓向量模型的特点,给出了一种维数少、有效性高的视觉语言特征—灰度轮廓权向量差分唇形特征模型。该特征融合了嘴唇图像的形状变化信息和灰度信息,能够较完善地描述嘴唇的变化。同时,得出了一种新的视觉特征提取算法。仿真结果表明,该算法与传统的函数变形模型相比,总的特征提取准确率提高了5个百分点,每个发音图像序列特征提取的准确率提高了1.6至9个百分点,每帧图像的特征提取时间由4.6495秒下降到0.4455秒。对“1”至“10”数字发音的嘴唇图像序列进行识别,获得了较高的识别率。因此,灰度轮廓权向量差分唇形特征是一种精炼、描述性强、适合于唇读识别的视觉语言特征,该算法能自动地完成模型的训练和视觉特征的提取,是一种有效的特征提取算法。     

A Domain-Specific Modeling approach for supporting the specification of Visual Instructional Design Languages and the building of dedicated editors

This paper presents, illustrates and discusses theories and practices about the application of a domain-specific modeling (DSM) approach to facilitate the specification of Visual Instructional Design Languages (VIDLs) and the development of dedicated graphical editors. Although this approach still requires software engineering skills, it tackles the need of building VIDLs allowing both visual models for human-interpretation purposes (explicit designs, communication, thinking, etc.) and machine-readable notations for deployment or other instructional design activities. This article proposes a theoretical application and a categorization, based on a domain-oriented separation of concerns of instructional design. It also presents some practical illustrations from experiments of specific DSM tooling. Key lessons learned as well as observed obstacles and challenges to deal with are discussed in order to further develop such an approach.     

Matlab与VisualC＋＋混合编程的实现

从不同角度分析和研究了VisualC 与工程计算语言Matlab的接口技术。为工程设计人员提供了比较全面的VisualC 与Matlab的混合编程方法,并给出了各种方法的优缺点。通过二者的结合,既有效地利用了Matlab强大的数值计算能力和众多的函数,大大减少程序设计的工作量,又继承了VisualC 良好的程序界面,证明是一种很好的程序设计方法。     

视觉问答与对话综述

视觉问答与对话是人工智能领域的重要研究任务,是计算机视觉与自然语言处理交叉领域的代表性问题之一.视觉问答与对话任务要求机器根据指定的视觉图像内容,对单轮或多轮的自然语言问题进行作答.视觉问答与对话对机器的感知能力、认知能力和推理能力均提出了较高的要求,在跨模态人机交互应用中具有实用前景.文中对近年来视觉问答与对话的研究...     

Towards Graph Transformation Based Generation of Visual Editors Using Eclipse

This work discusses the state-of-the-art of visual editor generation based on graph transformation concepts on one hand, and using the Eclipse technology which includes the Graphical Editor Framework (GEF), on the other hand. Due to existing shortcomings in both approaches, we present a combined approach for a tool environment that allows to generate a GEF-based editor from a formal, graph-transformation based visual language specification.     

书籍装帧设计的视觉语言再思考

书籍装帧设计是一种造型艺术,在创作上不同于一般绘画。它包含着更为广泛的内容与形式。设计手法与视觉语言是书籍装帧设计极为关键的元素。在设计中,我们必须结合书籍的内容特点和读者对象来进行构思,努力把握书籍内容的基调,了解作者的思想,灵活地运用视觉语言,使装帧艺术成为书籍的有机组成部分之一。     

Towards Incremental Parsing of Natural Language Using Recursive Neural Networks

In this paper we develop novel algorithmic ideas for building a natural language parser grounded upon the hypothesis of incrementality. Although widely accepted and experimentally supported under a cognitive perspective as a model of the human parser, the incrementality assumption has never been exploited for building automatic parsers of unconstrained real texts. The essentials of the hypothesis are that words are processed in a left-to-right fashion, and the syntactic structure is kept totally connected at each step.Our proposal relies on a machine learning technique for predicting the correctness of partial syntactic structures that are built during the parsing process. A recursive neural network architecture is employed for computing predictions after a training phase on examples drawn from a corpus of parsed sentences, the Penn Treebank. Our results indicate the viability of the approach and lay out the premises for a novel generation of algorithms for natural language processing which more closely model human parsing. These algorithms may prove very useful in the development of efficient parsers.     

自然语言是正则语言

自然语言由字母集、单词集、句子集、段落集和文章集5部分组成,而且,字母集包含于单词集,单词集包含于句子集,句子集包含于段落集,段落集包含于文章集。在此观点下,自然语言是正则语言。引入了字母空图和字母空图语言等10个概念。作为特例,英语由英语字母集、英语单词集、英语句子集、英语段落集和英语文章集5部分构成。在此观点下,英语是正则语言。引入了英语字母空图和英语字母空图语言等10个概念。汉语由汉字集、汉语词汇集、汉语句子集、汉语段落集和汉语文章集5部分构成。在此观点下,汉语是正则语言。引入了汉字空图和汉字空图语言等10个概念。这为计算机自然语言处理打开了一扇新的大门,开辟了语言学新的研究领域。     

可视化编程的实践与思考

对可视化编程中遇到的一些问题,如动态显示数据的擦除和重绘、不同数据源的切换、单例模式的使用、门面模式的使用进行了归纳总结。     

Visual secret sharing for multiple secrets

Conventional visual secret sharing schemes are designed for a single secret image so it is inefficient to generate numerous share images for multiple secret images simultaneously. Therefore, a novel visual secret sharing scheme for multiple secret images is proposed in this paper. In the proposed encryption process, a stacking relationship graph of secret pixels and share blocks is generated to indicate the encryption functions, and a set of visual patterns is defined to produce two share images according to this graph. Based on the stacking properties of these patterns, the secret images can be obtained from the two share images at aliquot stacking angles. The proposed scheme makes the number of secret images not restricted and further extends it to be general. As a result, the proposed scheme enhances visual secret sharing schemes’ ability for multiple secrets.     

Visual knowledge specification for conceptual design: Definition and tool support

Current CAD tools are not able to support the conceptual design phase, and none of them provides a consistency analysis for sketches produced by architects. This phase is fundamental and crucial for the whole design and construction process of a building. To give architects a better support, we developed a CAD tool for conceptual design and a knowledge specification tool. The knowledge is specific to one class of buildings and it can be reused. Based on a dynamic and domain-specific knowledge ontology, different types of design rules formalize this knowledge in a graph-based form. An expressive visual language provides a user-friendly, human readable representation. Finally, a consistency analysis tool enables conceptual designs to be checked against this formal conceptual knowledge.In this article, we concentrate on the knowledge specification part. For that, we introduce the concepts and usage of a novel visual language and describe its semantics. To demonstrate the usability of our approach, two graph-based visual tools for knowledge specification and conceptual design are explained.     

Towards an American Sign Language interface

In this paper, we present two major parts of an interface for American Sign Language (ASL) to computer applications currently under work; a hand tracker and an ASL-parser. The hand tracker extracts information about handshape, position and motion from image sequences. As an aid in this process, the signer wears a pair of gloves with colour-coded markers on joints and finger tips. We also present a computational model of American Sign Language. This model is realized in an ASL-parser which consists of a DCG-grammar and a non-lexical component that records non-manual and spatial information over an ASL-discourse.     

基于Visual Prolog的高校学生心理咨询专家系统

本文构建了一个基于VisualProlog7．0的高校学生心理咨询专家系统。本文首先将多种知识表示方式应用于学生心理问题的症状表示,综合采用推理策略,对咨询者的不同症状表现得出推理判断,提出相应的心理辅导方法,并提供简单明了的解释。     

Visual interaction using an iconic system

IBS (Icon Based System) is an experimental graphical query language based on icons. It demonstrates the capabilities of a workstation environment by integrating the aspects of database programming in one graphical setting. Namely, it allows direct manipulation of objects dealing with pictorial data as well as alphanumeric data. We point out the interaction techniques between users and database systems. Then we describe the design of IBS, illustrate its features, and show how queries are formulated in a medical context.