Building syntax-aware editors for visual languages

Syntax-aware editors are a class of editors prompting users into writing syntactically correct programs by exploiting visual language syntax. They are particularly useful in those application domains where the way a visual symbol spatially relates to others depends from the context. This does not mean constraining users to enter only correct syntactic states in a visual sentence, rather it means detecting both syntax and potential semantic errors as early as possible, and providing error feedbacks in a non-intrusive way during editing. As a consequence, error handling strategies are an essential part of this editing style.In this work, we present a strategy for the automatic generation of syntax-aware visual language editors integrating incremental subsentence parsers into freehand editors. The proposed parsing strategy has turned out to be useful in many application domains involving spatial information systems, thanks to the possibility of interactively prompting feasible visual sentence extensions, and to the presence of a non-correcting error recovery strategy. A first experimental prototype implementing the whole approach has been embedded into the VLDesk system, and empirical studies have been performed in order to verify the performance and the effectiveness of the proposed approach.     

Incremental parsing without a parser

This article describes an algorithm for incremental parsing of expressions in the context of syntax-directed editors for programming languages. Since a syntax-directed editor represents programs as trees and statements and expressions as nodes in trees, making minor modifications in an expression can be difficult. Consider, for example, changing a “ + ” operator to a “1” operator or adding a short subexpression at a syntactically but not structurally correct position, such as inserting “) 1 (d“ at the # mark in” (a + b # + c)”. To make these changes in a typical syntax-directed editor, the user must understand the tree structure and type a number of tree-oriented construction and manipulation commands. This article describes an algorithm that allows the user to think in terms of the syntax of the expression as it is displayed on the screen (in infix notation) rather than in terms of its internal representation (which is effectively prefix), while maintaining the benefits of syntax-directed editing. This algorithm is significantly different from other incremental parsing algorithms in that it does not involve modifications to a traditional parsing algorithm or the overhead of maintaining a parser stack or any data structure other than the syntax tree. Instead, the algorithm applies tree transformations, in real-time as each token is inserted or deleted, to maintain a correct syntax tree.     

Incremental parsing in language‐based editors: user needs and how to meet them

Incremental parsing has long been recognized as a technique of great utility in the construction of language‐based editors, and correspondingly, the area currently enjoys a mature theory. Unfortunately, many practical considerations have been largely overlooked in previously published algorithms. Many user requirements for an editing system necessarily impact on the design of its incremental parser, but most approaches focus only on one: response time. This paper details an incremental parser based on LR parsing techniques and designed for use in a modeless syntax recognition editor. The nature of this editor places significant demands on the structure and quality of the document representation it uses, and hence, on the parser. The strategy presented here is novel in that both the parser and the representation it constructs are tolerant of the inevitable and frequent syntax errors that arise during editing. This is achieved by a method that differs from conventional error repair techniques, and that is more appropriate for use in an interactive context. Furthermore, the parser aims to minimize disturbance to this representation, not only to ensure other system components can operate incrementally, but also to avoid unfortunate consequences for certain user‐oriented services. The algorithm is augmented with a limited form of predictive tree‐building, and a technique is presented for the determination of valid symbols for menu‐based insertion. Copyright © 2001 John Wiley & Sons, Ltd.     

Visual Language Parsing in GenGEd

GenGEd supports the visual specification of visual languages and the generation of syntax-directed editors. However, syntax-directed editing is not always desired by the user. Therefore we extended GenGEd by parsing facilities which allow for free editing as well.     

A design rationale for a language-based editor

This paper reviews the design issues that arise in the construction of effective language-based editors for the preparation of syntactically and static semantically correct language sentences, typically computer programs. The need for such editors to support a pluralistic view of program structure is identified, together with the need to observe the constraints on performance and storage consumption if such editors are to be accepted by professional programmers. From these basic needs, more specific requirements for the display, parsing and semantic checking components of such an editor are derived.     

RECOGNITION CAN BE HARDER THAN PARSING

The work presented here attempts to bring out some fundamental concepts that underlie some known parsing algorithms, usually called chart or dynamic programming parsers, in the hope of guiding the design of similar algorithms for other formalisms that could be considered for describing the "surface" syntax of languages. The key idea is that chart parsing is essentially equivalent to a simple construction of the intersection of the language (represented by its grammar) with a regular set containing only the input sentence to be parsed (represented by a finite state machine). The resulting grammar for that intersection is precisely what is usually called a shared forest: it represents all parses of a syntactically ambiguous sentence. Since most techniques for processing ill-formed input can be modeled by considering a nonsingleton regular set of input sentences, we can expect to generalize these ill-formed input processing techniques to all parsers describable with our approach.     

Incremental LL(1) parsing in language-based editors

This paper introduces an efficient incremental LL(1) parsing algorithm for use in language-based editors that use the structure recognition approach. It is designed to parse user input at intervals of very small granularity and to limit the amount of incremental parsing needed when changes are made internal to the editing buffer. The algorithm uses the editing focus as a guide in restricting parsing. It has been implemented in the Fred language-based editor     

Scalable framework for parsing: from Fortress to JavaScript

Programming languages grow over time that requires frequent changes in language manipulations such as compilation, interpretation, and analysis. Because the very first step of most language manipulations is parsing, whether parsing can adapt to changes easily, quickly, and correctly, it affects the scalability of language manipulations. Even though various parsing techniques have been well studied theoretically, their practical experiences in scalable frameworks have not been available. In this paper, we present our experiences with parsing in scalable frameworks. We first describe our trials and errors using various parsing techniques in developing parsers for the Fortress programming language. Because Fortress was a scientific language under development, its mathematical and growable syntax introduced new challenges in parsing. We summarize the lessons learned from parsing Fortress, and we share our experience of applying the lessons to parsing the JavaScript programming language. While JavaScript is one of the most widely used languages, JavaScript itself and its diverse variants keep extending its syntax, and the extremely dynamic features of JavaScript also add new challenges in parsing. Using automatic generation tools and methods like staged parsing and automatic extraction and testing of examples in language specifications, our methodology for scalable parsing has shown to be very effective in practice. Copyright © 2015 John Wiley & Sons, Ltd.     

Automating visual language generation

A system to generate and interpret customized visual languages in given application areas is presented. The generation is highly automated. The user presents a set of sample visual sentences to the generator. The generator uses inference grammar techniques to produce a grammar that generalizes the initial set of sample sentences, and exploits general semantic information about the application area to determine the meaning of the visual sentences in the inferred language. The interpreter is modeled on an attribute grammar. A knowledge base, constructed during the generation of the system, is then consulted to construct the meaning of the visual sentence. The architecture of the system and its use in the application environment of visual text editing (inspired by the Heidelberg icon set) enhanced with file management features are reported     

可视化查询语言CQL的实现

为使得许多非计算机专业的用户，尤其是未接受过数据库编程语言训练的用户，能够独立操纵一个数据库系统，有效地解决工作、生活中的问题，我们研究并提供了一种空间数据库可视化查询语言CQL。借助该语言，用户可以直观、方便地操纵数据库系统或查询数据库信息。本文介绍了CQL语言的语素、语义、语法及语用的定义，同时给出了CQL编辑器、编译器和查询结果可视化的设计方法。     

句法分析中基于词汇化树邻接语法的数据增强方法

句法分析是自然语言处理的基础技术,主流的由数据驱动的神经网络句法分析模型需要大规模的标注数据,但是通过人工标注扩展树库成本很高,因此如何利用现有标注树库进行数据增强成为研究焦点。在汉语句法分析的数据增强任务中,对于给定的标注树库,要求数据增强所生成的句子满足如下条件：第一,要求生成句具有多样化且完整的句法树结构;第二,要求生成句具有合理的语义。对此,我们首次提出基于词汇化树邻接语法的数据增强方法。针对第一个需求,该文设计实现基于词汇化树邻接语法的词汇化树抽取算法与句法树合成算法,基于该语法可以在句法树之间进行“接插”和“替换”的操作,从而推导生成新的句法树,并且用语言学的知识保证生成句符合语法规则且具有完整的句法树结构。针对第二个需求,该文利用语言模型对生成句进行语义合理性评估,选取语义合理的句子作为最终的增强数据,从而获取高质量的标注树库。我们以汉语为例开展研究,在汉语树库CTB5上进行句法分析的数据增强评测实验。实验结果显示,在小样本(CTB5的20%)实验中,通过该方法得到的增强数据使依存句法分析和成分句法分析的精度分别提高1.39%和2.14%。在鲁棒性实验中,该文通过构建扩展...     

面向Dockerfile的容器镜像构建工具

在容器虚拟化中,主流的镜像构建方式是通过Dockerfile来构建的.然而,在使用Dockerfile构建镜像时存在着明显的不足:由于Dockerfile语言的复杂性,使用文本编辑方式没有提供有效的语法引导,没有对Dockerfile可能存在的错误进行有效的检测,导致构建容器镜像工作效率低下.此外,使用说明不完整的第三方镜像,无法有效的确定镜像的功能和使用方法,安全性也是第三方镜像所面临的一大挑战,这带来了容器镜像的重复利用率低下的问题.针对上述问题,在分析Dockerfile语法和统计分析Dockerfile常见错误以及深入研究Docker镜像存储机制的基础上,设计了一个面向Dockerfile的镜像构建工具,并使用了可视化编辑,错误检测,逆向分析等关键技术进行实现.该工具能够在镜像构建中提供有效的语法引导,对Dockerfile常见的错误进行有效检测,为了验证第三方镜像的功能和安全性,设计了一种由Docker镜像逆向生成Dockerfile的方式,用户可以通过Dockerfile完全了解第三方镜像的功能和使用方式,另外通过二次构建的方式也可一定程度上解决第三方镜像的安全性问题.     

Automatic generation of visual programming environments

We have developed the visual language compiler-compiler (VLCC) system to automatically generate visual programming environments. VLCC is a grammar based system that can support implementation of any visual language by assisting the language designer in defining the language's graphical objects, syntax, and semantics. The final result of the generation process includes an integrated environment with a visual editor and a compiler for the defined visual language. In VLCC, graphical tools define visual languages to create both graphical objects and composition rules. Visual editors enable language designers to directly and visually manipulate the syntax of these languages. To capture the widest range of visual languages, the VLCC system can be configured for a specific language class. Different language classes can be characterized depending on their graphical objects' structure and on the way they can be composed. Also, box and arrow diagrams are defined for primitive objects with attaching points and for composition rules to join boxes and arrows at those attaching points. After choosing the visual language type to create, the designer can concentrate on language definition details. VLCC uses the positional grammar model as its underlying grammar formalism     

Semantic interpretation of a database query language

In this paper, we will discuss a system that semantically interprets a formal database accessing language and generates natural language from this interpretation. In the past, the major way of communication between a user and a database was by means of a formal language. One such language is the SQL query language. Even though constructed as a user friendly language, SQL exemplifies the same difficulties for users as do other formal languages, namely a fairly rigid syntax, the necessity of variable binding, the lack of pronouns, and in the case of erroneous queries error messages that do not provide much insight. To alleviate some of the formal language problems, yet utilize the power of the formal language, we set out to build a natural language ‘umbrella’ for the SQL user. Our goal was not to build a natural language query system, but rather to use semantic knowledge and natural language for paraphrasing the formal language (SQL) and producing error messages as a feedback mechanism. In this way we build a genuine help facility, which would not only aid the user in dealing with SQL, but also trap erroneous queries.     

An on-the-fly grammar modification mechanism for composing and defining extensible languages

Adaptable Parsing Expression Grammar (APEG) is a formal method for defining the syntax of programming languages. It provides an on-the-fly mechanism to perform modifications of the syntax of the language during parsing time. The primary goal of this dynamic mechanism is the formal specification and the automatic parser generation for extensible languages. In this paper, we show how APEG can be used for the definition of the extensible languages SugarJ and Fortress, clarifying many aspects of the syntax of these languages. We also show that the mechanism for on-the-fly modification of syntax rules can be useful for defining grammars in a modular way, implementing almost all types of language composition in the context of specification of extensible languages.     

A survey on Urdu and Urdu like language stemmers and stemming techniques

Stemming is one of the basic steps in natural language processing applications such as information retrieval, parts of speech tagging, syntactic parsing and machine translation, etc. It is a morphological process that intends to convert the inflected forms of a word into its root form. Urdu is a morphologically rich language, emerged from different languages, that includes prefix, suffix, infix, co-suffix and circumfixes in inflected and multi-gram words that need to be edited in order to convert them into their stems. This editing (insertion, deletion and substitution) makes the stemming process difficult due to language morphological richness and inclusion of words of foreign languages like Persian and Arabic. In this paper, we present a comprehensive review of different algorithms and techniques of stemming Urdu text and also considering the syntax, morphological similarity and other common features and stemming approaches used in Urdu like languages, i.e. Arabic and Persian analyzed, extract main features, merits and shortcomings of the used stemming approaches. In this paper, we also discuss stemming errors, basic difference between stemming and lemmatization and coin a metric for classification of stemming algorithms. In the final phase, we have presented the future work directions.     

Influence of visual technology on the evolution of languageenvironments

The increasing influence of visual technology on language environments that has resulted from the increasing availability of workstations is discussed. An evolution that began with the relatively straightforward translation of textual techniques into corresponding visual techniques and has progressed to uses of visual techniques that have no natural parallel using purely textural techniques is traced. Visual user interfaces, visual editing, and visual languages are surveyed     

Defining the abstract syntax of visual languages with advanced graph grammars—A case study based on behavior trees

Diagrammatic visual languages can increase the ability of engineers to model and understand complex systems. However, to effectively use visual models, the syntax and semantics of these languages should be defined precisely. Since most diagrammatic visual models that are currently used to specify systems can be described as (directed) typed graphs, graph grammars have been identified as a suitable formalism to describe the abstract syntax of visual modeling languages. In this article, we investigate how advanced graph-transformation techniques, such as conditional, structure-generic and type-generic graph-transformation rules, can help to improve and simplify the specification of the abstract syntax of a visual modeling language. To demonstrate the practicability of an approach that unifies these advanced graph-transformation techniques, we define the abstract syntax of behavior trees (BTs), a graphical specification language for functional requirements. Additionally, we provide a translational semantics of BTs by formalizing a translation scheme to the input language of the SAL model checking tool for each of the graph-transformation rules.