Automatic generation of visual programming environments

We have developed the visual language compiler-compiler (VLCC) system to automatically generate visual programming environments. VLCC is a grammar based system that can support implementation of any visual language by assisting the language designer in defining the language's graphical objects, syntax, and semantics. The final result of the generation process includes an integrated environment with a visual editor and a compiler for the defined visual language. In VLCC, graphical tools define visual languages to create both graphical objects and composition rules. Visual editors enable language designers to directly and visually manipulate the syntax of these languages. To capture the widest range of visual languages, the VLCC system can be configured for a specific language class. Different language classes can be characterized depending on their graphical objects' structure and on the way they can be composed. Also, box and arrow diagrams are defined for primitive objects with attaching points and for composition rules to join boxes and arrows at those attaching points. After choosing the visual language type to create, the designer can concentrate on language definition details. VLCC uses the positional grammar model as its underlying grammar formalism     

A general theory of spatial relations to support a graphical tool for visual information extraction

In this paper we present a general spatial composition framework which allows one to model the graphical objects and the spatial relations of a large class of visual languages. The new formalism has been implemented within the SRQ tool, a software system for the Visual Information Extraction, enabling it to work on a wider range of domains. In particular, in the paper we describe the application of SRQ to geospatial data.     

可视化语言技术在软件开发中的应用

可视化语言技术比一维文本语言在描述软件组成方面具有优越性.由于图表和图形概念在系统建模中的广泛使用,可视化语言可以应用于需求分析、设计、测试和维护等软件开发的各个阶段.除了具有直观易见的特点之外,图文法在计算机上的精确建模和验证能力,为设计可视化语言提供了一个坚实的理论基础.讨论了可视化语言的形式理论基础,回顾了相关的可视化图形编程环境.特别提出了一种空间图文法,并且用该图文法定义了统一建模语言的行为语义.基于空间图文法,开发了一种基于模式驱动的框架,以帮助软件架构与设计.     

Visual programming with analogical representations: Inspirations from a semiotic analysis of comics

Analogical representations based on pictures of domain objects can be used in visual programming to provide a close mapping between the program and the resulting runtime display, which can make programming easier for children and other users. The use of graphical rewrite rules with before and after pictures is an example of this approach. Graphical rewrite rules have some similarities with comics strips, which also use picture sequences of graphical objects to describe dynamics in a static form. However, the visual language of comics is not used to its full potential in visual programming. We discuss how a semiotic analysis of comics can be used to address some of the limitations of graphical rewrite rules. We use a visual programming system we have designed to illustrate that comic strips can express more general computations and be more intuitive and flexible than traditional graphical rewrites. Our conclusion is that the visual language of comics has a strong potential for increasing the expressiveness and flexibility of visual programming with analogical representations of domain objects, while maintaining a direct mapping between the program representation and the runtime representation.     

Visual sensitivities of dynamic graphical displays

Advanced display design, such as Ecological Interface Design (EID), makes extensive use of complex graphical objects. Research has shown that by following EID methodologies, supervisory operators have better performance with the EID displays than with non-EID displays. However, the effect of particular dynamic graphical objects seen in EID displays has never been studied. In this study, we examined how different visual features of graphical objects affect the performance of the objects. We used a modified dynamic just noticeable difference task to investigate the graphical objects that show changes most sensitively. We also investigated the sensitivities of graphical objects in determining target levels, directional changes, and proportions. A commercial EID design company generated the graphical objects examined. We had subjects perform four tasks with graphical objects that varied in their visual features but were still representative of objects currently being used in ecological design. It was found that for simple dynamic objects such as bars and polygon objects, a line changing in angle was the most noticeable emergent feature to show a departure from “normal” state. For complex graphical objects, those target-indicator displays that mimic a “bull's eye” when at the target value should be used for displays that show observers when a target value has been reached. Abrupt changes in shape should be used in trend meters to show when variables or processes have changed direction. Finally, “solid objects” that make use of vertical lines and shading should be used for comparison meters that compare two values and keep them in a particular ratio. The findings provide guidance for designers of dynamic advanced graphical displays by encouraging the consideration of visual aspects of graphical objects, as well as prescribing graphical objects that should be used in the types of tasks investigated.     

A graphical model framework for decoding in the visual ERP-based BCI speller

We present a graphical model framework for decoding in the visual ERP-based speller system. The proposed framework allows researchers to build generative models from which the decoding rules are obtained in a straightforward manner. We suggest two models for generating brain signals conditioned on the stimulus events. Both models incorporate letter frequency information but assume different dependencies between brain signals and stimulus events. For both models, we derive decoding rules and perform a discriminative training. We show on real visual speller data how decoding performance improves by incorporating letter frequency information and using a more realistic graphical model for the dependencies between the brain signals and the stimulus events. Furthermore, we discuss how the standard approach to decoding can be seen as a special case of the graphical model framework. The letter also gives more insight into the discriminative approach for decoding in the visual speller system.     

A Human-Cognitive Perspective of Users’ Password Choices in Recognition-Based Graphical Authentication

ABSTRACT

Graphical password composition is an important part of graphical user authentication which affects the strength of the chosen password. Considering that graphical authentication is associated with visual search, perception, and information retrieval, in this paper we report on an eye-tracking study (N = 109) that aimed to investigate the effects of users’ cognitive styles toward the strength of the created passwords and shed light into whether and how the visual strategy of the users during graphical password composition is associated with the passwords’ strength. For doing so, we adopted Witkin’s Field Dependence-Independence theory, which underpins individual differences in visual information and cognitive processing, as graphical password composition tasks are associated with visual search. The analysis revealed that users with different cognitive processing characteristics followed different patterns of visual behavior during password composition which affected the strength of the created passwords. The findings underpin the need of considering human-cognitive characteristics as a design factor in graphical password schemes. The paper concludes by discussing implications for improving recognition-based graphical passwords through adaptation and personalization techniques based on individual cognitive characteristics.     

Graphics by a Logic Database Management System

The GEDBLOG system allows applications which manipulate graphic objects to be developed following a declarative definitional style. GEDBLOG supports the consistent design and prototyping of graphic applications through an incremental development and makes it possible to guarantee automatically that the application meets its specifications. Typical GEDBLOG applications have graphics as their characterizing element and can be found, for example, in the CAD/CAM, visual languages or graphical interface areas. The system is obtained by integrating a graphical data language in an existing logic database management system, EDBLOG, so that graphic and non-graphic information is handled in a uniform declarative way.     

Defining the abstract syntax of visual languages with advanced graph grammars—A case study based on behavior trees

Diagrammatic visual languages can increase the ability of engineers to model and understand complex systems. However, to effectively use visual models, the syntax and semantics of these languages should be defined precisely. Since most diagrammatic visual models that are currently used to specify systems can be described as (directed) typed graphs, graph grammars have been identified as a suitable formalism to describe the abstract syntax of visual modeling languages. In this article, we investigate how advanced graph-transformation techniques, such as conditional, structure-generic and type-generic graph-transformation rules, can help to improve and simplify the specification of the abstract syntax of a visual modeling language. To demonstrate the practicability of an approach that unifies these advanced graph-transformation techniques, we define the abstract syntax of behavior trees (BTs), a graphical specification language for functional requirements. Additionally, we provide a translational semantics of BTs by formalizing a translation scheme to the input language of the SAL model checking tool for each of the graph-transformation rules.     

Color design for illustrative visualization

Professional designers and artists are quite cognizant of the rules that guide the design of effective color palettes, from both aesthetic and attention-guiding points of view. In the field of visualization, however, the use of systematic rules embracing these aspects has received less attention. The situation is further complicated by the fact that visualization often uses semi-transparencies to reveal occluded objects, in which case the resulting color mixing effects add additional constraints to the choice of the color palette. Color design forms a crucial part in visual aesthetics. Thus, the consideration of these issues can be of great value in the emerging field of illustrative visualization. We describe a knowledge-based system that captures established color design rules into a comprehensive interactive framework, aimed to aid users in the selection of colors for scene objects and incorporating individual preferences, importance functions, and overall scene composition. Our framework also offers new knowledge and solutions for the mixing, ordering and choice of colors in the rendering of semi-transparent layers and surfaces. All design rules are evaluated via user studies, for which we extend the method of conjoint analysis to task-based testing scenarios. Our framework's use of principles rooted in color design with application for the illustration of features in pre-classified data distinguishes it from existing systems which target the exploration of continuous-range density data via perceptual color maps.     

Generating tools from graph-based specifications

This paper describes an approach for generating graphical, structure-oriented software engineering tools from graph-based specifications. The approach is based on the formal meta modeling of visual languages using graph rewriting systems. Besides the syntactical and semantical rules of the language, these meta models include knowledge from the application domains. This enables the resulting tools to provide the user with high level operations for editing, analysis and execution of models. Tools are constructed by generating source code from the meta model of the visual language, which is written in the very high level programming language PROGRES. The source code is integrated into a framework which is responsible for the invocation of commands and the visualization of graphs. As a case study, a visual language for modeling development processes together with its formal meta model is introduced. The paper shows how a process management tool based on this meta model is generated and reports on our experiences with this approach.     

Parsing nonlinear languages

The diagrammatic approach to user interfaces for computer-aided software development toolkits, visual query systems, and visual programming environments, is based on the use of diagrams and charts traditionally drawn on paper. In particular, the VLG system (Visual Language Generator) has been proposed to generate icon-oriented visual languages customized for given applications. The syntactical model underlying the interpretation of a visual language in VLG has been designed to describe icon-oriented visual languages. In order to enable the VLG system to apply to any kind of graphical languages, like diagrammatic ones, it is necessary to find a more general syntactical model able to support both their generation and interpretation. This paper addresses the comprehension of the features that a grammatical formalism for nonlinear languages must have to match any requirement for an efficient parsing. To this aim, relation grammars support an easy implementation of a general parsing algorithm for multidimensional languages, parametric with respect to the rewriting rules of the grammar. We compare the expressive power of relation grammars to grammatical formalisms for graph grammars     

Design, construction, and application of a generic visual languagegeneration environment

The implementation of visual programming languages (VPLs) and their supporting environments is time-consuming and tedious. To ease the task, researchers have developed some high-level tools to reduce the development effort. None of these tools, however, can be easily used to create a complete visual language in a seamless way as the lex/yacc tools do for textual language constructions. This paper presents the design, construction and application of a generic visual language generation environment, called VisPro. The VisPro design model improves the conventional model-view-controller framework in that its functional modules are decoupled to allow independent development and integration. The VisPro environment consists of a set of visual programming tools. Using VisPro, the process of VPL construction can be divided into two steps: lexicon definition and grammar specification. The former step defines visual objects and a visual editor, and the latter step provides language grammars with graph rewriting rules. The compiler for the VPL is automatically created according to the grammar specification. A target VPL is generated as a programming environment which contains the compiler and the visual editor. The paper demonstrates how VisPro is used by building a simple visual language and a more complex visual modeling language for distributed programming     

基于fisheye views算法解决可视化程序设计语言问题的研究

可视化程序设计语言已经引起了越来越多学者的研究兴趣,并产生了一系列研究成果。在对编程元素由二维图形对象构成的传统可视化程序设计语言中产生的两个显示问题进行了详细分析,提出引入一种改进的fisheye views算法来解决这些问题的方法。并通过设计的原型系统VPMF表征了该方法的效果和可行性。     

What Approach for Searching Spatial Information?

This paper presents a survey of different approaches for searching information from geographic information systems (GISs) and spatial databases. The existing dichotomy between querying and navigation is highlighted and we emphasize the need to overcome it. Five categories of query languages can be identified: (1) natural languages, (2) artificial languages (e.g. extensions of SQL), (3) tabular languages: use of skeletons or forms, (4) graphical languages: use of symbols, which are only graphical conventions (like in the entity-relationship diagrams) and (5) visual languages: use of visual metaphors (e.g. icons, blackboard metaphor and map-overlay metaphor). The principles of each of them are presented. Respective strengths and weaknesses are pointed out, based on a predefined query. The last part of this paper is devoted to the hypermedia approach and describes the navigation in a hyperbase.     

Reliable parallel software construction using PARSE

The PARSE design methodology provides a hierarchical, object-based approach to the development of parallel software systems. A system design is initally structured into a collection of concurrently executing objects which communicate via message-passing. A graphical notation known as the process graph is then used to capture the structural and important dynamic properties of the system. Process graph designs can then be semi-mechanically transformed into complete Petri nets to give a detailed, executable and potentially verifiable design specification. From a complete design, translation rules for target programming languages are defined to enable the implementation to proceed in a systematic manner. The paper describes the steps in PARSE methodology and process graph notation, and illustrates the application of PARSE from design specification to program code using a network protocol example.     

Which animation effects improve indirect manipulation?

This paper describes an experiment to explore the effectiveness of animation in improving indirect manipulation operations. Indirect manipulation operations are those initiated by command menus and buttons to perform a transformation on a graphical object or set of graphical objects. The particular improvement is an operation's ability to show both what would happen if the operation were committed and what would happen if it were cancelled while an operation is being considered. The experiment required subjects to watch a simple alignment operation for a set of graphical objects. They were then asked to record the original placement of those graphical objects. Each task used one of four visual cues: modified telltale, wiggle, colour, or no visual cue. We found the modified telltale, wiggle, and colour visual effects significantly more effective than no visual feedback for cuing the user as to original position of the graphical objects. The modified telltale and colour effects were significantly more effective than the wiggle effect. The major conclusion drawn from this experiment is the use of visual feedback improves a user's ability to remember the previous position of graphical objects after an alignment operation.     

Towards language-to-language transformation

This paper proposes a simplicity-oriented approach and framework for language-to-language transformation of, in particular, graphical languages. Key to simplicity is the decomposition of the transformation specification into sub-rule systems that separately specify purpose-specific aspects. We illustrate this approach by employing a variation of Plotkin’s Structural Operational Semantics (SOS) for pattern-based transformations of typed graphs in order to address the aspect ‘computation’ in a graph rewriting fashion. Key to our approach are two generalizations of Plotkin’s structural rules: the use of graph patterns as the matching concept in the rules, and the introduction of node and edge types. Types do not only allow one to easily distinguish between different kinds of dependencies, like control, data, and priority, but may also be used to define a hierarchical layering structure. The resulting Type-based Structural Operational Semantics (TSOS) supports a well-structured and intuitive specification and realization of semantically involved language-to-language transformations adequate for the generation of purpose-specific views or input formats for certain tools, like, e.g., model checkers. A comparison with the general-purpose transformation frameworks ATL and Groove, illustrates along the educational setting of our graphical WebStory language that TSOS provides quite a flexible format for the definition of a family of purpose-specific transformation languages that are easy to use and come with clear guarantees.