Vu: visual user-interface design

Vu is an interactive visual programming environment, which enables interface designers to create highly interactive, graphical, direct-manipulation-style graphical user interfaces in a very natural manner. The general strategy of vu is to enable the designer to draw the screen display that the end user will see, and to test the interface by performing actions just as the end user would. This is achieved by providing exploratory programming in a highly visual and object-oriented environment. This paper describes the design principles and structure of vu.     

开放式Java可视化用户界面开发系统的设计和实现

描述了一个“升放式Ｊａｖａ可视化用户界面开发工具”,重点描述了该系统的通用接口模型及其动态扩充模型的设计思想、组成及实现方法。     

Implementing a user interface management system for existing applications using an object-oriented database tool

Numerous engineering application systems have been developed over the past twenty years, and many of these applications will continue to be used for many years to come. Examples of such applications include CAD Systems, finite-element analysis packages and inspection systems. Because many of these applications were developed before graphical workstations became available, they often have simple command-line user interfaces. Thus, there is a need for a graphical user interface management system (UIMS) that can be used to build point-and-click style interfaces for these existing engineering applications. In this paper we describe such a UIMS, and discuss its implementation using an object-oriented database tool. This UIMS allows users to create and modify user interfaces by editing graphical representations of the interfaces, thus eliminating the need to write code to build or modify an interface. The UIMS is implemented using an object-oriented database tool to take advantage of the data manipulation and storage management capabilities it provides. This approach reduces both the quantity and complexity of the code needed to implement the UIMS. It also allowed the UIMS to be implemented in a minimal amount of time.     

Supporting user participation design using a fuzzy analytic hierarchy process approach

There are three fundamental problems that may occur in the process of user participation design: first, the participants/users may not be able to express their requirements clearly; second, they have little knowledge about design; and third, they are generally unfamiliar with the software that designers use. Based on this understanding, a method that considers design rationale is proposed in this work to support the process of user participation design. In addressing the user participation process, a fuzzy analytic hierarchy process (AHP) approach is applied to grasp people's ideas, in the initial design phase. A case study on creating house layout design is employed to illustrate the proposed approach. In this regard, to help participants/users create layout designs, it is proposed that a 3D generative system is used, which integrates navigational concepts, direct manipulation, and the design rationale theory. In a nutshell, this research proposes a system to implement a design rational model and improve design communication in the user participation process. To demonstrate the effectiveness of the proposed prototype system, a user test is performed and we put forward some findings and research questions for further research and industry practices.     

Simplifying interactive design of solid models: A hypertext approach

Conceptual design and editing parameterized models of 3D solids often requires user interactions with many types of entities and relations. In this paper, we present a concept for simplifying the design process. The simplification is based on three paradigms; integration of data structures and control structures, multiple view interface, and use of hypertext techniques for fast traversal and selective display of data. Data structures and control structures are integrated into uniform building blocks called cells. Cells define assemblies of CSG solids and can be grouped bottom-up into more complex cells or, alternatively, can be refined top-down during the design process. Efficient specification of complex cells and patterns is made possible through the use of automatic default values, multiple instantiation, built-in repetition, and recursion. A multiple-view interface enables the designer to view and edit both the cell parameters and the hierarchical structure of the model. The interface provides integrated textual, 2D graphical, and 3D direct-manipulation techniques for specifying primitive dimensions and relative transformations.     

Visual query interfaces for semantic datasets: An evaluation study

The rapid growth of the Linked Open Data cloud, as well as the increasing ability to lift relational enterprise datasets to a semantic, ontology-based level means that vast amounts of information are now available in a representation that closely matches the conceptualizations of the potential users of this information. This makes it interesting to create ontology based, user-oriented tools for searching and exploring this data. Although initial efforts were intended for tech users with knowledge of SPARQL/RDF, there are ongoing proposals designed for lay users. One of the most promising approaches is to use visual query interfaces, but more user studies are needed to assess their effectiveness. In this paper, we compare the effect on usability of two important paradigms for ontology-based query interfaces: form-based and graph-based interfaces. In order to reduce the number of variables affecting the comparison, we performed a user study with two state-of-the-art query tools developed by ourselves, sharing a large part of the code base: the graph-based tool OptiqueVQS*, and the form-based tool PepeSearch. We evaluated these tools in a formal comparison study with 15 participants searching a Linked Open Data version of the Norwegian Company Registry. Participants had to respond to 6 non-trivial search tasks using alternately OptiqueVQS* and PepeSearch. Even without previous training, retrieval performance and user confidence were very high, thus suggesting that both interface designs are effective for searching RDF datasets. Expert searchers had a clear preference for the graph-based interface, and mainstream searchers obtained better performance and confidence with the form-based interface. While a number of participants spontaneously praised the capability of the graph interface for composing complex queries, our results evidence that graph interfaces are difficult to grasp. In contrast, form interfaces are more learnable and relieve problems with disorientation for mainstream users. We have also observed positive results introducing faceted search and dynamic term suggestion in semantic search interfaces.     

Query by diagram: A visual environment for querying databases

In recent years, several attempts have been made to define query languages characterized by both high expressive power and easy query formulation. Several issues concern graphical applications, based on the diagrammatic representation of a semantic model and visual interaction. This paper describes the architecture and the implementation of a graphical query system, based on the diagrammatic representation of entity relationship schemata. The query language underlying the system allows the formulation of recursive queries; moreover, user interaction in both managing diagrams and expressing queries is simplified by the presence of a fully visual environment and a rich set of interaction strategies.     

Towards useful and usable interaction design tools: CanonSketch

Despite all the effort dedicated to bringing better User-Centered Design (UCD) tools to market, current studies show that the industry is still dominated by tools that do not support the activities and workstyles of designers. Also, there is a growing need for interaction design tools aimed at software engineers, a problem related to bringing usability into the software engineering processes.

We propose a new workstyle model that can be effectively used to envision, design and evaluate a new generation of innovative interaction and software design tools, aimed at integrating usability and software engineering.

We illustrate the effectiveness of our model by describing a new tool, called CanonSketch, that was built in order to support UCD in terms of the dimensions in our workstyle model. We also describe an evaluation study aimed at contrasting paper prototyping with our tool as well as the level of workstyle support.     

Augmenting interaction and cognition using agent architectures and technology inspired by psychology and social worlds

Intelligent agents can play a pivotal role in providing both software systems and augmented interfaces, to individual users from all walks of life, to utilise the Internet 24 h a day, 7 days a week (24×7), including interaction with other users, over both wireless and broadband infrastructures. However, traditional approaches to user modelling are not adequate for this purpose, as they mainly account for a generic, approximate, idealised user. New user models are therefore required to be adaptable for each individual and flexible enough to represent the diversity of all users using information technology. Such models should be able to cover all aspects of an individual’s life—those aspects of most interest to the individual user themselves. This paper describes a novel intelligent agent architecture and methodology both called ShadowBoard, based on a complex user model drawn from analytical psychology. An equally novel software tool, called the DigitalFriend based on ShadowBoard, is also introduced. This paper illustrates how aspects of user cognition can be outsourced, using, for example, an internationalised book price quoting agent. The Locales Framework from Computer Supported Co-operative Work is then used to understand the problematic aspects of interaction involved in complex social spaces, identifying specific needs for technology intervention in such social spaces, and to understand how interactions amongst mobile users with different abilities might be technically assisted in such spaces. In this context, the single user-centred multi-agent technology demonstrated in the DigitalFriend is adapted to a multi-user system dubbed ShadowPlaces. The aim of ShadowPlaces is to outsource some of the interaction necessary, for a group of mobile individuals with different abilities to interact cooperatively and effectively in a social world, supported by wireless networks and backed by broadband Internet services. An overview of the user model, the architecture and methodology (ShadowBoard) and the resulting software tool (the DigitalFriend) is presented, and progress on ShadowPlaces—the multi-user version—is outlined.

DOGeye: Controlling your home with eye interaction

Nowadays home automation, with its increased availability, reliability and with its ever reducing costs is gaining momentum and is starting to become a viable solution for enabling people with disabilities to autonomously interact with their homes and to better communicate with other people. However, especially for people with severe mobility impairments, there is still a lack of tools and interfaces for effective control and interaction with home automation systems, and general-purpose solutions are seldom applicable due to the complexity, asynchronicity, time dependent behavior, and safety concerns typical of the home environment. This paper focuses on user-environment interfaces based on the eye tracking technology, which often is the only viable interaction modality for users as such. We propose an eye-based interface tackling the specific requirements of smart environments, already outlined in a public Recommendation issued by the COGAIN European Network of Excellence. The proposed interface has been implemented as a software prototype based on the ETU universal driver, thus being potentially able to run on a variety of eye trackers, and it is compatible with a wide set of smart home technologies, handled by the Domotic OSGi Gateway. A first interface evaluation, with user testing sessions, has been carried and results show that the interface is quite effective and usable without discomfort by people with almost regular eye movement control.     

Evaluation of a surgical interface for robotic cryoablation task using an eye-tracking system

Computer-assisted navigation systems coupled with surgical interfaces (SIs) are providing doctors with tools that are safer for patients compared to traditional methods. Usability analysis of the SIs that guides their development is hence important. In this study, we record the eye movements of doctors and other people with no medical expertise during interaction with an SI that directs a simulated cryoablation task. There are two different arrangements for the layout of the same SI, and the goal is to evaluate whether one of these arrangements is ergonomically better than the other. We use several gaze related statistics some of which are employed in an SI design context for the first time. Even though the performance and gaze related analysis reveals that the two arrangements are comparable in many respects, there are also differences. Specifically, one arrangement leads to more saccades along the vertical and horizontal directions, lower saccade amplitudes in the crucial phase of the task, more locally clustered and yet globally spread viewing. Accordingly, that arrangement is selected for future use. The present study provides a proof of concept for the integration of novel gaze analysis tools developed for scene perception studies into the interface development process.     

Visual coding by optimal graph-coloring

A formal study of visual codings in user interface design is presented. Visual codings for maximum distinction of different objects in displayed images are formulated as a discrete optimization problem of maximum-distance graph-coloring. The formulation is a generalization of the classical coloring problem in graph theory. Having pointed out that maximum-distance graph-coloring is NP-complete, we develop new, fast approximation algorithms for optimal visual codings. The proposed algorthms run inO (M N) time, whereM is the number of visual codes used andN is the number of objects to be encoded. Besides being efficient, the algorithms are simple and easy to implement. Our experiments showed that geographic maps automatically colored by the new algorithms were preferred to those colored by the previous graph-theoretical approach and they are competitive, if not better, in terms of the visual distinction of different regions than those drawn by hand.Supported by grans from Natural Science and Enginneering Research Council of Canada     

A language for simple interactive retrieval from a database system

Query interaction with a database system requires, in general, some understanding of the content and structure of the database, and knowledge of a suitable query language to encode the request for data. These factors impose barriers against access to a database on a casual or irregular basis. To overcome such restrictions we have investigated the use of a pseudo-intelligent front-end retrieval system. This system was designed to be independent of any specific database management system, although a relational database structure was considered to be the most appropriate. A prototype version of the system has been set up to run on top of Logica's relational database management system RAPPORT and the IBM relational database system SQL. As a result of this experience we have developed an adaptable language to facilitate intelligent interaction between an end user and a database management system.     

Visual language integration: A survey and open challenges

With the recent development of deep learning technology comes the wide use of artificial intelligence (AI) models in various domains. AI shows good performance for definite-purpose tasks, such as image recognition and text classification. The recognition performance for every single task has become more accurate than feature engineering, enabling more work that could not be done before. In addition, with the development of generation technology (e.g., GPT-3), AI models are showing stable performances in each recognition and generation task. However, not many studies have focused on how to integrate these models efficiently to achieve comprehensive human interaction. Each model grows in size with improved performance, thereby consequently requiring more computing power and more complicated designs to train than before. This requirement increases the complexity of each model and requires more paired data, making model integration difficult. This study provides a survey on visual language integration with a hierarchical approach for reviewing the recent trends that have already been performed on AI models among research communities as the interaction component. We also compare herein the strengths of existing AI models and integration approaches and the limitations they face. Furthermore, we discuss the current related issues and which research is needed for visual language integration. More specifically, we identify four aspects of visual language integration models: multimodal learning, multi-task learning, end-to-end learning, and embodiment for embodied visual language interaction. Finally, we discuss some current open issues and challenges and conclude our survey by giving possible future directions.     

A bootstrap approach to visual user-interface design and development

An approach to the design of effective computer-based systems is discussed. This approach exploits the user's traditional diagrammatic notations in an effort to achieve usability for experts other than computer professionals. Notations are formalized as visual languages, thus allowing the design of visual editors, interpreters, and compilers. The users themselves exploit these tools to define a hierarchy of environments by a bootstrapping approach. By navigating within these environments, they can progressively design visual interfaces and computing tools that allow them not only to execute the required computational tasks, but also to gain insight into and control the computational process, and check the results.     

Visual servoing for an autonomous rendezvous and capture system

Many autonomous space systems are under development to economically extend the lives of space systems, either by orbit modification or replenishment of consumables. This paper explains the theory behind and demonstrates experimentally the terminal control portion of an autonomous rendezvous and capture system. The system will grapple the launch bolt holes of common satellite interfaces in order to minimize the necessary client hardware. This is accomplished using a stereoscopic vision system using an edge detector and Hough transform to locate the target point. This project demonstrated the use of off-the-shelf hardware and open source algorithms to accomplish its objective.