A domain-driven approach to improving search effectiveness in traditional online catalogs

Search performance can be greatly improved by using domain knowledge to assist users in developing a problem specification tailored to the information contained in the system. A methodology is presented for utilizing intelligent information retrieval techniques and domain-specific knowledge to improve user searching. For databases involving a relatively narrow domain, a “system thesaurus” combined with expert systems technology can be used to create an intelligent front end to assist the user in retrieving information with greater precision and recall. Evaluation of the prototype showed greatly improved search effectiveness and satisfaction over the traditional catalog system.     

TADEUS: seamless development of task-based and user-orientedinterfaces

Task-based and user-oriented user interfaces utilize knowledge about user tasks and user characteristics to the utmost extent. They support users throughout their work flows, and must be constructed by a development process that avoids loss of application context and involves user feedback, from requirements specification to code generation. The concepts behind the task analysis/design/end users systems (TADEUS) approach to enable seamless task-based development are a semantically rich representation scheme, a model-driven development procedure, a diagrammatic notation and unifying specification scheme. Thus, interactive applications can be developed seamlessly. Specifications comprise problem domain knowledge, work processes, user roles and personal profiles, as well as interaction modalities (required for task accomplishment). For user-interface prototyping the TADEUS environment contains a model interpreter that executes structure and behavior specifications. This way, early feedback on task-based portals can be provided by users. In this paper we detail the latest developments in the TADEUS project when implementing a work-process based usability life cycle. We review the underlying methodology and the features of the TADEUS environment, in order to demonstrate the benefits for developers and users resulting of smooth transition support for and between the different stages of development     

Semantic layers for illustrative volume rendering

Direct volume rendering techniques map volumetric attributes (e.g., density, gradient magnitude, etc.) to visual styles. Commonly this mapping is specified by a transfer function. The specification of transfer functions is a complex task and requires expert knowledge about the underlying rendering technique. In the case of multiple volumetric attributes and multiple visual styles the specification of the multi-dimensional transfer function becomes more challenging and non-intuitive. We present a novel methodology for the specification of a mapping from several volumetric attributes to multiple illustrative visual styles. We introduce semantic layers that allow a domain expert to specify the mapping in the natural language of the domain. A semantic layer defines the mapping of volumetric attributes to one visual style. Volumetric attributes and visual styles are represented as fuzzy sets. The mapping is specified by rules that are evaluated with fuzzy logic arithmetics. The user specifies the fuzzy sets and the rules without special knowledge about the underlying rendering technique. Semantic layers allow for a linguistic specification of the mapping from attributes to visual styles replacing the traditional transfer function specification.     

Conceptual modeling of causal map: Object oriented causal map

This study has been started from question “Is there a methodology that can make causal map comprised of causality as a database by using conceptual modeling method?” In this research, causal map is proposed to represent causal relation by using conceptual modeling method. Therefore, we formalize causality as a cognitive rule to allow us to control changes in the decision making environment. Such causality is embedded in the real world (application domain). And, user (decision maker) use to represent set of causality which decision-makers have retrieved from the set of knowledge or experiences in application domain. Such set of causality that decision-makers possess in their know–how and short (long) term memory are usually formalized by causal map. It is verified for users whether this causal map is helpful in solving their problems. By extending the basis of conceptual modeling theory (ontology theory, classification theory, decomposition theory, and semantic network theory), we introduce a concept of causal entity diagram and address why causal map is needed to analyze a specific domain knowledge for given decision problem solving. Finally, object oriented causal map (O2CM) were employed to verify usefulness of causal map for user (decision maker) in this study.     

A Domain Knowledge Driven Approach for User Interface Software Development

A domain knowledge driven user interface development approach is described.As a conceptual design of the user interface,the domain knowledge defines the user interface in terms of objects,actions and their relationships that the user would use to interact with the application system.It also serves as input to a user interface management system (UIMS) and is the kernel of the target user interface.The principal ideas and the implementation techniques of the approach is discussed.The user interface model,user interface designer oriented high-level specification notatiopn,and the transformation algorithms on domain knowledge are presented.     

Are Two Better Than One? A Comparison Between Single- and Dual-Monitor Work Stations in Productivity and User’s Windows Management Style

Multiple monitors are commonly used in the workplace nowadays. This study compares user productivity and windows management style (WMS) on single- and dual-monitor work stations for engineering tasks of three complexity levels. Four productivity measures including task time, cursor movement, the number of window switches, and the number of mouse clicks were compared. The results showed that dual-monitor setting resulted in significantly less window switches and mouse clicks. Most users preferred dual-monitor setting. To understand how users manage multiple windows in completing their tasks, a new WMS categorization method is proposed, toggler and resizer, and user behavior was categorized into one of these two styles. More users adopted “toggler” style, but as the task complexity level increased, some “toggler” style users switched to “resizer” style.     

SEWEBAR-CMS: semantic analytical report authoring for data mining results

SEWEBAR-CMS is a set of extensions for the Joomla! Content Management System (CMS) that extends it with functionality required to serve as a communication platform between the data analyst, domain expert and the report user. SEWEBAR-CMS integrates with existing data mining software through PMML. Background knowledge is entered via a web-based elicitation interface and is preserved in documents conforming to the proposed Background Knowledge Exchange Format (BKEF) specification. SEWEBAR-CMS offers web service integration with semantic knowledge bases, into which PMML and BKEF data are stored. Combining domain knowledge and mining model visualizations with results of queries against the knowledge base, the data analyst conveys the results of the mining through a semi-automatically generated textual analytical report to the end user. The paper demonstrates the use of SEWEBAR-CMS on a real-world task from the cardiological domain and presents a user study showing that the proposed report authoring support leads to a statistically significant decrease in the time needed to author the analytical report.     

The virtual playground: an educational virtual reality environment for evaluating interactivity and conceptual learning

The research presented in this paper aims at investigating user interaction in immersive virtual learning environments, focusing on the role and the effect of interactivity on conceptual learning. The goal has been to examine if the learning of young users improves through interacting in (i.e. exploring, reacting to, and acting upon) an immersive virtual environment (VE) compared to non-interactive or non-immersive environments. Empirical work was carried out with more than 55 primary school students between the ages of 8 and 12, in different between-group experiments: an exploratory study, a pilot study, and a large-scale experiment. The latter was conducted in a virtual environment designed to simulate a playground. In this “Virtual Playground,” each participant was asked to complete a set of tasks designed to address arithmetical “fractions” problems. Three different conditions, two experimental virtual reality (VR) conditions and a non-VR condition, that varied the levels of activity and interactivity, were designed to evaluate how children accomplish the various tasks. Pre-tests, post-tests, interviews, video, audio, and log files were collected for each participant, and analysed both quantitatively and qualitatively. This paper presents a selection of case studies extracted from the qualitative analysis, which illustrate the variety of approaches taken by children in the VEs in response to visual cues and system feedback. Results suggest that the fully interactive VE aided children in problem solving but did not provide a strong evidence of conceptual change as expected; rather, it was the passive VR environment, where activity was guided by a virtual robot, that seemed to support student reflection and recall, leading to indications of conceptual change.     

Participatory knowledge-management design: A semiotic approach

The aim of this paper is to present a design strategy for collaborative knowledge-management systems based on a semiotic approach. The contents and structure of experts' knowledge is highly dependent on professional or individual practice. Knowledge-management systems that support cooperation between experts from different (sub-)fields need to be situated and tailored to provide effective support even if the common aspects of the data need to be described by ontologies that are generic in respect to the sub-disciplines involved. To understand and approach this design problem, we apply a semiotic perspective to computer application and human–computer interaction. From a semiotic perspective, the computer application is both a message from the designer to the user about the structure of the problem domain, as well as about interaction with it, and a structured channel for the user's communication with herself, himself or other users of the software. Tailoring or “end-user development” – i.e. adapting the knowledge-management system to a specific (sub-)discipline, task or context – then refines both the message and adapts the structure of the interaction to the situated requirements.The essential idea of this paper is to define a new perspective for designing and developing interactive systems to support collaborative knowledge management. The key concept is to involve domain experts in participatory knowledge design for mapping and translating their professional models into the proper vocabularies, notations, and suitable visual structures for navigating among interface elements. To this end, the paper describes how our semiotic approach supports processes for representing, storing, accessing, and transferring knowledge through which the information architecture of an interactive system can be defined. Finally, the results of applying our approach to a real-world case in an archaeological context are presented.     

Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI

AI is remarkably successful and outperforms human experts in certain tasks, even in complex domains such as medicine. Humans on the other hand are experts at multi-modal thinking and can embed new inputs almost instantly into a conceptual knowledge space shaped by experience. In many fields the aim is to build systems capable of explaining themselves, engaging in interactive what-if questions. Such questions, called counterfactuals, are becoming important in the rising field of explainable AI (xAI). Our central hypothesis is that using conceptual knowledge as a guiding model of reality will help to train more explainable, more robust and less biased machine learning models, ideally able to learn from fewer data. One important aspect in the medical domain is that various modalities contribute to one single result. Our main question is “How can we construct a multi-modal feature representation space (spanning images, text, genomics data) using knowledge bases as an initial connector for the development of novel explanation interface techniques?”. In this paper we argue for using Graph Neural Networks as a method-of-choice, enabling information fusion for multi-modal causability (causability – not to confuse with causality – is the measurable extent to which an explanation to a human expert achieves a specified level of causal understanding). The aim of this paper is to motivate the international xAI community to further work into the fields of multi-modal embeddings and interactive explainability, to lay the foundations for effective future human–AI interfaces. We emphasize that Graph Neural Networks play a major role for multi-modal causability, since causal links between features can be defined directly using graph structures.     

Round-Trip Prototyping Based on Integrated Functional and User Interface Requirements Specifications

Requirements engineering in the new millennium is facing an increasing diversity of computerised devices comprising an increasing diversity of interaction styles for an increasing diversity of user groups. Thus the incorporation of user interface requirements into software requirements specifications becomes more and more mandatory. Validating these requirements specifications with hand-made, throw-away prototypes is not only expensive, but also bears the danger that validation results are not accurately fed back into the requirements specification. In this paper, we propose an enhancement of the requirements specification method SCORES for an explicit capturing of user interface requirements. The advantages of the approach are threefold. First, the user interface requirements specification is UML-compliant and integrated into the functional requirements specification. Second, prototypes for validation purposes can semi-automatically be generated. Third, the model-based generation of prototypes allows for ‘round-trip prototyping’ such that manual changes of the prototype during the validation process are automatically fed back into the requirements specification.     

Compositional algebra for interactive data access

An important issue for the success of a database application is the effectiveness of its interface. Frequently a relevant part of the programming effort is devoted to the generation of interfaces. The visual programming environments reduce only partly this effort, and in particular, things become more complicated when data coming from different sources (different views in the same database or even views from different databases or systems) are to be related and must cooperate in the data navigation and manipulation task. To overcome this problem we present a new database access paradigm based on an algebra on the domain of computational abstractions called “services” which include both dimensions: the data access computation and the user interaction. This means that the interaction is not implemented by using separated constructs as happens for traditional computational models; on the contrary, as the interaction is an integral part of the service paradigm, the user interaction is computed starting from the declarative specification of the data access itself. The combination of services in a service expression through the operators defined by the service algebra makes it possible to generate cooperating user interfaces for complex data navigation and manipulation. Through algebraic properties, which hold both from the data and user interface point of view, the service expressions can be simplified and optimized guaranteeing their initial semantics. The paper shows the application of the service algebra to the relational environment by means of a simple extension to SQL. Finally, the paper describes a tool based on a three tier architecture and on Java technology for developing and distributing services in Web environment. Services and combination of services expressed with the service algebra are automatically translated into Java objects, allowing the rapid development of platform independent data access services.     

Taskmaster: an interactive, graphical environment for task specification, execution and monitoring

Taskmaster is an interactive environment that employs a unique blend of graphic technologies and iconic images to support user task specification. In this environment, problem solving is based on the selection, specification, and composition of tools that correspond to natural sets of ordered operations. The Taskmaster environment is novel in that it

(provides an interactive, visual-based approach to user task specification;

(encourages and supports task specification and refinement processes from both the top-down and bottom-up perspectives; and

(enables one to specify parallel tasks in a natural and convenient manner

To “program” a given task within the Taskmaster environment, one decomposes it into an ordered set of conceptually simple, high-level operations, and then combines (composes) a corresponding network of software tools that implements these operations. Execution of the specified network provides a task solution. Major system components supporting user task specification include a network editor, a tools database and a network execution monitor.     

Conceptual and content-based annotation of (multimedia) documents

This paper focuses on the techniques used in an NKRL environment (NKRL = Narrative Knowledge Representation Language) to deal with a general problem affecting the so-called “semantic/conceptual annotations” techniques. These last, mainly ontology-based, aim at “annotating” multimedia documents by representing, in some way, the “inner meaning/deep content” of these documents. For documents of sufficient size, the content modeling operations are separately executed on ‘significant fragments’ of the documents, e.g., “sentences” for natural language texts or “segments” (minimal units for story advancement) in a video context. The general problem above concerns then the possibility of collecting all the partial conceptual representations into a global one. This integration operation must, moreover, be carried out in such a way that the meaning of the full document could go beyond the simple addition of the ‘meanings’ conveyed by the single fragments. In this context, NKRL makes use of second order knowledge representation structures, “completive construction” and “binding occurrences”, for collecting within the conceptual annotation of a whole “narrative” the basic building blocks corresponding to the representation of its composing elementary events. These solutions, of a quite general nature, are discussed in some depth in this paper. This last includes also a short “state of the art” in the annotation domain and some comparisons with the different methodologies proposed in the past for solving the above ‘integration’ problem.     

Hybrid gaming environments: keeping the human in the loop within the Internet of things

This paper discusses the issues and realization of hybrid gaming environments that allow human users to utilize ubiquitous computing technology in order to provide novel interaction experiences. The paper first discusses recent trends in human–computer interaction and the emerging application domain of hybrid gaming that is realized with tangible user interfaces and smart artifacts. Then, the field of hybrid games is discussed and analyzed in more detail through the thorough discussion of a conceptual model for hybrid games. Finally, the concrete realization of the hybrid game “In search for the Amulet” is presented as an example for the emerging field of hybrid games.     

Design and comparative evaluation of Smoothed Pointing: A velocity-oriented remote pointing enhancement technique

The increasing use of remote pointing devices in various application domains is fostering the adoption of pointing enhancement techniques which are aimed at counterbalancing the shortcomings of desk-free interaction. This paper describes the strengths and weaknesses of existing methods for ray pointing facilitation, and presents a refinement of Smoothed Pointing, an auto-calibrating velocity-oriented precision enhancing technique. Furthermore, the paper discusses the results of a user study aimed at empirically investigating how velocity-oriented approaches perform in target acquisition and in trajectory-based interaction tasks, considering both laser-style and image–plane pointing modalities. The experiments, carried out in a low precision scenario in which a Wiimote was used both as a wand and a tracking system, show that Smoothed Pointing allows a significant decrease in the error rate and achieves the highest values of throughput in trajectory-based tasks. The results also indicate that the effectiveness of precision enhancing techniques is significantly affected by the pointing modality and the type of pointing task.     

Debugging formal specifications: a practical approach using model-based diagnosis and counterstrategies

Creating a formal specification for a design is an error-prone process. At the same time, debugging incorrect specifications is difficult and time consuming. In this work, we propose a debugging method for formal specifications that does not require an implementation. We handle conflicts between a formal specification and the informal design intent using a simulation-based refinement loop, where we reduce the problem of debugging overconstrained specifications to that of debugging unrealizability. We show how model-based diagnosis can be applied to locate an error in an unrealizable specification. The diagnosis algorithm computes properties and signals that can be modified in such a way that the specification becomes realizable, thus pointing out potential error locations. In order to fix the specification, the user must understand the problem. We use counterstrategies to explain conflicts in the specification. Since counterstrategies may be large, we propose several ways to simplify them. First, we compute the counterstrategy not for the original specification but only for an unrealizable core. Second, we use a heuristic to search for a countertrace, i.e., a single input trace which necessarily leads to a specification violation. Finally, we present the countertrace or the counterstrategy as an interactive game against the user, and as a graph summarizing possible plays of this game. We introduce a user-friendly implementation of our debugging method and present experimental results for GR(1) specifications.