Providing RIA user interfaces with accessibility properties

Rich Internet Applications (RIAs) technologies are challenging the way in which the Web is being developed. However, from the UI accessibility point of view, these technologies pose new challenges that the Web Accessibility Initiative of the W3C is trying to solve through the use of a standard specification for Accessible Rich Internet Applications (WAI-ARIA). Currently, the introduction of properties defined in WAI-ARIA is being done in an ad-hoc manner due to the lack of models, methodologies and tools to support the design of accessible RIA UIs. In this paper we propose a semantic approach to deal with this modeling issue by extending the RUX-Method, a model-based method to build RIA UIs. The approach includes the validation process of the accessibility issues at two different levels: the UI structure and the interactions behavior.     

JAVA新类库的UI机制的分析

为了给程序员提供更多的方便，ＳＵＮ公司对ＡＷＴ作了重大改进，发布了ＪＦＣ，在其中使用了全新的事件机制和ＵＩ机制。本文对ＪＦＣ中所使用的ＵＩ机制作了初步分析，包括对ＵＩ使用的类的介绍，ＪＦＣ向第三方程序员提供的编程接口等内容，具有较好的参考价值。     

Evolution towards smart home environments: empirical evaluation of three user interfaces

Smart home environments have evolved to the point where everyday objects and devices at home can be networked to give the inhabitants new means to control them. Familiar information appliances can be used as user interfaces (UIs) to home functions to achieve a more convenient user experience. This paper reports an ethnographic study of smart home usability and living experience. The purpose of the research was to evaluate three UIs—a PC, a media terminal, and a mobile phone—for smart home environments. The results show two main types of activity patterns, pattern control and instant control, which require different UI solutions. The results suggest that a PC can act as a central unit to control functions for activity patterns that can be planned and determined in advance. The mobile phone, on the other hand, is well suited for instant control. The mobile phone turned out to be the primary and most frequently used UI during the 6-month trial period in the smart apartment.     

Design patterns for user interface for mobile applications

The topic of this paper is a collection of user interface (UI) design patterns for mobile applications. In the paper we present the structure of the patterns collection – the patterns are suggested solutions to problems that are grouped into a set of problem areas that are further grouped into three main problem areas – a structure which is valuable both as an index to identifying patterns to use, and it gives a fairly comprehensive overview of issues when designing user interfaces for mobile applications. To show the breadth of the patterns collection we present six individual problems with connected design patterns in some detail – each coming from different problem areas. They represent important and relevant problems, and are on different levels of abstraction, thus showing how patterns may be used to present problems and solutions on different levels of detail. To show the relevance and usefulness of the patterns collection for usability professionals with a mixed background, we present some relevant findings from a validation of the patterns collection. In addition to verifying the relevance and usefulness of the patterns collection, it also shows both expected and surprising correlations between background and perceived relevance and usefulness. One important finding from the validation is an indication that the patterns collection is best suited for experienced UI developers wanting to start developing mobile UIs. Using a patterns collection for documenting design knowledge and experience has been a mixed experience, so we discuss pros and cons of this. Finally, we present related work and future research.     

Framework and authoring tool for an extension of the UIML language

This paper presents a framework for the design of User Interfaces (UIs). By applying model transformations, the framework allows different UIs to be generated for different computing platforms. The tool presented in this work helps designers to build an abstract user interface which is later transformed into a concrete user interface by means of transformation techniques based on graph grammars. These techniques can be used to generate implementation code for several UI platforms including desktop applications, dynamic websites and mobile applications. The generated user interfaces are integrated with a multi-tier application by referencing external services and communicating with the application core over Web Service protocols. Our tool also allows the concrete interfaces to be enhanced before generating the final UI. The approach uses an adaptation of UIML (User Interface Markup Language). The adaptation focuses on defining a data model and a services model, and it also introduces a navigation model that allows data communication from one UI to another. The obtained UIs together with Web Services can conform complete applications instead of just being prototypes.     

基于日志挖掘的移动应用用户访问模型建模技术研究

提出了一种基于监控日志挖掘的移动应用用户访问模型自动构造方法,该方法包括监控代码注入和界面访问模型构造两部分。首先,提出了一种监控代码自动注入方法,即通过对移动应用代码的静态分析,自动地在相应位置插入监控代码以支持在运行时动态地监控用户的访问行为。其次,提出了一种基于状态机的移动应用用户访问模型构造方法。访问模型中状态机的节点和节点间跳转上的附加属性描述了UI界面之间的跳转行为和界面内控件的使用情况。对移动应用进行的实验表明,这种基于监控日志挖掘的移动应用用户访问模型自动构造方法能够成功地自动注入移动应用的监控代码,并能够有效获得移动应用用户界面访问行为。     

Distributed orchestration of user interfaces

Workflow management systems focus on the coordination of people and work items, service composition approaches on the coordination of service invocations, and, recently, web mashups have started focusing on the integration and coordination of pieces of user interfaces (UIs), e.g., a Google map, inside simple web pages. While these three approaches have evolved in a rather isolated fashion – although they can be seen as evolution of the componentization and coordination idea from people to services to UIs – in this paper we describe a component-based development paradigm that conciliates the core strengths of these three approaches inside a single model and language. We call this new paradigm distributed UI orchestration, so as to reflect the mashup-like and process-based nature of our target applications. In order to aid developers in implementing UI orchestrations, we equip the described model and language with suitable design, deployment, and runtime instruments, covering the whole life cycle of distributed UI orchestrations.     

一种复杂产品界面可用性定量评估方法的研究和应用

可用性评估是保证界面质量的重要手段。利用粗糙集灰色分析解决复杂产品界面可用性评估中的模糊问题,建立界面可用性评估模型;基于任务将难以直接评估的复杂界面分解成多个子界面,建立新的复杂界面可用性评价指标的测量方法和实验范式,从而实现对复杂界面可用性的客观评价。经实例验证,本方法可实现对多个复杂界面方案进行可用性评估和优选,最后通过脑电实验验证了其有效性。     

Model-driven engineering: A survey supported by the unified conceptual model

During the last decade a new trend of approaches has emerged, which considers models not just documentation artefacts, but also central artefacts in the software engineering field, allowing the creation or automatic execution of software systems starting from those models. These proposals have been classified generically as Model-Driven Engineering (MDE) and share common concepts and terms that need to be abstracted, discussed and understood. This paper presents a survey on MDE based on a unified conceptual model that clearly identifies and relates these essential concepts, namely the concepts of system, model, metamodel, modeling language, transformations, software platform, and software product. In addition, this paper discusses the terminologies relating MDE, MDD, MDA and others. This survey is based on earlier work, however, contrary to those, it intends to give a simple, broader and integrated view of the essential concepts and respective terminology commonly involved in the MDE, answering to key questions such as: What is a model? What is the relation between a model and a metamodel? What are the key facets of a modeling language? How can I use models in the context of a software development process? What are the relations between models and source code artefacts and software platforms? and What are the relations between MDE, MDD, MDA and other MD approaches?     

An extension of UML for the modeling of WIMP user interfaces

The Unified Modeling Language (UML) [OMG, Unified Modeling Language Specification, Version 2.0, Technical Report, Object Management Group http://www.omg.org/technology/documents/formal/uml.htm, 2005] provides system architects working on analysis and design (A&D) with one consistent language for specifying, visualizing, constructing, and documenting the artifacts of software systems, as well as for the business modeling. The user interface (UI), as a significant part of most applications, should be modeled using UML, and automatic CASE tools may help to generate UIs from UML designs. In this paper, we describe how to use and specialize UML diagrams in order to describe the UIs of a software system based on WIMP (Windows, Icons, Menus and Pointers). Use case diagrams are used for extracting the main UIs. Use cases are described by means of user-interaction diagrams, a special kind of activity diagrams in which states represent data output actions and transitions represent data input events. Input and output interactions in the user-interaction diagrams help the designer to extract the UI components used in each UI. We obtain a new and specialized version of the use case diagram for the UI modeling (called UI diagram) and a class diagram for UI components—called UI-class diagram. The user-interaction, UI and UI-class diagrams, can be seen as the UML-based UI models of the system. Finally, UI prototypes can be generated from UI-class diagrams with CASE tool support. As case study of our technique, we will describe an Internet book shopping system.     

Multi-modal approach to evaluate adaptive visual stimuli of remote operation system using gaze behavior

A multi-modal approach is proposed to evaluate the usability of Adaptive Visual Stimuli for User Interface (AVS4UI) of remote operation systems. This study focuses on the evaluation of AVS4UI for forklift work because the operation complexity includes driving and cargo handling, which typically requires multiple salient attention. Presenting this amount of information simultaneously on a User Interface (UI) tends to cause confusions to operators and reduces operation efficiency. AVS4UI can therefore be one of the promising solutions where the optimal visual stimuli are autonomously presented for different work conditions. However, evaluation of AVS4UI is challenging because operators may be disoriented by adaptive information and worked without safety considerations. Therefore, novel gaze metrics are proposed to evaluate responses of forklift operators to AVS4UI so that undesired behavior can be evaluated. The proposed metrics implicitly represent gaze pattern in terms of transition and distribution between UI elements, operation safety, and familiarity with the adaptive system. The ideal AVS4UI is expected to minimize the proposed gaze metrics and outperform the non-adaptive UI. More importantly, the results of these metrics are consistent with those of perceived workload defined by NASA-Task Load Index. We also propose a correlation model using stepwise linear regression that provides reasonable estimation of perceived workload. Such novel metrics and correlation model enable objective and online evaluation to minimize biases of subjective response. Therefore, online work support system can be developed to support workers.     

Constraint programming for type inference in flexible model-driven engineering

Domain experts typically have detailed knowledge of the concepts that are used in their domain; however they often lack the technical skills needed to translate that knowledge into model-driven engineering (MDE) idioms and technologies. Flexible or bottom-up modelling has been introduced to assist with the involvement of domain experts by promoting the use of simple drawing tools. In traditional MDE the engineering process starts with the definition of a metamodel which is used for the instantiation of models. In bottom-up MDE example models are defined at the beginning, letting the domain experts and language engineers focus on expressing the concepts rather than spending time on technical details of the metamodelling infrastructure. The metamodel is then created manually or inferred automatically. The flexibility that bottom-up MDE offers comes with the cost of having nodes in the example models left untyped. As a result, concepts that might be important for the definition of the domain will be ignored while the example models cannot be adequately re-used in future iterations of the language definition process. In this paper, we propose a novel approach that assists in the inference of the types of untyped model elements using Constraint Programming. We evaluate the proposed approach in a number of example models to identify the performance of the prediction mechanism and the benefits it offers. The reduction in the effort needed to complete the missing types reaches up to 91.45% compared to the scenario where the language engineers had to identify and complete the types without guidance.     

A security-aware metamodel for multi-agent systems (MAS)

This paper adopts a model-based security (MBS) approach to identify security requirements during the early stages of multi-agent system development. Our adopted MBS approach is underpinned by a metamodel independent of any specific methodology. It allows for security considerations to be embedded within any situated agent methodology which then prescribes security considerations within its work products. Using a standard model-driven engineering (MDE) approach, these work products are initially constructed as high abstraction models and then transformed into more precise models until code-specific models can be produced. A multi-agent system case study is used to illustrate the applicability of the proposed security-aware metamodel.     

Querying through a user interface

In contrast to a traditional setting where users express queries against the database schema, we assert that the semantics of data can often be understood by viewing the data in the context of the user interface (UI) of the software tool used to enter the data. That is, we believe that users will understand the data in a database by seeing the labels, drop-down menus, tool tips, or other help text that are built into the user interface. Our goal is to allow domain experts with little technical skill to understand and query data. In this paper, we present our GUi As View (Guava) framework and describe how we use forms-based UIs to generate a conceptual model that represents the information in the user interface. We then describe how we generate a query interface from the conceptual model. We characterize the resulting query language using a subset of the relational algebra. Since most application developers want to craft a physical database to meet desired performance needs, we present here a transformation channel that can be configured by instantiating one or more of our transformation operators. The channel, once configured, automatically transforms queries from our query interface into queries that address the underlying physical database and delivers query results that conform to our query interface. In this paper, we define and formalize our database transformation operators. The contributions of this paper are that first, we demonstrate the feasibility of creating a query interface based directly on the user interface and second, we introduce a general purpose database transformation channel that will likely shorten the application development process and increase the quality of the software by automatically generating software artifacts that are often made manually and are prone to errors.     

Reusing UI elements with Model-Based User Interface Development

This paper introduces the potential for reusing UI elements in the context of Model-Based UI Development (MBUID) and provides guidance for future MBUID systems with enhanced reutilization capabilities. Our study is based upon the development of six inter-related projects with a specific MBUID environment which supports standard techniques for reuse such as parametrization and sub-specification, inclusion or shared repositories.We analyze our experience and discuss the benefits and limitations of each technique supported by our MBUID environment. The system architecture, the structure and composition of UI elements and the models specification languages have a decisive impact on reusability. In our case, more than 40% of the elements defined in the UI specifications were reused, resulting in a reduction of 55% of the specification size. Inclusion, parametrization and sub-specification have facilitated modularity and internal reuse of UI specifications at development time, whereas the reuse of UI elements between applications has greatly benefited from sharing repositories of UI elements at run time.     

Influential Characteristics of Enterprise Information System User Interfaces

ABSTRACT

End-user acceptance is considered as a significant factor influencing the success of enterprise information system (EIS) implementations and operations. This study conceptualizes three aspects of EIS user interfaces (UIs), namely information overload, control familiarity, and UI fit, and proposes a model to understand their effect on two major factors that are considered to influence the end-user acceptance of these systems: EIS end user’s performance expectancy and effort expectancy. We developed a theoretical model and multiitem scales for the proposed EIS UI characteristics and tested the model empirically with data from a survey performed with a sample 98 EIS end users. The results from our test provide evidence for the key role that EIS UI design plays in the end user’s performance and effort expectancy.     

Development of user interface for tele-operated cranes

This research focuses on one of the major challenges in a tele-operated crane system, namely the user interface (UI). This UI should provide rich information retrieved from the field and display it properly in order to enhance the operation and decision-making processes involved in crane activities. In this research, we have designed two UIs specifically for a tele-operated crane system. The first UI is a four view system (quad-view) with a top view, left-side view, right-side view, and global view. The second UI has four views but uses additional guidance from Augmented Reality (AR) technologies. To test the UIs, we used a robot arm (KUKA KR16) to simulate a tele-operated crane in a testing environment. We also compared the UIs we designed against a conventional operation interface (i.e. operator’s view with oral guidance from the ground). We conducted a user test with two groups of participants: 5 crane operators and 30 students. Students constitute a novice group, and their results are interpreted from a statistical perspective. Using the student group, the interface’s learning curve can be evaluated. Operators constitute an expert group, which provides evidences for evaluating if the developed UIs are realistic and fit the needs of the field. We found that use of the UIs we designed resulted in a shorter erection time (336 and 343 s) than if the participants used the conventional operation interface (380 s). A self-evaluated index showing the difficulty of the tasks, the NASA task loading index (TLX), was calculated for each of the UIs. The UIs resulted in a higher TLX (52.0 and 53.2) than the conventional operation interface (32.2). In summary, the two UIs developed in this research are able to assist operators in operating remote cranes more efficiently and with less mental load than by using the conventional operation interface.     

Contributions of Dichotomic View of plasticity to seamlessly embed accessibility and adaptivity support in user interfaces

Recent legal changes have increased the need for developing accessible user interfaces in computer-based systems. In this sense, previously existing user interfaces are intended to be modified and new user interfaces are intended to be designed taking accessibility guidelines into account. Typically, model-based approaches have been used when developing accessible user interfaces or redefining existing ones. But the use of static models leads to the development of not dynamically adaptable user interfaces. Dynamic adaptation in accessible user interfaces is important due to the fact that interaction difficulties on people with disabilities may change through use. In this paper, we present some contributions that can be obtained from the application of the Dichotomic View of plasticity in the personalization of user interfaces. With the double perspective defined in this approach, it is intended to go further from a mere adaptation to certain user stereotypes, offering also a dynamic support to real limitations or difficulties users can encounter during the use of the UI. This goal is achieved analyzing user logs by an inference engine that dynamically infers modifications in the user interface to adjust it to varying user needs. A case study is presented in order to show how the guidelines and software support defined in the Dichotomic View of plasticity can be applied to develop a component for a particular system aimed at performing dynamic user interface adaptations with accessibility purposes. This approach includes some innovations that make it different from conventional adaptable mechanisms applied to accessibility in some important aspects.     

Model theory approach to transaction processing system development

This paper presents a new approach, model theory approach, to small and medium scale transaction processing system (TPS) development. A TPS of this paper is an information system designed to process day-to-day business event data at operational level of an organization. The paper is not concerned with data base construction but with transaction processing.

The model theory approach is not a software engineering approach but a systems theory approach. In the approach a model of the target system, which is called a user model, is constructed in set theory using a formal system structure of a TPS. The user model is, then, compiled into an extended Prolog (extProlog) model. The extProlog is an extension of Prolog to meet requirements for management information system development. On compilation a standardized user interface (UI) called internal UI is attached. The extProlog model with the internal UI is, then, executed under control of another standardized UI called an external UI. Implementation is an integral part of the approach. Because the UIs are designed for the formalized (abstract) structure of a TPS, they can be standardized and are provided as black box components to system development. Because a systems developer is required to only build a user model in set theory based on a model theoretic structure in the approach, it is called a model theory approach. Advantages of this approach are that it provides a theoretical structure to information systems development so that systems development can be made an engineering discipline, and facilitates rapid systems development.     

A formalisation of the copy-modify-merge approach to version control in MDE

Models are the primary artefacts of the software development process in Model-Driven Engineering (MDE). Like other software artefacts, models undergo a complex evolution during their life cycles. Version control is one of the key techniques which enable developers to tackle this complexity. Traditional version control systems (VCS) are based on the copy-modify-merge approach which is not fully exploited in MDE since current implementations lack model-orientation. In this paper we provide a formalisation of the copy-modify-merge approach in the context of MDE. In particular, we analyse how the identification of commonalities and the calculation of differences can be defined by means of category-theoretical constructions. Moreover, we demonstrate how the properties of these constructions can be used to synchronise models and detect conflicting modifications.