Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

As a result of the importance of the usability approach in system development and the EC's 'Directive concerning the minimum safety and health requirements for VDT workers' (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, 'How can usability requirements comprehensively be considered and evaluated in system development?' Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?' Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

Abstract

As a result of the importance of the usability approach in system development and the EC's ‘Directive concerning the minimum safety and health requirements for VDT workers’ (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, ‘How can usability requirements comprehensively be considered and evaluated in system development?’ Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?’ Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

Empirical validation of an automatic usability evaluation method

Today, the success of a software application strongly depends on the usability of its interface, so the evaluation of interfaces has become a crucial aspect of software engineering. It is recognized that automatic tools for graphical user interface evaluation may greatly reduce the costs of traditional activities performed during expert evaluation or user testing in order to estimate the success probability of an application. However, automatic methods need to be empirically validated in order to prove their effectiveness with respect to the attributes they are supposed to evaluate.In this work, we empirically validate a usability evaluation method conceived to assess consistency aspects of a GUI with no need to analyze the back-end. We demonstrate the validity of the approach by means of a comparative experimental study, where four web sites and a stand-alone interactive application are analyzed and the results compared to those of a human-based usability evaluation. The analysis of the results and the statistical correlation between the tool׳s rating and humans׳ average ratings show that the proposed methodology can indeed be a useful complement to standard techniques of usability evaluation.     

基于软件体系结构的用户界面设计方法

为实现软件体系结构指导下的软件工程设计，该文提出了一个在体系结构指导下、支持自动生成的用户界面工程开发方法。该方法把界面的体系结构作为工程模型的直接描述对象，在界面体系结构的引导和约束下，通过可视化用户干预建立界面的展示模型，在代码生成阶段采用了界面体系结构指导下的界面设计模式。研究表明，以体系结构为指导的软件模型化设计是可行的。     

自动工具辅助用户行为捕捉与分析方法

为研究APP内的用户行为,优化用户体验,提出基于自动工具的用户行为捕捉与分析方法。基于一种用户行为捕捉工具CAUX,设计具有情境感知能力的数据采集模块,自动捕捉指定APP内的用户行为数据,无需修改APP源代码,对用户干扰性低。基于图像对比算法,建立数据分析平台,自动识别并输出用户行为序列。对用户行为序列进行聚类分析,总结用户行为模式并与预期任务序列进行比较,可以发现可用性问题。以有道词典为案例,验证了该方法的可行性。     

多通道融合算法和软件平台的实现

针对多通道用户界面的核心问题－多通道融合,分析了多通道输入流的特点,针对多通道任务的特点,把多通道输入区分为动作输入和参数输入。提出了任务注册的概念,并由此实现了以多通道任务管理器为核心的融合平台,最后提出一种结合多通道任务管理器的通用融合算法。     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

Modeling input modality choice in mobile graphical and speech interfaces

In this paper, we review three experiments with a mobile application that integrates graphical input with a touch-screen and a speech interface and develop a model for input modality choice in multimodal interaction. The model aims to enable simulation of multimodal human–computer interaction for automatic usability evaluation. The experimental results indicate that modality efficiency and input performance are important moderators of modality choice. Accordingly, we establish a utility-driven model that provides probability estimations of modality usage, based on the parameters of modality efficiency and input performance. Four variants of the model that differ in training data are fitted by means of Sequential Least Squares Programming. The analysis reveals a considerable fit regarding averaged modality usage. When applied to individual modality usage profiles, the accuracy decreases significantly. In an application example it is shown how the modality choice mechanism can be deployed for simulating interaction in the field of automatic usability evaluation. Results and possible limitations are discussed.     

Exploring the benefits of the combination of a software architecture analysis and a usability evaluation of a mobile application

Designing easy to use mobile applications is a difficult task. In order to optimize the development of a usable mobile application, it is necessary to consider the mobile usage context for the design and the evaluation of the user-system interaction of a mobile application. In our research we designed a method that aligns the inspection method “Software ArchitecTure analysis of Usability Requirements realizatioN” SATURN and a mobile usability evaluation in the form of a user test. We propose to use mobile context factors and thus requirements as a common basis for both inspection and user test. After conducting both analysis and user test, the results described as usability problems are mapped and discussed. The mobile context factors identified define and describe the usage context of a mobile application. We exemplify and apply our approach in a case study. This allows us to show how our method can be used to identify more usability problems than with each method separately. Additionally, we could confirm the validity and identified the severity of usability problems found by both methods. Our work presents how a combination of both methods allows to address usability issues in a more holistic way. We argue that the increased quantity and quality of results can lead to a reduction of the number of iterations required in early stages of an iterative software development process.     

Measuring Cognitive Load Using Linguistic Features: Implications for Usability Evaluation and Adaptive Interaction Design

An intelligent adaptable system, aware of a user’s experienced cognitive load, may help improve performance in complex, time-critical situations by dynamically deploying more appropriate output strategies to reduce cognitive load. However, measuring a user’s cognitive load robustly, in real-time is not a trivial task. Many research studies have attempted to assess users’ cognitive load using different measurements, but these are often unsuitable for deployment in real-life applications due to high intrusiveness. Relatively novel linguistic behavioral features as potential indices of user’s cognitive load is proposed. These features may be collected implicitly and nonintrusively supporting real-time assessment of users’ cognitive load and accordingly allowing adaptive usability evaluation and interaction. Results from a laboratory experiment show significantly different linguistic patterns under different task complexities and cognitive load levels. Implications of the research for adaptive interaction are also discussed, that is, how the cognitive load measurement-based approach could be used for user interface evaluation and interaction design improvement.     

Usability engineering laboratories: limitations and challenges toward a unifying tools/practices environment

This article discusses the limitations and challenges surrounding the present usability engineering (UE) labs. Usability laboratories include hardware and software tools to observe users, collect and analyse diverse data about the users' interactions, behaviours, actions, and reactions including their raw feedback regarding their experiences. Using statistical analysis and data mining software, these qualitative and quantitative data are transformed into design insights and recommendations for future usability improvements. First, we survey the existing stationary, portable, and remote laboratories. We then review the current usability tools while highlighting the gap between the existing tools/labs and the UE practices. We will show how this gap can be closed via a roadmap using a computer-assisted usability engineering environment (CAUTE). A CAUTE provides a unifying user interface that exploits a process-sensitive architecture for integrating the large variety of the existing tools into the best UE practices. Beyond the technical problems, there is also a need to address research issues including determining the interest of the CAUTE approach in comparison with the current usability labs.     

Is talking to an automated teller machine natural and fun?

Usability and affective issues of using automatic speech recognition technology to interact with an automated teller machine (ATM) are investigated in two experiments. The first uncovered dialogue patterns of ATM users for the purpose of designing the user interface for a simulated speech ATM system. Applying the Wizard-of-Oz methodology, multiple mapping and word spotting techniques, the speech driven ATM accommodates bilingual users of Bahasa Melayu and English. The second experiment evaluates the usability of a hybrid speech ATM, comparing it with a simulated manual ATM. The aim is to investigate how natural and fun can talking to a speech ATM be for these first-time users. Subjects performed the withdrawal and balance enquiry tasks. The ANOVA was performed on the usability and affective data. The results showed significant differences between systems in the ability to complete the tasks as well as in transaction errors. Performance was measured on the time taken by subjects to complete the task and the number of speech recognition errors that occurred. On the basis of user emotions, it can be said that the hybrid speech system enabled pleasurable interaction. Despite the limitations of speech recognition technology, users are set to talk to the ATM when it becomes available for public use.     

Reconciling usability and interactive system architecture using patterns

Traditional interactive system architectures such as MVC [Goldberg, A., 1984. Smaltalk-80: The Interactive Programming Environment, Addison-Wesley Publ.] and PAC [Coutaz, J., 1987. PAC, an implementation model for dialog design. In: Interact’87, Sttutgart, September 1987, pp. 431-436; Coutaz, J., 1990. Architecture models for interactive software: faillures and trends. In: Cockton, G. (Ed.), Engineering for Human-Computer Interaction, Elsevier Science Publ., pp. 137-153.] decompose the system into subsystems that are relatively independent, thereby allowing the design work to be partitioned between the user interfaces and underlying functionalities. Such architectures extend the independence assumption to usability, approaching the design of the user interface as a subsystem that can designed and tested independently from the underlying functionality. This Cartesian dichotomy can be fallacious, as functionalities buried in the application’s logic can sometimes affect the usability of the system. Our investigations model the relationships between internal software attributes and externally visible usability factors. We propose a pattern-based approach for dealing with these relationships. We conclude by discussing how these patterns can lead to a methodological framework for improving interactive system architectures, and how these patterns can support the integration of usability in the software design process.     

Using shared representations to improve coordination and intent inference

In groupware, users must communicate about their intentions and aintain common knowledge via communication channels that are explicitly designed into the system. Depending upon the task, generic communication tools like chat or a shared whiteboard may not be sufficient to support effective coordination. We have previously reported on a methodology that helps the designer develop task specific communication tools, called coordinating representations, for groupware systems. Coordinating representations lend structure and persistence to coordinating information. We have shown that coordinating representations are readily adopted by a user population, reduce coordination errors, and improve performance in a domain task. As we show in this article, coordinating representations present a unique opportunity to acquire user information in collaborative, user-adapted systems. Because coordinating representations support the exchange of coordinating information, they offer a window onto task and coordination-specific knowledge that is shared by users. Because they add structure to communication, the information that passes through them can be easily exploited by adaptive technology. This approach provides a simple technique for acquiring user knowledge in collaborative, user-adapted systems. We document our application of this approach to an existing groupware system. Several empirical results are provided. First, we show how information that is made available by a coordinating representation can be used to infer user intentions. We also show how this information can be used to mine free text chat for intent information, and show that this information further enhances intent inference. Empirical data shows that an automatic plan generation component, which is driven by information from a coordinating representation, reduces coordination errors and cognitive effort for its users. Finally, our methodology is summarized, and we present a framework for comparing our approach to other strategies for user knowledge acquisition in adaptive systems.     

A method to elicit architecturally sensitive usability requirements: its integration into a software development process

In the Human-Computer Interaction (HCI) community, software usability has primarily been concerned with the presentation of information, more precisely with the user interface. However, some usability problems can prove costly to fix if the changes require modifications that reach beyond the presentation layer, namely those that cannot be easily accommodated by the software architecture. Taking into account some usability requirements earlier in the software development cycle, specifically prior to the architectural design phase, can reduce the cost of these modifications. There is a scarcity of methods and guidelines with the scope to direct users in eliciting the usability requirements that can impact the software architecture. This paper proposes a usability-driven adaptation of the quality attribute workshop (QAW) to assist software development organizations in discovering and documenting usability requirements. It shows how this method can be integrated into a software development process, by discussing how the existing software framework workflows can be adjusted to take this new activity into consideration. A preliminary exercise was conducted to help discern the utility and the limits of the proposed method. Participants with different levels of knowledge of usability and comprehension of the system being developed found the method constructive, as it guided them in identifying the architecturally relevant usability requirements. It also helped determine the usability aspects that would not necessarily have been defined if this technique had not been employed.     

面向路径的测试数据自动生成工具及其图形界面的Tcl/Tk设计

面向路径的测试数据生成问题是软件测试中一个基本问题。文章介绍了自主开发的面向路径的测试数据自动生成工具。Tcl/Tk是一种图形界面设计工具,其功能强大,可运行于Windows,UNIX等操作系统上,具有良好的可移植性。文章简要介绍Tcl/Tk,并给出用Tcl/Tk设计面向路径的测试数据自动生成工具的图形界面的方法。