Integration of usability evaluation and model-based software development

Model-based software development is carried out as a well defined process. Depending on the applied approach, different phases can be distinguished, e.g. requirements specification, design, prototyping, implementation and usability evaluation. During this iterative process manifold artifacts are developed and modified, including, e.g. models, source code and usability evaluation data. CASE tools support the development stages well, but lack a seamless integration of usability evaluation methods. We aim at bridging the gap between development and usability evaluation, through enabling the cooperative use of artifacts with the particular tools. As a result of integration usability experts save time to prepare an evaluation and evaluation results can be easier incorporated back into the development process. We show exemplary our work on enhancing the Eclipse framework to support usability evaluation for task model-based software development.     

An approach to usability evaluation of e-learning applications

Despite recent advances of electronic technologies in e-learning, a consolidated evaluation methodology for e-learning applications is not available. The evaluation of educational software must consider its usability and more in general its accessibility, as well as its didactic effectiveness. This work is a first step towards the definition of a methodology for evaluating e-learning applications. Specific usability attributes capturing the peculiar features of these applications are identified. A preliminary user study involving a group of e-students, observed during their interaction with an e-learning application in a real situation, is reported. Then, the proposal is put forward to adapt to the e-learning domain a methodology for systematic usability evaluation, called SUE. Specifically, evaluation patterns are proposed that are able to drive the evaluators in the analysis of an e-learning application.     

The evaluation of an innovative intelligent thermostat interface: universal usability and age differences

Usability trials were carried out with a simulation of an early version of an intelligent thermostat interface. The multimodal interaction dialog took place in speech, sound, graphics and touch. Usability of new concepts of interaction was assessed. Most user behavior and problems of use were equal for old and young subjects (between 25 and 73 years of age). Relative high usability was observed with the new interaction styles, in particular with the task-based dialog and with the suggestions given by the thermostat. Even the oldest subjects could accomplish complicated programming. Age-dependent differences in usability were found for certain interface details, such as finding a hidden menu item. The summarized typical "modern" interaction details, which caused problems, were significantly less usable for the older cohort (over 59 years of age), compared to the young (under 46 years of age). Electronic Publication     

A systematic mapping study of API usability evaluation methods

An Application Programming Interface (API) provides a programmatic interface to a software component that is often offered publicly and may be used by programmers who are not the API’s original designers. APIs play a key role in software reuse. By reusing high quality components and services, developers can increase their productivity and avoid costly defects. The usability of an API is a qualitative characteristic that evaluates how easy it is to use an API. Recent years have seen a considerable increase in research efforts aiming at evaluating the usability of APIs. An API usability evaluation can identify problem areas and provide recommendations for improving the API. In this systematic mapping study, we focus on 47 primary studies to identify the aim and the method of the API usability studies. We investigate which API usability factors are evaluated, at which phases of API development is the usability of API evaluated and what are the current limitations and open issues in API usability evaluation. We believe that the results of this literature review would be useful for both researchers and industry practitioners interested in investigating the usability of API and new API usability evaluation methods.     

Empirical validation of an automatic usability evaluation method

Today, the success of a software application strongly depends on the usability of its interface, so the evaluation of interfaces has become a crucial aspect of software engineering. It is recognized that automatic tools for graphical user interface evaluation may greatly reduce the costs of traditional activities performed during expert evaluation or user testing in order to estimate the success probability of an application. However, automatic methods need to be empirically validated in order to prove their effectiveness with respect to the attributes they are supposed to evaluate.In this work, we empirically validate a usability evaluation method conceived to assess consistency aspects of a GUI with no need to analyze the back-end. We demonstrate the validity of the approach by means of a comparative experimental study, where four web sites and a stand-alone interactive application are analyzed and the results compared to those of a human-based usability evaluation. The analysis of the results and the statistical correlation between the tool׳s rating and humans׳ average ratings show that the proposed methodology can indeed be a useful complement to standard techniques of usability evaluation.     

The effect of design on the usability and real world effectiveness of medical devices: A case study with adolescent users

Adolescents are currently overlooked in many fields of healthcare research and as a result are often required to use medical devices that have been designed for use by either children or adults. This can lead to poor adherence and a reduction in health outcomes.     

Identifying organizational barriers—A case study of usability work when developing software in the automation industry

This study investigates connections between usability efforts and organizational factors. This is an important field of research which so far appears to be insufficiently studied and discussed. It illustrates problems when working with software engineering tasks and usability requirements. It deals with a large company that manufactures industrial robots with an advanced user interface, which wanted to introduce usability KPIs, to improve product quality. The situation in the company makes this difficult, due to a combination of organizational and behavioural factors that led to a “wicked problem” that caused conflicts, breakdowns and barriers. Addressing these problems requires a holistic view that places context in the foreground and technological solutions in the background. Developing the right product requires communication and collaboration between multiple stakeholders. The inclusion of end users, who fully understand their own work context, is vital. Achieving this is dependent on organizational change, and management commitment. One step to beginning this change process may be through studying ways to introduce user-centred design processes.     

Information quality measurement of medical encoding support based on usability

Medical encoding support systems for diagnoses and medical procedures are an emerging technology that begins to play a key role in billing, reimbursement, and health policies decisions. A significant problem to exploit these systems is how to measure the appropriateness of any automatically generated list of codes, in terms of fitness for use, i.e. their quality. Until now, only information retrieval performance measurements have been applied to estimate the accuracy of codes lists as quality indicator. Such measurements do not give the value of codes lists for practical medical encoding, and cannot be used to globally compare the quality of multiple codes lists. This paper defines and validates a new encoding information quality measure that addresses the problem of measuring medical codes lists quality. It is based on a usability study of how expert coders and physicians apply computer-assisted medical encoding. The proposed measure, named ADN, evaluates codes Accuracy, Dispersion and Noise, and is adapted to the variable length and content of generated codes lists, coping with limitations of previous measures. According to the ADN measure, the information quality of a codes list is fully represented by a single point, within a suitably constrained feature space. Using one scheme, our approach is reliable to measure and compare the information quality of hundreds of codes lists, showing their practical value for medical encoding. Its pertinence is demonstrated by simulation and application to real data corresponding to 502 inpatient stays in four clinic departments. Results are compared to the consensus of three expert coders who also coded this anonymized database of discharge summaries, and to five information retrieval measures. Information quality assessment applying the ADN measure showed the degree of encoding-support system variability from one clinic department to another, providing a global evaluation of quality measurement trends.     

The influence of mobile phone location and screen orientation on driving safety and the usability of car-sharing software in-car use

One of the important parts of the Internet economy is the car-sharing economy. Nevertheless, a few numbers of studies have been dedicated to the issues of safety of driving and usability resulting from the car-sharing software application. The current study aims at analyzing the effect of phone location and screen orientation on usability and safety of driving in Chinese drivers who use car-sharing software (e.g., UBER) and comparing the differences in the safety of driving and usability among various car-sharing tasks. To this end, 24 experienced Chinese Uber drivers were employed using a driving simulator, and three tasks of car-sharing software were investigated. The variables of driving safety and usability were analyzed by Repeat Measurement ANOVA as well as Two-way ANOVA. It was found that when the phone is located on the left side of the steering wheel, usability and driving safety were better compared to locating it on the right side. The orientation of the mobile phone screen had a major impact on usability, while its impact on driving safety was trivial. The left-portrait mode showed the best performance. No significant impact on usability was found in terms of tasks (T1: ordering; T2: route checking; T3: destination search), while tasks showed a significant effect on driving safety. Among these tasks, the T3 task showed the highest impact on driving safety. The T2 task was in the second rank, followed by the T1 task (T1 < T2 < T3). The findings obtained from Chinese participants were different from the findings obtained in Japan and the United States.Relevance to industryThis finding can be used as a basis for the optimization design of ca-sharing software and as a foundation for improving the safety and usability of car-sharing software used by drivers in the car.     

Contextualising design: Aspects of using usability context analysis and hierarchical task analysis for software design

Usability Context Analysis is part of ISO 9241-11 and, as such, is recommended for use in user-centred design. However, studies involving a critical assessment of usability context analysis within the software domain are scarce; this paper details two small studies that involved the use of usability context analysis (UCA) and hierarchical task analysis (HTA). The first system involved designing a small subsite of a larger website, while the second system was a small database designed as part of an exercise by UK university students. Post-design evaluation suggested that the use of both UCA and HTA, rather than one or the other, was beneficial in gathering and analysing user requirements.     

Contextualising design: Aspects of using usability context analysis and hierarchical task analysis for software design

Usability Context Analysis is part of ISO 9241-11 and, as such, is recommended for use in user-centred design. However, studies involving a critical assessment of usability context analysis within the software domain are scarce; this paper details two small studies that involved the use of usability context analysis (UCA) and hierarchical task analysis (HTA). The first system involved designing a small subsite of a larger website, while the second system was a small database designed as part of an exercise by UK university students. Post-design evaluation suggested that the use of both UCA and HTA, rather than one or the other, was beneficial in gathering and analysing user requirements.     

An expert system for the selection of software design patterns

Abstract: This paper describes the development of a prototype expert system for the selection of design patterns that are used in object-oriented software. Design patterns provide one method of software reuse, which supports the goal of improved software development productivity. The prototype system represents an initial step towards providing an automated solution regarding the design pattern application problem, i.e. leading a designer to a suitable design pattern which is applicable to the problem at hand. The feasibility of using expert system technology to aid in the selection problem is demonstrated.     

Exploring the benefits of the combination of a software architecture analysis and a usability evaluation of a mobile application

Designing easy to use mobile applications is a difficult task. In order to optimize the development of a usable mobile application, it is necessary to consider the mobile usage context for the design and the evaluation of the user-system interaction of a mobile application. In our research we designed a method that aligns the inspection method “Software ArchitecTure analysis of Usability Requirements realizatioN” SATURN and a mobile usability evaluation in the form of a user test. We propose to use mobile context factors and thus requirements as a common basis for both inspection and user test. After conducting both analysis and user test, the results described as usability problems are mapped and discussed. The mobile context factors identified define and describe the usage context of a mobile application. We exemplify and apply our approach in a case study. This allows us to show how our method can be used to identify more usability problems than with each method separately. Additionally, we could confirm the validity and identified the severity of usability problems found by both methods. Our work presents how a combination of both methods allows to address usability issues in a more holistic way. We argue that the increased quantity and quality of results can lead to a reduction of the number of iterations required in early stages of an iterative software development process.     

A structured methodology for comparative evaluation of user interface designs using usability criteria and measures

Previous research in software ergonomics has indicated the importance of evaluating the usability of computer user interfaces. This paper presents a quantitative basis for selecting from among multiple alternative interfaces relative to multiple criteria of usability concern. The proposed model consists of two main phases: the prescreening phase and the evaluation phase. The prescreening phase involves expert judgment-based assessment with qualitative criteria. The first phase uses absolute measurement analytic hierarchy process (AHP) to filter possible alternative interfaces to a reasonable subset. The evaluation phase involves user-based assessment such as user testing, with quantitative criteria. The objective of the second phase is to evaluate a subset of alternatives using objective measures and to select the best alternative. A set of criteria and measures for evaluating the usability of computer software designs is presented. The proposed methodology is demonstrated in the interface design of a database system used to analyze trip cases information of nuclear power plant.Relevance to industry

There is currently a focus on the usability of interactive computer software. Software developers, interface designers or human factors engineers often confront the task of comparative evaluation among systems, versions or interface designs. The proposed methodology provides practitioners with a structured approach to select the best interface based on usability criteria and measures.     

Usability and user experience of medical devices: An overview of the current state,analysis methodologies,and future challenges

Improvements to the reliability and safety of medical devices are vital for healthcare systems. It is necessary to consider the user experience (UX) of patients and healthcare professionals at all the development stages of medical devices. Ergonomic design principles can also reduce the cost of medical device production. This study is a multilateral analytical literature review of UX and usability issues in healthcare and medical device design. The total number of literature sources presented is n = 88. The literature sources are classified according to the difference between UX and usability for various target products and services, including healthcare. The literature is reviewed with a focus on human-oriented aspects. This includes medical technology and device design, which are dependent on the user type, medical device scope, and area of use. A review of key medical device standards and regulation documents is presented. The main analysis methodologies for the UX and their advantages and disadvantages are discussed. The future challenges in healthcare and medical ergonomics are briefly highlighted. Above all, this study tried to examine the difference between usability and UX of general products and those of medical devices through the review of existing literature. Even standards do not reflect this well and need to be considered based on further research in academia and industry.Relevance to industryThe obtained results will help medical-device designers and healthcare professionals understand the main medical-research trends and improve the design process. Additionally, they will be helpful for increasing the satisfaction level among medical-device users and reducing user risks.     

Interpretation of a cross-cultural usability evaluation: A case study based on a hypermedia system for rare species management in Namibia

We present the results of a usability evaluation of a locally developed hypermedia information system aiming at conservation biologists and wildlife managers in Namibia. Developer and end user come from different ethnic backgrounds, as is common to software development in Namibia and many developing countries. To overcome both the cultural and the authoritarian gap between usability evaluator and user, the evaluation was held as a workshop with usability evaluators who shared the target users’ ethnic and social backgrounds. Different data collection methods were used and results as well as specific incidences recorded. Results suggest that it is difficult for Namibian computer users to evaluate functionality independently from content. Users displayed evidence of a passive search strategy and an expectation that structure is provided rather than self generated. The comparison of data collection methods suggests that questionnaires are inappropriate in Namibia because they do not elicit a truthful response from participants who tend to provide answers they think are “expected”. The paper concludes that usability goals and methods have to be determined and defined within the target users’ cultural context.     

A medical hand tool physical interaction evaluation approach for prototype testing using patient care simulators

The purpose of this study was to develop and test a physical ergonomics assessment approach for medical device handles. The method assesses wrist posture and force of exertion simultaneously by task element. Electrogoniometers and EMG sensors were connected through a data acquisition module for synchronization with video recordings of trials. Task analysis of video recordings were performed offline with Observer XT software. Average posture and force data across several repetitions of individual task elements were calculated and presented in a format suitable for informing product designers of specific issues during a test trial. A handle comfort questionnaire is proposed to survey subjective responses. The evaluation approach was applied to an endoscope needle in sampling a biopsy from the stomach wall using a gastrointestinal track simulator with ten physician surrogates. The results showed that for all task elements the wrist was in extension (33(°)-45(°)). Peak muscle forces ranged from 28% to 68% MVC across the three muscles studied. Muscle peak forces were above ACGIH HAL maximum threshold limits for four of the seven task elements, and above the action limit for all seven task elements for two muscles. The handle comfort questionnaire data also supported the high muscle force findings, and also on force distribution on the handle due to contact stresses. This combined approach could be used to collect and report detailed early stage ergonomics data from user trials on patient care simulators. The approach is proposed for use by medical device designers at the design stage of new products using prototypes, but it could also be used on existing products with real patients.     

Exploring principles of user-centered agile software development: A literature review

ContextIn the last decade, software development has been characterized by two major approaches: agile software development, which aims to achieve increased velocity and flexibility during the development process, and user-centered design, which places the goals and needs of the system’s end-users at the center of software development in order to deliver software with appropriate usability. Hybrid development models, referred to as user-centered agile software development (UCASD) in this article, propose to combine the merits of both approaches in order to design software that is both useful and usable.ObjectiveThis paper aims to capture the current state of the art in UCASD approaches and to derive generic principles from these approaches. More specifically, we investigate the following research question: Which principles constitute a user-centered agile software development approach?MethodWe conduct a systematic review of the literature on UCASD. Identified works are analyzed using a coding scheme that differentiates four levels of UCASD: the process, practices, people/social and technology dimensions. Through subsequent synthesis, we derive generic principles of UCASD.ResultsWe identified and analyzed 83 relevant publications. The analysis resulted in a comprehensive coding system and five principles for UCASD: (1) separate product discovery and product creation, (2) iterative and incremental design and development, (3) parallel interwoven creation tracks, (4) continuous stakeholder involvement, and (5) artifact-mediated communication.ConclusionOur paper contributes to the software development body of knowledge by (1) providing a broad overview of existing works in the area of UCASD, (2) deriving an analysis framework (in form a coding system) for works in this area, going beyond former classifications, and (3) identifying generic principles of UCASD and associating them with specific practices and processes.     

An empirical analysis of the computerization of interactive software evaluation systems

This research studies factor analysis of traditional survey- questionnaires for interactive software evaluation in order to construct a Principal Factor Conversion Function (PFCF). Such a PFCF may be used in several ways: first, for exploratory purposes, to discover the principal factors of the traditional questionnaire; second to test the potential of data reduction within significant loss of information; and third to compare the traditional manual evaluation to a condensed computerized interactive software evaluation. A CAI system was used as the software to be evaluated in this research.