A systematic mapping study of API usability evaluation methods

An Application Programming Interface (API) provides a programmatic interface to a software component that is often offered publicly and may be used by programmers who are not the API’s original designers. APIs play a key role in software reuse. By reusing high quality components and services, developers can increase their productivity and avoid costly defects. The usability of an API is a qualitative characteristic that evaluates how easy it is to use an API. Recent years have seen a considerable increase in research efforts aiming at evaluating the usability of APIs. An API usability evaluation can identify problem areas and provide recommendations for improving the API. In this systematic mapping study, we focus on 47 primary studies to identify the aim and the method of the API usability studies. We investigate which API usability factors are evaluated, at which phases of API development is the usability of API evaluated and what are the current limitations and open issues in API usability evaluation. We believe that the results of this literature review would be useful for both researchers and industry practitioners interested in investigating the usability of API and new API usability evaluation methods.     

Exploring perceptions of web accessibility: a survey approach

The equality of access – accessibility – is difficult to quantify, define, or agree upon. Our previous work analysed the responses of web accessibility specialists in regard to a number of pre-defined definitions of accessibility. While uncovering much, this analysis did not allow us to quantify the communities’ understanding of the relationship accessibility has with other domains and assess how the community scopes accessibility. In this case, we asked over 300 people, with an interest in accessibility, to answer 33 questions surrounding the relationship between accessibility, user experience (UX), and usability; inclusion and exclusion; and evaluation, in an attempt to harmonise our understanding of web accessibility. We found that respondents think that accessibility and usability are highly related and also think that accessibility is applicable to everyone and not just people with disabilities. Respondents strongly agree that accessibility must be grounded on user-centred practices and that accessibility evaluation is more than just inspecting source code; however, they are divided as to whether training in ‘Web Content Accessibility Guidelines’ is necessary or not to assess accessibility. These perceptions are important for usability and UX professionals, developers of automated evaluation tools, and those practitioners running website evaluations.     

How to reduce the UX bottleneck – train your software developers

Usability and user experience (UX) methods come from academic environments, where industrial conditions such as time and resources are not of prime importance. Furthermore, usability and UX methods originate from a time when almost all software developments followed a traditional approach, such as the waterfall model. These two facts entail that existing methods often are too resource demanding and complex to apply directly into today’s agile, industrial environments. In this paper we make the claim that methods must be updated and tailored in order to be applicable within the agile, industrial development framework of today. We pursue a solution to simplify well-known methods and to train software developers to perform the UX work. To do this, three methods are modified via an iterative process together with the development of supporting materials. Software developers in three companies are trained in the methods to assess the approach. We find that it indeed is feasible to update and tailor existing usability and UX methods to fit into an agile, industrial environment. Furthermore, we show that it is possible to train developers to perform the usability and UX methods via one-day, in-situ sessions using an ‘instructor’-teaching approach. The training is based on hands-on exercises and real-life tasks. This further boosts the developers’ confidence in performing UX work and promises a better consideration of UX in the development phases. We evaluate our approach through observations of the developers performing the UX tasks on their own at a later point in time.     

Understanding of website usability: Specifying and measuring constructs and their relationships

Developing a usable website is pivotal for e-business success. Researchers have devoted effort to develop metrics, guidelines and theories of website usability, yet there still is a lack of consensus on the multifaceted dimensions of website usability and lack of investigation of the nomological networks among website usability constructs. This study first investigated the common dimensions of website usability by integrating the findings of previous studies and a focus group study with website usability experts. Instruments to measure the constructs were developed and empirically validated. Then nomological networks between website usability constructs and between those constructs and online purchase intention and purchase were examined. Three field studies including two questionnaire surveys and a causal mapping analysis were conducted. The research identified ten website usability constructs with strong psychometric properties. A number of nomological networks between usability constructs were discovered, contributing to identification of sources of variances of purchase intention and purchase behavior. Findings of this study are expected to provide useful insights for practitioners to develop more usable websites and for researchers to better assess the effect of website usability on online customer behavior.     

Integration of usability evaluation and model-based software development

Model-based software development is carried out as a well defined process. Depending on the applied approach, different phases can be distinguished, e.g. requirements specification, design, prototyping, implementation and usability evaluation. During this iterative process manifold artifacts are developed and modified, including, e.g. models, source code and usability evaluation data. CASE tools support the development stages well, but lack a seamless integration of usability evaluation methods. We aim at bridging the gap between development and usability evaluation, through enabling the cooperative use of artifacts with the particular tools. As a result of integration usability experts save time to prepare an evaluation and evaluation results can be easier incorporated back into the development process. We show exemplary our work on enhancing the Eclipse framework to support usability evaluation for task model-based software development.     

High and low spatial frequencies in website evaluations

Statement of Relevance:

Which features of websites are important for users' perceptions regarding aesthetics or usability? This study investigates how evaluations of aesthetic appeal and usability depend on high vs. low spatial frequencies. High spatial frequencies convey information on fine details, whereas low spatial frequencies convey information about the global layout. Participants rated aesthetic appeal and usability of 50 website screenshots from different domains. Screenshots were presented unfiltered, low-pass filtered with blurred targets or high-pass filtered with high-pass filtered targets. The main result is that low spatial frequencies can be seen to have a unique contribution in perceived website aesthetics, thus confirming a central prediction from processing fluency theory. There was no connection between low spatial frequencies and usability evaluations, whereas strong correlations were found between ratings of high-pass filtered websites and those of unfiltered websites in aesthetics and usability. This study thus offers a new perspective on the biological basis of users' website perceptions.

This research links ergonomics to neurocognitive models of visual processing. This paper investigates how high and low spatial frequencies, which are neurologically processed in different visual pathways, independently contribute to users' perceptions of websites. This is very relevant for theories of website perceptions and for practitioners of web design.     

A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors

As a mobile phone has various advanced functionalities or features, usability issues are increasingly challenging. Due to the particular characteristics of a mobile phone, typical usability evaluation methods and heuristics, most of which are relevant to a software system, might not effectively be applied to a mobile phone. Another point to consider is that usability evaluation activities should help designers find usability problems easily and produce better design solutions. To support usability practitioners of the mobile phone industry, we propose a framework for evaluating the usability of a mobile phone, based on a multi-level, hierarchical model of usability factors, in an analytic way. The model was developed on the basis of a set of collected usability problems and our previous study on a conceptual framework for identifying usability impact factors. It has multi-abstraction levels, each of which considers the usability of a mobile phone from a particular perspective. As there are goal-means relationships between adjacent levels, a range of usability issues can be interpreted in a holistic as well as diagnostic way. Another advantage is that it supports two different types of evaluation approaches: task-based and interface-based. To support both evaluation approaches, we developed four sets of checklists, each of which is concerned, respectively, with task-based evaluation and three different interface types: Logical User Interface (LUI), Physical User Interface (PUI) and Graphical User Interface (GUI). The proposed framework specifies an approach to quantifying usability so that several usability aspects are collectively measured to give a single score with the use of the checklists. A small case study was conducted in order to examine the applicability of the framework and to identify the aspects of the framework to be improved. It showed that it could be a useful tool for evaluating the usability of a mobile phone. Based on the case study, we improved the framework in order that usability practitioners can use it more easily and consistently.     

What Makes APIs Hard to Learn? Answers from Developers

The paper discusses the application program interface (API). Most software projects reuse components exposed through APIs. In fact, current-day software development technologies are becoming inseparable from the large APIs they provide. An API is the interface to implemented functionality that developers can access to perform various tasks. APIs support code reuse, provide high-level abstractions that facilitate programming tasks, and help unify the programming experience. A study of obstacles that professional Microsoft developers faced when learning to use APIs uncovered challenges and resulting implications for API users and designers. The article focuses on the obstacles to learning an API. Although learnability is only one dimension of usability, there's a clear relationship between the two, in that difficult-to-use APIs are likely to be difficult to learn as well. Many API usability studies focus on situations where developers are learning to use an API. The author concludes that as APIs keep growing larger, developers will need to learn a proportionally smaller fraction of the whole. In such situations, the way to foster more efficient API learning experiences is to include more sophisticated means for developers to identify the information and the resources they need-even for well-designed and documented APIs.     

Measuring website quality: asymmetric effect of user satisfaction

Website quality measurement tools have been largely static and have struggled to determine relevant attributes of user satisfaction. This study compares and contrasts attributes of user satisfaction based on usability guidelines seeking to identify practical easy-to-administer measurement tools. The website users assessed business school homepages according to six criteria and fulfilled a randomly assigned yet typical task. After completing the task, respondents assessed the same six website quality/satisfaction criteria again. The consumer–product relationship seems similar to the link between a user and a website. User satisfaction, just like consumer satisfaction, is asymmetric and non-linear. Content and navigation have been identified as key ingredients when users judged website quality, alerting web designers and website practitioners to focus more closely on those attributes. Similar lessons can be drawn for marketing professionals, who typically supervise or determine the content, structure and other website facets.     

基于模糊层次分析法的软件易用性过程综合评价方法

对软件易用性过程进行综合评价是开发可信软件产品的必要因素。通过构建软件易用性生命周期模型和软件易用性度量模型搭建软件易用性过程综合评价模型, 通过此模型对软件开发中的易用性问题进行有效的指导, 规范易用性开发过程, 从而保证能够产出高易用性的产品; 利用模糊层次分析法对此模型进行度量评价, 量化评估易用性过程优劣; 通过一个实例分析说明该方法可以有效地应用于软件开发过程中, 能够尽早地确保软件产品的易用性, 为开发高易用度的软件产品指明了方向。     

Collaborating for Change: Leveraging Campus Partnerships to Create a User-Centered Library Website

This article details a series of website usability studies conducted by librarians at Fresno State in collaboration with faculty and students from the university's anthropology department. Building on an ethnographic study of the library, we sought to understand how students used the site. Collaboration with individuals outside the library allowed us to use their expertise in behavioral research, accomplish a larger series of studies, and redesign the website from a less library-centric perspective. In this article, we detail the study's genesis, methodology, results, and offer our experiences using a large-scale usability test to drive changes to an academic library website.     

Designing usable online stores: A landscape preference perspective

Developing an explanatory theoretical model of website usability is pivotal for understanding usable website design. Such a model would explain and help predict the effects of website usability on online purchases, but few studies have been devoted to such model development. As an exploratory effort, we adopted and extended Kaplan's landscape preference model by including factors of legibility, coherence, variety, and mystery, and examined their effect on cognitive and affective appraisals and their impact on purchase intention. A field survey with 495 online customers using two different categories of websites was conducted to validate the research model. A multi-group analysis with gender and age was performed to cross-validate it. Our findings demonstrated that the proposed model explained a large amount of the variance of purchase intention, invariant across different subgroups. Key implications for theory and practice are discussed.     

An application of fuzzy AHP for evaluating course website quality

Although previous studies have identified various influences on course website effectiveness, the evaluation of the relative importance of these factors across different online learning experience groups has not been empirically determined. This study develops an evolution model that integrates triangular fuzzy numbers and analytic hierarchy process to develop a fuzzy evaluation model which prioritized the relative weights of course website quality factors. Firstly, this study conducts a review of the literature on course website quality to generate 16 sub-criteria along with four criteria used to measure course website quality. Secondly, a fuzzy AHP approach is adopted to determine the relative weights linking the above criteria between high and low online learning experience groups. The results indicated that there are some similarities and differences between high- and low-experience groups with regard to the evaluation of course website quality. The evaluation model and results can provide a valuable reference for system designers seeking to enhance course website effectiveness.     

Usability Meanings and Interpretations in ISO Standards

The usability of a software product has recently become a key software quality factor. The International Organization for Standardization (ISO) has developed a variety of models to specify and measure software usability but these individual models do not support all usability aspects. Furthermore, they are not yet well integrated into current software engineering practices and lack tool support. The aim of this research is to survey the actual representation (meanings and interpretations) of usability in ISO standards, indicate some of existing limitations and address them by proposing an enhanced, normative model for the evaluation of software usability.     

Analyzing compliance and application of usability guidelines and recommendations by web developers

There are many guidelines and recommendations for improving the user experience in websites. Web developers can follow these recommendations to improve the usability of the website and the user experience. Although guidelines and recommendations could not be the final solution to get the optimum usability level, they are useful and fast to apply, in comparison to other kind of usability improvements may involve many time and resources. Currently we can find many websites which contain big usability problems, some of these problems could be solved applying usability guidelines and recommendations. In this research work we analyse the degree of compliance of usability recommendations by web developers. We did an extensive compilation of usability recommendations designed to focus the user attention in the important parts of the website and reduce noise generated by unimportant parts. We analyse the behaviour of 20 web developers to obtain the degree of application of usability recommendations, before they received a specific course on usability, and after it. The purpose of the research is to analyse the degree of compliance with the usability recommendations focused on reducing noise. The two main objectives are (1) Determine which recommendations are the most forgotten by web developers without training in usability. (2) Determine which recommendations are most important according to web developers’ thinking. In the conclusions we determine which are the most important recommendations. What are the most applied recommendations and their degree of compliance. Reflections are also extracted about the relationship of the results obtained.     

Work-domain knowledge in usability evaluation: Experiences with Cooperative Usability Testing

Usability evaluation helps to determine whether interactive systems support users in their work tasks. However, knowledge about those tasks and, more generally, about the work-domain is difficult to bring to bear on the processes and outcome of usability evaluation. One way to include such work-domain knowledge might be Cooperative Usability Testing, an evaluation method that consists of (a) interaction phases, similar to classic usability testing, and (b) interpretation phases, where the test participant and the moderator discuss incidents and experiences from the interaction phases. We have studied whether such interpretation phases improve the relevance of usability evaluations in the development of work-domain specific systems. The study included two development cases. We conclude that the interpretation phases generate additional insight and redesign suggestions related to observed usability problems. Also, the interpretation phases generate a substantial proportion of new usability issues, thereby providing a richer evaluation output. Feedback from the developers of the evaluated systems indicates that the usability issues that are generated in the interpretation phases have substantial impact on the software development process. The benefits of the interpretation phases may be explained by the access these provide both to the test participants’ work-domain knowledge and to their experiences as users.     

A study on how usability flaws in GUI design increase mouse movements and consequently may affect users' health

The objective of this study is to discuss how software usability flaws may cause a significant increase in mouse movements and, as a potential side effect, may even affect users' health. During the literature review, this article examines the potential relationship between mouse movement and musculoskeletal disorders of the upper extremity, based on studies from medical sciences. Subsequently, in the main part of the present study, three software products were selected that had at least one usability flaw related to extra mouse movement (the selection of the software was made out of over 20 software programs that had been evaluated in our Software Quality Assessment Laboratory). For these products, all additional mouse movements were measured, involving actual users in various settings and computer configurations. The findings showed that even a single usability flaw may increase mouse movement to a magnitude of between 3.6 and 4.7 m/h. The article concludes that the role of software engineering is to focus on software usability as well, taking into account that a user friendly graphical user interface (GUI) which is able to eliminate unnecessary mouse movement may also eventually contribute to the reduction of fatigue and discomfort, caused by musculoskeletal disorders.     

An empirical study of factors affecting the perceived usability of websites for student Internet users

This study analysed Student Internet Users’ (SIUs’) perception of Web usability. Adopting a user testing method, seven Web Usability Factors (WUFs) were tested for their significance in affecting the ease of use of website. Several elements in websites were also tested for their significance in affecting the WUFs. Result shows the most significant WUF is related to the aesthetic appeal of a website, i.e., Use of Colour and Font. However, it was found that most Web developers are not focusing on this important WUF. Elements such as site map, site search, product image catalogue and others were also found to positively affect SIUs’ perception of the WUFs. The results presented in this paper can be used as guidelines for designing usable websites for SIUs.