Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

Algorithms to Separate $\{0,\frac{1}{2}\}$ -Chvátal-Gomory Cuts

Chvátal-Gomory cuts are among the most well-known classes of cutting planes for general integer linear programs (ILPs). In case the constraint multipliers are either 0 or , such cuts are known as -cuts. It has been proven by Caprara and Fischetti (Math. Program. 74:221–235, 1996) that separation of -cuts is -hard. In this paper, we study ways to separate -cuts effectively in practice. We propose a range of preprocessing rules to reduce the size of the separation problem. The core of the preprocessing builds a Gaussian elimination-like procedure. To separate the most violated -cut, we formulate the (reduced) problem as integer linear program. Some simple heuristic separation routines complete the algorithmic framework. Computational experiments on benchmark instances show that the combination of preprocessing with exact and/or heuristic separation is a very vital idea to generate strong generic cutting planes for integer linear programs and to reduce the overall computation times of state-of-the-art ILP-solvers.     

A flexible,object-centric approach for business process modelling

Mainstream business process modelling techniques often promote a design paradigm wherein the activities that may be performed within a case, together with their usual execution order, form the backbone on top of which other aspects are anchored. This Fordist paradigm, while effective in standardised and production-oriented domains, breaks when confronted with processes in which case-by-case variations and exceptions are the norm. We contend that the effective design of flexible processes calls for a substantially different modelling paradigm. Motivated by requirements from the human services domain, we explore the hypothesis that a framework consisting of a small set of coordination concepts, combined with established object-oriented modelling principles, provides a suitable foundation for designing highly flexible processes. Several human service delivery processes have been designed using this framework, and the resulting models have been used to realise a system to support these processes in a pilot environment.     

Multirate Explicit Adams Methods for Time Integration of Conservation Laws

This paper constructs multirate linear multistep time discretizations based on Adams-Bashforth methods. These methods are aimed at solving conservation laws and allow different timesteps to be used in different parts of the spatial domain. The proposed family of discretizations is second order accurate in time and has conservation and linear and nonlinear stability properties under local CFL conditions. Multirate timestepping avoids the necessity to take small global timesteps—restricted by the largest value of the Courant number on the grid—and therefore results in more efficient computations. Numerical results obtained for the advection and Burgers’ equations confirm the theoretical findings. This work was supported by the National Science Foundation through award NSF CCF-0515170.     

Reduced Aggregate Scattering Operators for Path Tracing

Aggregate scattering operators (ASOs) describe the overall scattering behavior of an asset (i.e., an object or volume, or collection thereof) accounting for all orders of its internal scattering. We propose a practical way to precompute and compactly store ASOs and demonstrate their ability to accelerate path tracing. Our approach is modular avoiding costly and inflexible scene‐dependent precomputation. This is achieved by decoupling light transport within and outside of each asset, and precomputing on a per‐asset level. We store the internal transport in a reduced‐dimensional subspace tailored to the structure of the asset geometry, its scattering behavior, and typical illumination conditions, allowing the ASOs to maintain good accuracy with modest memory requirements. The precomputed ASO can be reused across all instances of the asset and across multiple scenes. We augment ASOs with functionality enabling multi‐bounce importance sampling, fast short‐circuiting of complex light paths, and compact caching, while retaining rapid progressive preview rendering. We demonstrate the benefits of our ASOs by efficiently path tracing scenes containing many instances of objects with complex inter‐reflections or multiple scattering.     

Expedited simulation‐driven design optimization of UWB antennas by means of response features

In this work, a method for fast design optimization of broadband antennas is considered. The approach is based on a feature‐based optimization (FBO) concept where reflection characteristics of the structure at hand are formulated in terms of suitably defined feature points. Redefinition of the design problem allows for reducing the design optimization cost, because the dependence of feature point coordinates on antenna dimensions is less nonlinear than for the original frequency characteristics (here, S‐parameters). This results in faster convergence of the optimization algorithm. The cost of the design process is further reduced using variable‐fidelity electromagnetic (EM) simulation models. In case of UWB antennas, the feature points are defined, among others, as the levels of the reflection characteristic at its local in‐band maxima, as well as location of the frequency point which corresponds to acceptable reflection around the lower corner frequency within the UWB band. Also, the number of characteristic points depends on antenna topology and its dimensions. Performance of FBO‐based design optimization is demonstrated using two examples of planar UWB antennas. Moreover, the computational cost of the approach is compared with conventional optimization driven by a pattern search algorithm. Experimental validation of the numerical results is also provided.     

An industry experiment on the effects of test-driven development on external quality and productivity

Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an industrial setting. We conducted an experiment with 24 professionals from three different sites of a software organization. We chose a repeated-measures design, and asked subjects to implement TDD and incremental test last development (ITLD) in two simple tasks and a realistic application close to real-life complexity. To analyse our findings, we applied a repeated-measures general linear model procedure and a linear mixed effects procedure. We did not observe a statistical difference between the quality of the work done by subjects in both treatments. We observed that the subjects are more productive when they implement TDD on a simple task compared to ITLD, but the productivity drops significantly when applying TDD to a complex brownfield task. So, the task complexity significantly obscured the effect of TDD. Further evidence is necessary to conclude whether TDD is better or worse than ITLD in terms of external quality and productivity in an industrial setting. We found that experimental factors such as selection of tasks could dominate the findings in TDD studies.     

Examining the Effectiveness of Gamification in Human Computation

Within the human computation paradigm, gamification is increasingly gaining interest. This is because an enjoyable experience generated by game features can be a powerful approach to attract participants. Although potentially useful, little research has been conducted into understanding the effectiveness of gamification in human computation. In this experimental study, we operationalized effectiveness as perceived engagement and user acceptance and examined it by comparing the performance of a gamified human computation system against a non-gamified version. We also investigate the determinants of acceptance and how their effects differ between these two systems. Analysis of our data found that participants experienced more engagement and showed higher behavioral intentions toward the gamified system. Moreover, perceived output quality and perceived engagement were significant determinants of acceptance of the gamified system. In contrast, determinants for acceptance of the non-gamified system were perceived output quality and perceived usability.     

Maximal controllability of input constrained unstable systems by the addition of implicit constraints

The null controllable set of a system is the largest set of states that can be controlled to the origin. Control systems that have a region of attraction equal to the null controllable set are said to be maximally controllable closed-loop systems. In the case of open-loop unstable plants with amplitude constrained control it is well known that the null controllable set does not cover the entire state-space. Further the combination of input constraints and unstable system dynamics results in a set of state constraints which we call implicit constraints. It is shown that the simple inclusion of implicit constraints in a controller formulation results in a controller that achieves maximal controllability for a class of open-loop unstable systems.     

Discovery of Intrinsic Primitives on Triangle Meshes

The discovery of meaningful parts of a shape is required for many geometry processing applications, such as parameterization, shape correspondence, and animation. It is natural to consider primitives such as spheres, cylinders and cones as the building blocks of shapes, and thus to discover parts by fitting such primitives to a given surface. This approach, however, will break down if primitive parts have undergone almost‐isometric deformations, as is the case, for example, for articulated human models. We suggest that parts can be discovered instead by finding intrinsic primitives, which we define as parts that posses an approximate intrinsic symmetry. We employ the recently‐developed method of computing discrete approximate Killing vector fields (AKVFs) to discover intrinsic primitives by investigating the relationship between the AKVFs of a composite object and the AKVFs of its parts. We show how to leverage this relationship with a standard clustering method to extract k intrinsic primitives and remaining asymmetric parts of a shape for a given k. We demonstrate the value of this approach for identifying the prominent symmetry generators of the parts of a given shape. Additionally, we show how our method can be modified slightly to segment an entire surface without marking asymmetric connecting regions and compare this approach to state‐of‐the‐art methods using the Princeton Segmentation Benchmark.