Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

UX Curve: A method for evaluating long-term user experience

The goal of user experience design in industry is to improve customer satisfaction and loyalty through the utility, ease of use, and pleasure provided in the interaction with a product. So far, user experience studies have mostly focused on short-term evaluations and consequently on aspects relating to the initial adoption of new product designs. Nevertheless, the relationship between the user and the product evolves over long periods of time and the relevance of prolonged use for market success has been recently highlighted. In this paper, we argue for the cost-effective elicitation of longitudinal user experience data. We propose a method called the “UX Curve” which aims at assisting users in retrospectively reporting how and why their experience with a product has changed over time. The usefulness of the UX Curve method was assessed in a qualitative study with 20 mobile phone users. In particular, we investigated how users’ specific memories of their experiences with their mobile phones guide their behavior and their willingness to recommend the product to others. The results suggest that the UX Curve method enables users and researchers to determine the quality of long-term user experience and the influences that improve user experience over time or cause it to deteriorate. The method provided rich qualitative data and we found that an improving trend of perceived attractiveness of mobile phones was related to user satisfaction and willingness to recommend their phone to friends. This highlights that sustaining perceived attractiveness can be a differentiating factor in the user acceptance of personal interactive products such as mobile phones. The study suggests that the proposed method can be used as a straightforward tool for understanding the reasons why user experience improves or worsens in long-term product use and how these reasons relate to customer loyalty.     

一种可信软件设计方法及可信性评价

针对可信计算组织TCG(Trusted Computing Group)的信任链无法保障软件运行时动态可信的问题,对该信任链进行扩充,引入对软件运行时动态可信性的检测,提出了可信引擎驱动的可信软件信任链模型,并在此基础上提出了一种可信软件设计方法及可信性评价策略.通过引入描述软件可信行为轨迹的可信视图,在可信软件检查点处植入检查点传感器,将软件可信性融入软件设计中.通过对软件的完整性度量以及运行过程中软件行为轨迹的监测,实现软件的可信性保障.实验分析表明:采用该方法设计的软件能够有效地检测软件异常,并且成功检测软件异常的能力明显优于基于TCG信任链的软件.     

P2P环境下电子商务信任评估机制研究

为了解决P2P环境下电子商务安全问题,在深入研究P2P环境下电子商务信任模型的基础上,针对传统信任模型不足的问题,提出了一种利用哈希函数将加入群的节点随机地分配到各个群的方法,降低了节点共谋攻击的危险。考虑到当一个节点偶尔提供一次恶意服务时,并不能证明是恶意节点,只需要给节点警告。引入了交易时间和交易规模、交易次数等因素来进行信任值的计算,利用惩罚项来限制交易节点提供恶意服务。节点的信任值很大程度上依靠同群节点和有外连接的群的节点来进行评估,仿真实验结果表明,该方法可以更加准确地评估节点的信任值,保证了网络的性能。     

信息安全视角下的C~4ISR系统耦合性研究

针对以耦合率为指标之一的C4ISR系统体系结构设计合理性验证评估问题,从功能系统间交互信息的安全性需求出发,通过推理揭示出密钥关系的耦合本质,进而提出并分析了密钥关联耦合的概念及其属性。给出了密钥关联耦合的定量度量方法,采用该方法能在体系结构设计结束后就实施并完成度量,且不必考虑密码算法及密钥长度等细节问题。利用Briand等人提出的准则证明了度量方法的科学性与合理性,并结合案例说明了度量方法的可行性与易操作性。     

基于GIS的村镇建设用地节地效果评价系统

针对目前村镇建设用地集约节约利用效果评价方法空缺及信息化低等问题,依据村镇建设用地节约集约利用效果评价模型,在地理信息系统组件技术和数据库技术支持下,采用可视化编程语言MicrosoftC#.net和GIS控件技术开发了村镇建设用地节约集约利用效果评价系统。以太仓市陆渡镇为开发案例,探讨了系统分析、设计和功能实现等主要技术问题。并利用该系统对太仓市陆渡镇村镇建设用地集约节约利用效果评价进行了评价,评价结果表明,该系统能够满足当前村镇建设用地集约节约利用效果评价工作的需要,为快速准确地实施节地效果评价提供技术支持。     

基于收发平衡判定的TCP流量回放方法

基于真实网络流量的互动式回放测试是当前针对防火墙、IPS等串接型安全设备进行测评的最新方法.文中在分析现有基于状态判定的TCP流量互动式回放方法基础之上,引入收发平衡机制,提出了一种基于收发平衡和状态判定相结合的新的TCP流量回放方法.通过在发送TCP数据包前优先进行收发平衡判定将数据包发送出去,提出的方法能够有效减少TCP流量在发送过程中的状态判定开销,提高回放性能.对引入收发平衡机制前后的TCP流量回放方法的差异进行了分析比较.从单个TCP会话特性、并发会话流量特性、网络传输延迟与丢包等角度分析验证了影响引入收发平衡机制后的算法有效性的因素.实际流量实验表明,文中所提方法在回放TCP流量时性能有显著提升,适用于在更大规模的流量环境下对防火墙、IPS等串接型网络安全设备进行测评.     

Rater reliability and concurrent validity of the Keyboard Personal Computer Style instrument (K-PeCS)

This paper describes the inter-rater and intra-rater reliability, and the concurrent validity of an observational instrument, the Keyboard Personal Computer Style instrument (K-PeCS), which assesses stereotypical postures and movements associated with computer keyboard use. Three trained raters independently rated the video clips of 45 computer keyboard users to ascertain inter-rater reliability, and then re-rated a sub-sample of 15 video clips to ascertain intra-rater reliability. Concurrent validity was assessed by comparing the ratings obtained using the K-PeCS to scores developed from a 3D motion analysis system. The overall K-PeCS had excellent reliability [inter-rater: intra-class correlation coefficients (ICC)=.90; intra-rater: ICC=.92]. Most individual items on the K-PeCS had from good to excellent reliability, although six items fell below ICC=.75. Those K-PeCS items that were assessed for concurrent validity compared favorably to the motion analysis data for all but two items. These results suggest that most items on the K-PeCS can be used to reliably document computer keyboarding style.