Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

Network mapping services for dynamic selection of web services: promises and challenges

Web services provide the possibility of dynamically integrating distributed service components scattered over the Internet to fulfill sophisticated business demands. The effectiveness of service integration depends critically on the selection of best-fit service components. In this paper, we first review several research streams related to server selection. We then propose a framework of dynamic service discovery, termed Network Mapping Services (NMS), which combines prior efforts on network-aware applications and server selection into a coherent structure. As successful implementation of NMS requires the collaboration among all participating service providers, we also discuss several incentive schemes and compare their effectiveness in supporting collaboration.This research was sponsored in part by the National Science Foundation under contract number IIS-0219825.     

A systematic mapping study of gamification models oriented to motivational characteristics

ABSTRACT

Context: Gamification focuses on the improvement of users' engagement when performing tasks by making use of game mechanics and elements in order to increase their motivations. Many researches have developed gamification models supporting a variety of motivational characteristics to provide engagement solutions for different areas. Objective: This paper carries out a systematic mapping in the field of gamification, looking for models with motivational characteristics in an attempt to characterise the state of the art of this field, identifying gaps and tendencies for further research. Method: We carried out a systematic mapping aiming at finding the primary studies in the existing literature, which were later classified and analysed according to twelve criteria. Results: We analysed 70 papers that resulted in 17 primary studies, published until September 2016. Most of them focus on Education, making use of Gamification to increase the motivation of a learning process. The gamification mechanics and elements most used were Badges/Achievements and Points/ExperiencePoints(XP), and most of the studies were not validated, thus not providing empirical evidence of the impact of gamification. Conclusions: Existing research in the field is somehow preliminary, and more research effort to analyse the applicability of the models and their respective evaluations would be needed.     

Extensible soft semantic web services agent

Web services technology is critical for the success of business integration and other application fields such as bioinformatics. However, there are two challenges facing the practicality of Web services: (a) efficient location of the Web service registries that contain the requested Web services and (b) efficient retrieval of the requested services from these registries with high quality of service (QoS). The main reason for this problem is that current Web services technology is not semantic-oriented. Several proposals have been made to add semantics to Web services to facilitate discovery and composition of relevant Web services. Such proposals are being referred to as semantic Web services (SWS). However, most of these proposals do not address the second problem of retrieval of web services with high QoS. In this paper, we propose a framework called soft semantic Web services agent (soft SWS agent) for providing high QoS Semantic Web services using soft computing methodology. Since different application domains have different requirement for QoS, it is impractical to use classical mathematical modeling methods to evaluate the QoS of semantic Web services. We use fuzzy neural networks with Genetic Algorithms (GA) as our study case. Simulation results show that the soft computing methodology is practicable to handle fuzzy and uncertain QoS metrics effectively.     

Client-side selection of replicated web services: An empirical assessment

Replicating web services over physically distributed servers can offer client applications a number of QoS benefits, including higher availability and reduced response time. However, selecting the “best” service replica to invoke at the client-side is not a trivial task, as this requires taking into account factors such as local and external network conditions, and the servers’ current workload. This paper presents an empirical assessment of five representative client-side service selection policies for accessing replicated web services. The assessment measured the response time obtained with each of the five policies, at two different client configurations, when accessing a world-wide replicated service with four replicas located in three continents. The assessment’s results were analyzed both quantitatively and qualitatively. In essence, the results show that, in addition to the QoS levels provided by the external network and the remote servers, characteristics of the local client environment can have a significant impact on the performance of some of the policies investigated. In this regard, the paper presents a set of guidelines to help application developers in identifying a server selection policy that best suits a particular service replication scenario.     

Software product line testing - A systematic mapping study

Context

Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped.

Objective

This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research.

Method

A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type.

Results

A majority of the papers are of proposal research types (64%). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority.

Conclusions

More validation and evaluation research is needed to provide a better foundation for SPL testing.     

Quality of service for workflows and web service processes

Workflow management systems (WfMSs) have been used to support various types of business processes for more than a decade now. In workflows or Web processes for e-commerce and Web service applications, suppliers and customers define a binding agreement or contract between the two parties, specifying quality of service (QoS) items such as products or services to be delivered, deadlines, quality of products, and cost of services. The management of QoS metrics directly impacts the success of organizations participating in e-commerce. Therefore, when services or products are created or managed using workflows or Web processes, the underlying workflow engine must accept the specifications and be able to estimate, monitor, and control the QoS rendered to customers. In this paper, we present a predictive QoS model that makes it possible to compute the quality of service for workflows automatically based on atomic task QoS attributes. We also present the implementation of our QoS model for the METEOR workflow system. We describe the components that have been changed or added, and discuss how they interact to enable the management of QoS.     

Combining service-orientation and software product line engineering: A systematic mapping study

ContextService-Orientation (SO) is a rapidly emerging paradigm for the design and development of adaptive and dynamic software systems. Software Product Line Engineering (SPLE) has also gained attention as a promising and successful software reuse development paradigm over the last decade and proven to provide effective solutions to deal with managing the growing complexity of software systems.ObjectiveThis study aims at characterizing and identifying the existing research on employing and leveraging SO and SPLE.MethodWe conducted a systematic mapping study to identify and analyze related literature. We identified 81 primary studies, dated from 2000–2011 and classified them with respect to research focus, types of research and contribution.ResultThe mapping synthesizes the available evidence about combining the synergy points and integration of SO and SPLE. The analysis shows that the majority of studies focus on service variability modeling and adaptive systems by employing SPLE principles and approaches.In particular, SPLE approaches, especially feature-oriented approaches for variability modeling, have been applied to the design and development of service-oriented systems. While SO is employed in software product line contexts for the realization of product lines to reconcile the flexibility, scalability and dynamism in product derivations thereby creating dynamic software product lines.ConclusionOur study summarizes and characterizes the SO and SPLE topics researchers have investigated over the past decade and identifies promising research directions as due to the synergy generated by integrating methods and techniques from these two areas.     

A high performance backoff protocol for fast execution of composite web services

Using web services to expose applications over the Internet is now a widely accepted practice. Currently, there are several ongoing efforts that provide ways to effectively compose web services distributed across different organizations. One of the problems underlying the deployment of such composite services on the web, however, is service co-allocation that arises when a composite service needs to ensure all the required component services to be available for execution at the same time. Motivated by this, this paper presents a new decentralized protocol, named web service co-allocation protocol (WSCP), which can facilitate fast execution of composite web services. The proposed framework is an enhancement of the famous two phase commit protocol through the incorporation of tentative hold phase as well as the employment of a new high performance backoff protocol developed to better address the dynamics of the service co-allocation problem. The simulation results show that the proposed approach yields significant improvements over existing protocols.     

Supporting the semi-automatic semantic annotation of web services: A systematic literature review

ContextSemantically annotating web services is gaining more attention as an important aspect to support the automatic matchmaking and composition of web services. Therefore, the support of well-known and agreed ontologies and tools for the semantical annotation of web services is becoming a key concern to help the diffusion of semantic web services.ObjectiveThe objective of this systematic literature review is to summarize the current state-of-the-art for supporting the semantical annotation of web services by providing answers to a set of research questions.MethodThe review follows a predefined procedure that involves automatically searching well-known digital libraries. As a result, a total of 35 primary studies were identified as relevant. A manual search led to the identification of 9 additional primary studies that were not reported during the automatic search of the digital libraries. Required information was extracted from these 44 studies against the selected research questions and finally reported.ResultsOur systematic literature review identified some approaches available for semantically annotating functional and non-functional aspects of web services. However, many of the approaches are either not validated or the validation done lacks credibility.ConclusionWe believe that a substantial amount of work remains to be done to improve the current state of research in the area of supporting semantic web services.     

A systematic review on the functional testing of semantic web services

Semantic web services are gaining more attention as an important element of the emerging semantic web. Therefore, testing semantic web services is becoming a key concern as an essential quality assurance measure. The objective of this systematic literature review is to summarize the current state of the art of functional testing of semantic web services by providing answers to a set of research questions. The review follows a predefined procedure that involves automatically searching 5 well-known digital libraries. After applying the selection criteria to the results, a total of 34 studies were identified as relevant. Required information was extracted from the studies and summarized. Our systematic literature review identified some approaches available for deriving test cases from the specifications of semantic web services. However, many of the approaches are either not validated or the validation done lacks credibility. We believe that a substantial amount of work remains to be done to improve the current state of research in the area of testing semantic web services.     

A framework for QoS-aware binding and re-binding of composite web services

QoS-aware dynamic binding of composite services provides the capability of binding each service invocation in a composition to a service chosen among a set of functionally equivalent ones to achieve a QoS goal, for example minimizing the response time while limiting the price under a maximum value.This paper proposes a QoS-aware binding approach based on Genetic Algorithms. The approach includes a feature for early run-time re-binding whenever the actual QoS deviates from initial estimates, or when a service is not available. The approach has been implemented in a framework and empirically assessed through two different service compositions.     

Per-flow optimal service selection for Web services based processes

With the development of the Service-Oriented Computing (SOC) paradigm, flexible business processes can be defined from independently developed services. Multiple services corresponding to the same functionality but characterized by different Quality of Service (QoS) attributes can be offered by different service providers and the best set of Web services can be selected at run-time in order to maximize the QoS for end users.In the literature many approaches have been proposed for the optimal service selection which is usually performed on a per-request basis, i.e., considering a single process invocation. In this paper we propose a broker-based framework which solves the optimal service selection on a per-flow basis. Multiple applications, defined as different BPEL processes are considered at the same time and multiple requests to the same process are optimized concurrently.Service selection is formulated as a constrained non-linear multi-criteria optimization problem and an heuristic algorithm is proposed to determine a scalable and efficient solution.A comparison with top performing state-of-the-art approaches for a number of different scenarios of interest is also provided. Results show that the overall optimization time reduction which can be achieved by our solution is proportional to the system incoming workload. Order of magnitudes optimization time improvements can be obtained if compared to alternative methods. Moreover, our solution is robust to workload prediction uncertainty.     

Cloud computing service composition: A systematic literature review

The increasing tendency of network service users to use cloud computing encourages web service vendors to supply services that have different functional and nonfunctional (quality of service) features and provide them in a service pool. Based on supply and demand rules and because of the exuberant growth of the services that are offered, cloud service brokers face tough competition against each other in providing quality of service enhancements. Such competition leads to a difficult and complicated process to provide simple service selection and composition in supplying composite services in the cloud, which should be considered an NP-hard problem. How to select appropriate services from the service pool, overcome composition restrictions, determine the importance of different quality of service parameters, focus on the dynamic characteristics of the problem, and address rapid changes in the properties of the services and network appear to be among the most important issues that must be investigated and addressed. In this paper, utilizing a systematic literature review, important questions that can be raised about the research performed in addressing the above-mentioned problem have been extracted and put forth. Then, by dividing the research into four main groups based on the problem-solving approaches and identifying the investigated quality of service parameters, intended objectives, and developing environments, beneficial results and statistics are obtained that can contribute to future research.     

QCWS: an implementation of QoS-capable multimedia web services

QoS (Quality of Service), which defines service quality such as latency, availability, timeliness and reliability, is important for web applications that provide real-time information, multimedia content, or time-critical services. Many web applications are best implemented by servers with a guaranteed server processing capacity. In this research, we study the QoS control issues using the current Web services standards. We propose a QoS-capable Web service architecture, QCWS, by deploying a QoS broker between Web service clients and providers. The functions of the QoS broker module include tracking QoS information about servers, making selection decisions for clients, and negotiating with servers to get QoS agreements. We have implemented a QCWS prototype using IBM WSDK, enhanced with simple QoS capabilities. We have measured the performance running under different service priorities.     

一种基于度量的Web服务质量机制

提出了一种简单、准确和高效的带宽调节算法,该算法对每一个服务请求设置一个带宽标准,同时根据对服务请求的分类和Web服务器系统的有效带宽,来分配及调节每一个服务请求的实际带宽,构成一种基于度量的Web服务质量机制,使Web服务器系统能为不同类型的客户请求提供有区分的和有服务质量的Web服务。     

基于信誉模型的Web服务优化选择

随着Web服务技术的发展与广泛应用,提供相同功能的Web服务越来越多,而如何在其所提供的Web服务中选择最优服务成为关键问题.传统的基于服务质量(QoS)模型的选择方案已不能满足此要求.提出了一种基于信誉模型并结合QoS的优化选择方案,首先利用工作流组织模型确定服务使用者之间的信任度结,结合可信用户的反馈评价值,确定服务的主观信誉度,然后根据Web服务的QoS量化模型,计算出客观质量值,最后结合二者选出最优的服务.     

Process-context aware matchmaking for web service composition

Web service composition can help software developer design more powerful and flexible applications according to requirements of enterprise. But during compositing, how to discover suitable web services is a critical problem in design and implementing application-oriented web service technologies. The traditional keyword-based matchmaking approach is difficult to help developer to find suitable service. Current researches find that to attaching semantics to each registered service can help improve the precision of matchmaking. The improvement can help developer find more suitable service for business process. This paper proposes a novel approach of semantics-based matchmaking, which is named process-context aware matchmaking. The process-context aware matchmaking discovers the suitable service during web service composite modeling. During matchmaking, the approach utilizes not only semantics of technical process but also that of business process of a registered service, thus further improving the precision of matchmaking. We integrate the process-context aware matchmaking with business-process-driven web service composition in an integrated development environment based on Eclipse. The performance evaluation shows that performance overhead of this novel approach is acceptable.     

Method of design and specification of web services based on quality system documentation

In this paper, we present a method for development and specification of web services based on the quality system documentation. The general assumption is that service oriented architecture is based on business services and these business services mostly correspond to exchanged documentation in a real business system. Documentation of a quality system is recognized in form of documents that describe business processes in a real business system and identify exchanged documentation with environment. Presented method uses documentation of quality system and documentation flow for web service specification. We developed the CASE tool for web service specification based on a new approach, and we compare it to other existing tools.