FSM-based incremental conformance testing methods

The development of appropriate test cases is an important issue for conformance testing of protocol implementations and other reactive software systems. A number of methods are known for the development of a test suite based on a specification given in the form of a finite state machine. In practice, the system requirements evolve throughout the lifetime of the system and the specifications are modified incrementally. We adapt four well-known test derivation methods, namely, the HIS, W, Wp, and UIOv methods, for generating tests that would test only the modified parts of an evolving specification. Some application examples and experimental results are provided. These results show significant gains when using incremental testing in comparison with complete testing, especially when the modified part represents less than 20 percent of the whole specification.     

Predicting where faults can hide from testing

Sensitivity analysis, which estimates the probability that a program location can hide a failure-causing fault, is addressed. The concept of sensitivity is discussed, and a fault/failure model that accounts for fault location is presented. Sensitivity analysis requires that every location be analyzed for three properties: the probability of execution occurring, the probability of infection occurring, and the probability of propagation occurring. One type of analysis is required to handle each part of the fault/failure model. Each of these analyses is examined, and the interpretation of the resulting three sets of probability estimates for each location is discussed. The relationship of the approach to testability is considered     

User modelling and user adapted interaction in an intelligent tutoring system

User modelling and user-adapted interaction are crucial to the provision of true individualised instruction, which intelligent tutoring systems strive to achieve. This paper presents how user (student) modelling and student adapted instruction is achieved in FITS, an intelligent tutoring system for the fractions domain. Some researchers have begun questioning both the need for detailed student models as well as the pragmatic possibility of building them. The key contributions of this paper are in its attempt to rehabilitate student modelling/adaptive tutoring within ITSs and in FITS's practical use of simple techniques to realise them with seemingly encouraging results; some illustrations are given to demonstrate the latter.     

Extending mutation testing to find environmental bugs

Environmental bugs are bugs caused by limitations of precision or capacity in the environment of a piece of software. These bugs may be difficult to activate and even more difficult to find. This paper reports on an extension to traditional mutation testing that enables testing specifically for environmental bugs involving integer arithmetic. This method is both simple and effective, and provides some insight into other possible extensions of the mutation-testing methodology that can be used to expose environmental bugs.     

Statistical methods for testing the conforms nee of products to user performance standards

Abstract

User performance testing is an essential part of the development of information technology products. Usability engineering techniques and also methods included in the new International Organization for Standardization (ISO) draft standard for VDTs involve testing products against a standard or benchmark. This paper highlights some of the statistical and methodological problems involved in conformance testing and outlines appropriate procedures. These include the use of sequential tests which, in comparison with conventional tests, can achieve a radical reduction in the number of subjects required.     

Statistical methods for testing the conforms nee of products to user performance standards

User performance testing is an essential part of the development of information technology products. Usability engineering techniques and also methods included in the new International Organization for Standardization (ISO) draft standard for VDTs involve testing products against a standard or benchmark. This paper highlights some of the statistical and methodological problems involved in conformance testing and outlines appropriate procedures. These include the use of sequential tests which, in comparison with conventional tests, can achieve a radical reduction in the number of subjects required.     

Recommender systems: from algorithms to user experience

Since their introduction in the early 1990??s, automated recommender systems have revolutionized the marketing and delivery of commerce and content by providing personalized recommendations and predictions over a variety of large and complex product offerings. In this article, we review the key advances in collaborative filtering recommender systems, focusing on the evolution from research concentrated purely on algorithms to research concentrated on the rich set of questions around the user experience with the recommender. We show through examples that the embedding of the algorithm in the user experience dramatically affects the value to the user of the recommender. We argue that evaluating the user experience of a recommender requires a broader set of measures than have been commonly used, and suggest additional measures that have proven effective. Based on our analysis of the state of the field, we identify the most important open research problems, and outline key challenges slowing the advance of the state of the art, and in some cases limiting the relevance of research to real-world applications.     

The influence of product aesthetics and user state in usability testing

An empirical study examined the effects of two influencing factors in usability tests on user performance and usability ratings. Product aesthetics (high vs. low) as the main independent factor and prior usage event (positive vs. negative) as a subsidiary independent factor were varied in a between-participants design. 60 participants took part in the experiment, completing a number of typical tasks of mobile phone users. The results showed that increased product aesthetics had a positive effect on perceived usability and led to longer task completion times. After a negative usage event had been experimentally induced, usability ratings dropped as expected but user performance on subsequent tasks remained unaffected. An important implication of the study is that the aesthetic properties of a product may have multiple effects that go beyond perceived product attractiveness.     

Customization of user interfaces to reduce errors and enhance user acceptance

Customization is assumed to reduce error and increase user acceptance in the human–machine relation. Reconfiguration gives the operator the option to customize a user interface according to his or her own preferences. An experimental study with 72 computer science students using a simulated process control task was conducted. The reconfiguration group (RG) interactively reconfigured their user interfaces and used the reconfigured user interface in the subsequent test whereas the control group (CG) used a default user interface. Results showed significantly lower error rates and higher acceptance of the RG compared to the CG while there were no significant differences between the groups regarding situation awareness and mental workload. Reconfiguration seems to be promising and therefore warrants further exploration.     

Fault detection effectiveness of mutation and data flow testing

We report results from an experiment to compare the fault detection effectiveness of mutation, its variants and the all-uses data flow criteria. Adequate test sets were generated randomly, as opposed to by human testers as in some previous studies. We view our results in the light of those from earlier studies comparing mutation with path-oriented testing strategies. We identify and discuss factors that one might consider while evaluating an adequacy criterion for use in practice. Results from our experiments strengthen a hypothesis that an adequacy criterion based on one of the two variants of mutation has superior fault detection effectiveness than that of the all-uses criterion.     

Effectiveness of user testing and heuristic evaluation as a function of performance classification

For different levels of user performance, different types of information are processed and users will make different types of errors. Based on the error's immediate cause and the information being processed, usability problems can be classified into three categories. They are usability problems associated with skill-based, rule-based and knowledge-based levels of performance. In this paper, a user interface for a Web-based software program was evaluated with two usability evaluation methods, user testing and heuristic evaluation. The experiment discovered that the heuristic evaluation with human factor experts is more effective than user testing in identifying usability problems associated with skill-based and rule-based levels of performance. User testing is more effective than heuristic evaluation in finding usability problems associated with the knowledge-based level of performance. The practical application of this research is also discussed in the paper.     

A knowledge-based approach to user interface adaptation from preferences and for special needs

Moving between devices is omnipresent, but not for people with disabilities or those who require specific accessibility options. Setting up assistive technologies or finding settings to overcome a certain barrier can be a demanding task for people without technical skills. Context-sensitive adaptive user interfaces are advancing, although migrating access features from one device to another is very rarely addressed. In this paper, we describe the knowledge-based component of the Global Public Inclusive Infrastructure that infers how a device shall be best configured at the operating system layer, the application layer and the web layer to meet the requirements of a user including possible special needs or disabilities. In this regard, a mechanism to detect and resolve conflicting accessibility policies as well as recommending preference substitutes is a main requirement, as elaborated in this paper. As the proposed system emulates decision-making of accessibility experts, we validated the automatic deduced configurations against manual configurations of ten accessibility experts. The assessment result shows that the average matching score of the developed system is high. Thus, the proposed system can be considered capable of making precise decisions towards personalizing user interfaces based on user needs and preferences.     

Detection of all single and multiple stuck-at faults in combinational digital circuits using index vector testing

A new method for the testing of combinational digital circuits is presented. The method is based on the concept of the ‘index vector’ of a switching function (Gupta 1987), and represents an extension of syndrome testing. A large percentage of syndrome untestable faults are found to be index vector testable. An approach to testing index vector untestable circuits that relies only on the function realized by the circuit and is independent of the circuit topology is presented. The method can be used for the detection of both single and multiple stuck-at faults in a combinational circuit.     

Effectiveness for detecting faults within and outside the scope of testing techniques: an independent replication

The verification and validation activity plays a fundamental role in improving software quality. Determining which the most effective techniques for carrying out this activity are has been an aspiration of experimental software engineering researchers for years. This paper reports a controlled experiment evaluating the effectiveness of two unit testing techniques (the functional testing technique known as equivalence partitioning (EP) and the control-flow structural testing technique known as branch testing (BT)). This experiment is a literal replication of Juristo et al. (2013). Both experiments serve the purpose of determining whether the effectiveness of BT and EP varies depending on whether or not the faults are visible for the technique (InScope or OutScope, respectively). We have used the materials, design and procedures of the original experiment, but in order to adapt the experiment to the context we have: (1) reduced the number of studied techniques from 3 to 2; (2) assigned subjects to experimental groups by means of stratified randomization to balance the influence of programming experience; (3) localized the experimental materials and (4) adapted the training duration. We ran the replication at the Escuela Politécnica del Ejército Sede Latacunga (ESPEL) as part of a software verification & validation course. The experimental subjects were 23 master’s degree students. EP is more effective than BT at detecting InScope faults. The session/program and group variables are found to have significant effects. BT is more effective than EP at detecting OutScope faults. The session/program and group variables have no effect in this case. The results of the replication and the original experiment are similar with respect to testing techniques. There are some inconsistencies with respect to the group factor. They can be explained by small sample effects. The results for the session/program factor are inconsistent for InScope faults. We believe that these differences are due to a combination of the fatigue effect and a technique x program interaction. Although we were able to reproduce the main effects, the changes to the design of the original experiment make it impossible to identify the causes of the discrepancies for sure. We believe that further replications closely resembling the original experiment should be conducted to improve our understanding of the phenomena under study.     

The RE-UWA approach to recover user centered conceptual models from Web applications

Large-scale Web Applications, especially those intended to publish contents and provide information to its users, are by nature subject to continuous and fast changes. This often means fast obsolescence of the design documentation and a lot of effort required to comprehend the application when performing maintenance and evolution tasks. This paper presents a reverse engineering approach for Web Applications enabling the semi-automatic recovery of user-centered conceptual models describing, from a user perspective, key aspects, such as the delivered contents and navigational paths. The abstracted models are formalized according to the Ubiquitous Web Applications (UWA) design methodology, but any other design method for Web Applications could be used instead. The paper describes the recovery process, a tool developed to support the process, and the results from a case study conducted to validate the approach on a set of real world Web Applications.     

A heuristic approach to discovering user correlations from organized social stream data

Multimedia Tools and Applications - Recently, with the widespread popularity of SNS (Social Network Service), such as Twitter, Facebook, people are increasingly accustomed to sharing feeling,...     

The influence of design aesthetics in usability testing: Effects on user performance and perceived usability

This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.