WAS: A weighted attribute-based strategy for cluster test selection

In past decades, many techniques have been proposed to generate and execute test cases automatically. However, when a test oracle does not exist, execution results have to be examined manually. With increasing functionality and complexity of today's software, this process can be extremely time-consuming and mistake-prone. A CTS-based (cluster test selection) strategy provides a feasible solution to mitigate such deficiency by examining the execution results only with respect to a small number of selected test cases. It groups test cases with similar execution profiles into the same cluster and selects them from each cluster. Some well-known CTS-based strategies are one per cluster, n (a predefined value which is greater than 1) per cluster, adaptive sampling, and execution-spectra-based sampling (ESBS). The ultimate goal is to reduce testing cost by quickly identifying the executions that are likely to fail. However, improperly grouping the test cases will significantly diminish the effectiveness of these strategies (by examining results of more successful executions and fewer failed executions). To overcome this problem, we propose a weighted attribute-based strategy (WAS). Instead of clustering test cases based on the similarity of their execution profiles only once like the aforementioned CTS-based strategies, WAS will conduct more than one iteration of clustering using weighted execution profiles by also considering the suspiciousness of each program element (statement, basic block, decision, etc.), where the suspiciousness in terms of the likelihood of containing bugs can be computed by using various software fault localization techniques. Case studies using seven programs (make, ant, sed, flex, grep, gzip, and space) and four CTS-based strategies (one per cluster sampling, n per cluster sampling, adaptive sampling, and ESBS) were conducted to evaluate the effectiveness of WAS on 184 faulty versions containing either single or multiple bugs. Experimental results suggest that the proposed WAS strategy outperforms other four CTS-based strategies with respect to both recall and precision such that output verification is focused more strongly on failed executions.