跨项目缺陷预测中训练数据选择方法

跨项目缺陷预测（CPDP）利用来自其他项目的缺陷数据预测目标项目的缺陷情况,为解决以往缺陷预测方法面临的训练数据受限问题提供了一个新的视角。训练数据的质量将直接影响跨项目缺陷预测模型的性能,因此,需尽可能选择与目标项目更相似的数据用于模型的训练。利用PROMISE提供的34个公开数据集,从训练数据选择方面,分析了四种典型的相似性度量方法对跨项目预测结果的影响以及各种方法之间的差异。研究结果表明：使用不同的相似性度量方法选出的训练数据质量不同,其中余弦相似性与相关系数两种方法效果更好,且最大改进比例达到6.7%;同时,根据目标项目的缺陷率,发现余弦相似性更适合于缺陷率高于0.25的项目。     

Studying just-in-time defect prediction using cross-project models

Unlike traditional defect prediction models that identify defect-prone modules, Just-In-Time (JIT) defect prediction models identify defect-inducing changes. As such, JIT defect models can provide earlier feedback for developers, while design decisions are still fresh in their minds. Unfortunately, similar to traditional defect models, JIT models require a large amount of training data, which is not available when projects are in initial development phases. To address this limitation in traditional defect prediction, prior work has proposed cross-project models, i.e., models learned from other projects with sufficient history. However, cross-project models have not yet been explored in the context of JIT prediction. Therefore, in this study, we empirically evaluate the performance of JIT models in a cross-project context. Through an empirical study on 11 open source projects, we find that while JIT models rarely perform well in a cross-project context, their performance tends to improve when using approaches that: (1) select models trained using other projects that are similar to the testing project, (2) combine the data of several other projects to produce a larger pool of training data, and (3) combine the models of several other projects to produce an ensemble model. Our findings empirically confirm that JIT models learned using other projects are a viable solution for projects with limited historical data. However, JIT models tend to perform best in a cross-project context when the data used to learn them are carefully selected.     

A transfer cost-sensitive boosting approach for cross-project defect prediction

Software defect prediction has been regarded as one of the crucial tasks to improve software quality by effectively allocating valuable resources to fault-prone modules. It is necessary to have a sufficient set of historical data for building a predictor. Without a set of sufficient historical data within a company, cross-project defect prediction (CPDP) can be employed where data from other companies are used to build predictors. In such cases, a transfer learning technique, which extracts common knowledge from source projects and transfers it to a target project, can be used to enhance the prediction performance. There exists the class imbalance problem, which causes difficulties for the learner to predict defects. The main impacts of imbalanced data under cross-project settings have not been investigated in depth. We propose a transfer cost-sensitive boosting method that considers both knowledge transfer and class imbalance for CPDP when given a small amount of labeled target data. The proposed approach performs boosting that assigns weights to the training instances with consideration of both distributional characteristics and the class imbalance. Through comparative experiments with the transfer learning and the class imbalance learning techniques, we show that the proposed model provides significantly higher defect detection accuracy while retaining better overall performance. As a result, a combination of transfer learning and class imbalance learning is highly effective for improving the prediction performance under cross-project settings. The proposed approach will help to design an effective prediction model for CPDP. The improved defect prediction performance could help to direct software quality assurance activities and reduce costs. Consequently, the quality of software can be managed effectively.     

Effort-aware cross-project just-in-time defect prediction framework for mobile apps

As the boom of mobile devices, Android mobile apps play an irreplaceable roles in people’s daily life, which have the characteristics of frequent updates involving in many code commits to meet new requirements. Just-in-Time (JIT) defect prediction aims to identify whether the commit instances will bring defects into the new release of apps and provides immediate feedback to developers, which is more suitable to mobile apps. As the within-app defect prediction needs sufficient historical data to label the commit instances, which is inadequate in practice, one alternative method is to use the cross-project model. In this work, we propose a novel method, called KAL, for cross-project JIT defect prediction task in the context of Android mobile apps. More specifically, KAL first transforms the commit instances into a high-dimensional feature space using kernel-based principal component analysis technique to obtain the representative features. Then, the adversarial learning technique is used to extract the common feature embedding for the model building. We conduct experiments on 14 Android mobile apps and employ four effort-aware indicators for performance evaluation. The results on 182 cross-project pairs demonstrate that our proposed KAL method obtains better performance than 20 comparative methods.     

Value-cognitive boosting with a support vector machine for cross-project defect prediction

It is well-known that software defect prediction is one of the most important tasks for software quality improvement. The use of defect predictors allows test engineers to focus on defective modules. Thereby testing resources can be allocated effectively and the quality assurance costs can be reduced. For within-project defect prediction (WPDP), there should be sufficient data within a company to train any prediction model. Without such local data, cross-project defect prediction (CPDP) is feasible since it uses data collected from similar projects in other companies. Software defect datasets have the class imbalance problem increasing the difficulty for the learner to predict defects. In addition, the impact of imbalanced data on the real performance of models can be hidden by the performance measures chosen. We investigate if the class imbalance learning can be beneficial for CPDP. In our approach, the asymmetric misclassification cost and the similarity weights obtained from distributional characteristics are closely associated to guide the appropriate resampling mechanism. We performed the effect size A-statistics test to evaluate the magnitude of the improvement. For the statistical significant test, we used Wilcoxon rank-sum test. The experimental results show that our approach can provide higher prediction performance than both the existing CPDP technique and the existing class imbalance technique.     

Combined classifier for cross-project defect prediction: an extended empirical study

To facilitate developers in effective allocation of their testing and debugging efforts, many software defect prediction techniques have been proposed in the literature. These techniques can be used to predict classes that are more likely to be buggy based on the past history of classes, methods, or certain other code elements. These techniques are effective provided that a sufficient amount of data is available to train a prediction model. However, sufficient training data are rarely available for new software projects. To resolve this problem, cross-project defect prediction, which transfers a prediction model trained using data from one project to another, was proposed and is regarded as a new challenge in the area of defect prediction. Thus far, only a few cross-project defect prediction techniques have been proposed. To advance the state of the art, in this study, we investigated seven composite algorithms that integrate multiple machine learning classifiers to improve cross-project defect prediction. To evaluate the performance of the composite algorithms, we performed experiments on 10 open-source software systems from the PROMISE repository, which contain a total of 5,305 instances labeled as defective or clean. We compared the composite algorithms with the combined defect predictor where logistic regression is used as the meta classification algorithm (CODEP _Logistic ), which is the most recent cross-project defect prediction algorithm in terms of two standard evaluation metrics: cost effectiveness and F-measure. Our experimental results show that several algorithms outperform CODEP _Logistic : Maximum voting shows the best performance in terms of F-measure and its average F-measure is superior to that of CODEP _Logistic by 36.88%. Bootstrap aggregation (Bagging_J48) shows the best performance in terms of cost effectiveness and its average cost effectiveness is superior to that of CODEP _Logistic by 15.34%.     

基于Box-Cox转换的集成跨项目软件缺陷预测方法

软件缺陷预测通过预先识别出被测项目内的潜在缺陷程序模块,可以优化测试资源的分配并提高软件产品的质量。论文对跨项目缺陷预测问题展开了深入研究,在源项目实例选择时,考虑了三种不同的实例相似度计算方法,并发现这些方法的缺陷预测结果存在多样性,因此提出了一种基于Box-Cox转换的集成跨项目软件缺陷预测方法BCEL,具体来说,首先基于不同的实例相似度计算方法,从候选集中选出不同的训练集,随后针对这些数据集,进行针对性的Box-Cox转化,并借助特定分类方法构造出不同的基分类器,最后将这三个基分类器进行有效集成。基于实际项目的数据集,验证了BCEL方法的有效性,并深入分析了BCEL方法内的影响因素对缺陷预测性能的影响。     

An empirical study of factors affecting cross-project aging-related bug prediction with TLAP

Software aging is a phenomenon in which long-running software systems show an increasing failure rate and/or progressive performance degradation. Due to their nature, Aging-Related Bugs (ARBs) are hard to discover during software testing and are also challenging to reproduce. Therefore, automatically predicting ARBs before software release can help developers reduce ARB impact or avoid ARBs. Many bug prediction approaches have been proposed, and most of them show effectiveness in within-project prediction settings. However, due to the low presence and reproducing difficulty of ARBs, it is usually hard to collect sufficient training data to build an accurate prediction model. A recent work proposed a method named Transfer Learning based Aging-related bug Prediction (TLAP) for performing cross-project ARB prediction. Although this method considerably improves cross-project ARB prediction performance, it has been observed that its prediction result is affected by several key factors, such as the normalization methods, kernel functions, and machine learning classifiers. Therefore, this paper presents the first empirical study to examine the impact of these factors on the effectiveness of cross-project ARB prediction in terms of single-factor pattern, bigram pattern, and triplet pattern and validates the results with the Scott-Knott test technique. We find that kernel functions and classifiers are key factors affecting the effectiveness of cross-project ARB prediction, while normalization methods do not show statistical influence. In addition, the order of values in three single-factor patterns is maintained in three bigram patterns and one triplet pattern to a large extent. Similarly, the order of values in the three bigram patterns is also maintained in the triplet pattern.

     

基于概率的无监督缺陷预测方法

软件缺陷预测能够提高软件开发和测试的效率,保障软件质量。无监督缺陷预测方法具有不需要标签数据的特点,从而能够快速应用于工程实践中。提出了基于概率的无监督缺陷预测方法—PCLA,将度量元值与阈值的差值映射为概率,使用概率评估类存在缺陷的可能性,然后再通过聚类和标记来完成缺陷预测,以解决现有无监督方法直接根据阈值判断时对阈值比较敏感而引起的信息丢失问题。将PCLA方法应用在NetGen和Relink两组数据集,共7个软件项目上,实验结果表明PCLA方法在查全率、查准率、F-measure上相对现有无监督方法分别平均提升4.1%、2.52%、3.14%。     

An effort prediction framework for software defect correction

This article tackles the problem of predicting effort (in person–hours) required to fix a software defect posted on an Issue Tracking System. The proposed method is inspired by the Nearest Neighbour Approach presented by the pioneering work of Weiss et al. (2007) [1]. We propose four enhancements to Weiss et al. (2007) [1]: Data Enrichment, Majority Voting, Adaptive Threshold and Binary Clustering. Data Enrichment infuses additional issue information into the similarity-scoring procedure, aiming to increase the accuracy of similarity scores. Majority Voting exploits the fact that many of the similar historical issues have repeating effort values, which are close to the actual. Adaptive Threshold automatically adjusts the similarity threshold to ensure that we obtain only the most similar matches. We use Binary Clustering if the similarity scores are very low, which might result in misleading predictions. This uses common properties of issues to form clusters (independent of the similarity scores) which are then used to produce the predictions. Numerical results are presented showing a noticeable improvement over the method proposed in Weiss et al. (2007) [1].     

基于缺陷分类和缺陷预测的软件缺陷预防

为降低软件缺陷率,对现有的缺陷预测模型进行了优化,同时引入正交缺陷分类方法,并对该方法加以改进,使其能够支持缺陷的原因分析,将缺陷预测与改进的正交缺陷分类方法结合起来,形成一套软件缺陷预防流程并应用在实际项目中.实验结果表明,该成果可以在软件生命周期的各个阶段有效预防缺陷,最大限度地提高软件质量.     

An empirical study on software defect prediction with a simplified metric set

ContextSoftware defect prediction plays a crucial role in estimating the most defect-prone components of software, and a large number of studies have pursued improving prediction accuracy within a project or across projects. However, the rules for making an appropriate decision between within- and cross-project defect prediction when available historical data are insufficient remain unclear.ObjectiveThe objective of this work is to validate the feasibility of the predictor built with a simplified metric set for software defect prediction in different scenarios, and to investigate practical guidelines for the choice of training data, classifier and metric subset of a given project.MethodFirst, based on six typical classifiers, three types of predictors using the size of software metric set were constructed in three scenarios. Then, we validated the acceptable performance of the predictor based on Top-k metrics in terms of statistical methods. Finally, we attempted to minimize the Top-k metric subset by removing redundant metrics, and we tested the stability of such a minimum metric subset with one-way ANOVA tests.ResultsThe study has been conducted on 34 releases of 10 open-source projects available at the PROMISE repository. The findings indicate that the predictors built with either Top-k metrics or the minimum metric subset can provide an acceptable result compared with benchmark predictors. The guideline for choosing a suitable simplified metric set in different scenarios is presented in Table 12.ConclusionThe experimental results indicate that (1) the choice of training data for defect prediction should depend on the specific requirement of accuracy; (2) the predictor built with a simplified metric set works well and is very useful in case limited resources are supplied; (3) simple classifiers (e.g., Naïve Bayes) also tend to perform well when using a simplified metric set for defect prediction; and (4) in several cases, the minimum metric subset can be identified to facilitate the procedure of general defect prediction with acceptable loss of prediction precision in practice.     

An empirical investigation on the challenges of creating custom static analysis rules for defect localization

Software Quality Journal - Custom static analysis rules, i.e., rules specific for one or more applications, have been successfully applied to perform corrective and preventive software maintenance....     

基于复杂性的软件缺陷预测

为解决软件缺陷预测问题引入了最小二乘支持向量机算法(LS-SVM),加速了超参数的选择过程,给出了逐个加入新的样本用以模型校正的快捷方法,以软件复杂性度量为线索,建立了基于FLS-SVM的软件缺陷预测模型。通过具体实例阐明了模型的执行过程及小样本情况下比神经网络更为出色的预测能力,并根据回归方程指出了对软件缺陷影响显著的复杂性度量。     

基于PSO-BP的软件缺陷预测模型

软件缺陷检测旨在自动检测程序模块中是否包含缺陷,从而加速软件测试过程,提高软件系统的质量。针对传统软件缺陷预测模型被限制在一定的应用范围而影响其预测的准确性和适用性,提出了一种基于PSO-BP软件缺陷预测模型。该模型运用粒子群优化算法优化BP神经网络的权值和阈值,采用交叉验证的方式进行实验,并与传统的机器学习方法J48和BP神经网络等方法进行了比较。实验结果表明提出的方法具有较高的预测准确性。     

软件缺陷集成预测模型研究

利用单一分类器构造的缺陷预测模型已经遇到了性能瓶颈, 而集成分类器相比单一分类器往往具有显著的性能优势。以构造高效的集成缺陷预测模型为出发点, 比较了七种不同类型集成分类器的算法和特点。在14个基准数据集上的实验显示, 部分集成预测模型的性能优于基于朴素贝叶斯的单一预测模型。其中, 基于投票的集成分类框架具有最优的预测性能以及统计学意义上的性能优势显著性, 随机森林算法次之。Stacking集成框架也具有较强的泛化能力。     

基于不平衡数据集的软件缺陷预测

数据的不平衡性是软件缺陷预测研究中一个严峻且无法规避的问题,为了解决这一问题,本文提出一种利用分布函数合成新样本的过抽样和随机向下抽样相结合的算法。该算法首先对降维后的主成分进行分布函数拟合,然后利用分布函数生成随机数,并对生成的随机数进行筛选,最后与随机向下抽样相结合。实验所用数据取自NASA MDP数据集,并与经典的SMOTE 向下抽样方法进行对比,从G-mean和F-measure值可以看出前者的预测结果明显优于后者,预测精度更高。     

基于生命周期的软件缺陷预测技术

为保证软件可靠性和软件质量,在基于软件开发周期的基础上,提出了一种利用PCA-BP模糊神经网络的软件缺陷预计方法.针对影响软件可靠性的各种因素,依据相关的标准,结合工程实践,选取了影响软件可靠性的度量元.收集了实际工程中的一类飞行控制软件的度量数据,利用提出的模型进行缺陷预测,并将预测结果与传统的BP神经网络模型计算的结果进行了对比.对比结果表明,与基于BP神经网络的预测方法相比较,结合了主成分分析方法的PCA-BP神经网络预测方法具有更快的收敛速度和更高的预测准确度.     

Software defect association mining and defect correction effort prediction

Much current software defect prediction work focuses on the number of defects remaining in a software system. In this paper, we present association rule mining based methods to predict defect associations and defect correction effort. This is to help developers detect software defects and assist project managers in allocating testing resources more effectively. We applied the proposed methods to the SEL defect data consisting of more than 200 projects over more than 15 years. The results show that, for defect association prediction, the accuracy is very high and the false-negative rate is very low. Likewise, for the defect correction effort prediction, the accuracy for both defect isolation effort prediction and defect correction effort prediction are also high. We compared the defect correction effort prediction method with other types of methods - PART, C4.5, and Naive Bayes - and show that accuracy has been improved by at least 23 percent. We also evaluated the impact of support and confidence levels on prediction accuracy, false-negative rate, false-positive rate, and the number of rules. We found that higher support and confidence levels may not result in higher prediction accuracy, and a sufficient number of rules is a precondition for high prediction accuracy.     

改进PSO-ISVM算法的软件缺陷预测

提出基于改进的粒子群优化支持向量机方法（PSO-ISVM）的测控软件缺陷预测方法。通过引入代价惩罚系数,定义粒子群优化算法中的适应度函数,利用最小化适应度函数值作为优化目标,排除大量的冗余干扰信息,提高对测控软件有缺陷模块的预测准确度,寻找支持向量机的最优参数。通过仿真实例分析测控软件有效性,并与常用缺陷预测方法进行比较,表明该模型能加快软件缺陷预测速度和提高对有缺陷模块的预测准确度。