A new fuzzy rule based algorithm for estimating software faults in early phase of development

Estimation of reliability and the number of faults present in software in its early development phase, i.e., requirement analysis or design phase is very beneficial for developing reliable software with optimal cost. Software reliability prediction in early phase of development is highly desirable to the stake holders, software developers, managers and end users. Since, the failure data are unavailable in early phase of software development, different reliability relevant software metrics and similar project data are used to develop models for early software fault prediction. The proposed model uses the linguistic values of software metrics in fuzzy inference system to predict the total number of faults present in software in its requirement analysis phase. Considering specific target reliability, weightage of each input software metrics and size of software, an algorithm has been proposed here for developing general fuzzy rule base. For model validation of the proposed model, 20 real software project data have been used here. The linguistic values from four software metrics related to requirement analysis phase have been considered as model inputs. The performance of the proposed model has been compared with two existing early software fault prediction models.     

A fuzzy rule-based generation algorithm in interval type-2 fuzzy logic system for fault prediction in the early phase of software development

Reliability, a measure of software, deals in total number of faults count up to a certain period of time. The present study aims at estimating the total number of software faults during the early phase of software life cycle. Such estimation helps in producing more reliable software as there may be a scope to take necessary corrective actions for improving the reliability within optimum time and cost by the software developers. The proposed interval type-2 fuzzy logic-based model considers reliability-relevant software metric and earlier project data as model inputs. Type-2 fuzzy sets have been used to reduce uncertainties in the vague linguistic values of the software metrics. A rule formation algorithm has been developed to overcome inconsistency in the consequent parts of large number of rules. Twenty-six software project data help to validate the model, and a comparison has been provided to analyse the proposed model’s performance.     

Estimating software readiness using predictive models

In this study, defect tracking is used as a proxy method to predict software readiness. The number of remaining defects in an application under development is one of the most important factors that allow one to decide if a piece of software is ready to be released. By comparing predicted number of faults and number of faults discovered in testing, software manager can decide whether the software is likely ready to be released or not.The predictive model developed in this research can predict: (i) the number of faults (defects) likely to exist, (ii) the estimated number of code changes required to correct a fault and (iii) the estimated amount of time (in minutes) needed to make the changes in respective classes of the application. The model uses product metrics as independent variables to do predictions. These metrics are selected depending on the nature of source code with regards to architecture layers, types of faults and contribution factors of these metrics. The use of neural network model with genetic training strategy is introduced to improve prediction results for estimating software readiness in this study. This genetic-net combines a genetic algorithm with a statistical estimator to produce a model which also shows the usefulness of inputs.The model is divided into three parts: (1) prediction model for presentation logic tier (2) prediction model for business tier and (3) prediction model for data access tier. Existing object-oriented metrics and complexity software metrics are used in the business tier prediction model. New sets of metrics have been proposed for the presentation logic tier and data access tier. These metrics are validated using data extracted from real world applications. The trained models can be used as tools to assist software mangers in making software release decisions.     

A fuzzy logic based approach for phase-wise software defects prediction using software metrics

ContextThe software defect prediction during software development has recently attracted the attention of many researchers. The software defect density indicator prediction in each phase of software development life cycle (SDLC) is desirable for developing a reliable software product. Software defect prediction at the end of testing phase may not be more beneficial because the changes need to be performed in the previous phases of SDLC may require huge amount of money and effort to be spent in order to achieve target software quality. Therefore, phase-wise software defect density indicator prediction model is of great importance.ObjectiveIn this paper, a fuzzy logic based phase-wise software defect prediction model is proposed using the top most reliability relevant metrics of the each phase of the SDLC.MethodIn the proposed model, defect density indicator in requirement analysis, design, coding and testing phase is predicted using nine software metrics of these four phases. The defect density indicator metric predicted at the end of the each phase is also taken as an input to the next phase. Software metrics are assessed in linguistic terms and fuzzy inference system has been employed to develop the model.ResultsThe predictive accuracy of the proposed model is validated using twenty real software project data. Validation results are satisfactory. Measures based on the mean magnitude of relative error and balanced mean magnitude of relative error decrease significantly as the software project size increases.ConclusionIn this paper, a fuzzy logic based model is proposed for predicting software defect density indicator at each phase of the SDLC. The predicted defects of twenty different software projects are found very near to the actual defects detected during testing. The predicted defect density indicators are very helpful to analyze the defect severity in different artifacts of SDLC of a software project.     

A critique of software defect prediction models

Many organizations want to predict the number of defects (faults) in software systems, before they are deployed, to gauge the likely delivered quality and maintenance effort. To help in this numerous software metrics and statistical models have been developed, with a correspondingly large literature. We provide a critical review of this literature and the state-of-the-art. Most of the wide range of prediction models use size and complexity metrics to predict defects. Others are based on testing data, the “quality” of the development process, or take a multivariate approach. The authors of the models have often made heroic contributions to a subject otherwise bereft of empirical studies. However, there are a number of serious theoretical and practical problems in many studies. The models are weak because of their inability to cope with the, as yet, unknown relationship between defects and failures. There are fundamental statistical and data quality problems that undermine model validity. More significantly many prediction models tend to model only part of the underlying problem and seriously misspecify it. To illustrate these points the Goldilock's Conjecture, that there is an optimum module size, is used to show the considerable problems inherent in current defect prediction approaches. Careful and considered analysis of past and new results shows that the conjecture lacks support and that some models are misleading. We recommend holistic models for software defect prediction, using Bayesian belief networks, as alternative approaches to the single-issue models used at present. We also argue for research into a theory of “software decomposition” in order to test hypotheses about defect introduction and help construct a better science of software engineering     

基于贝叶斯网络的克隆代码有害性预测

在软件开发过程中,程序员的复制、粘贴活动会产生大量的克隆代码,而那些发生不一致变化的克隆代码往往对程序是有害的。为了解决该问题,有效地发现程序中的有害克隆代码,提出一种基于贝叶斯网络的克隆有害性预测方法。首先,结合软件缺陷研究领域与克隆演化领域的相关研究成果,提出了两大类表征克隆代码信息的特征,分别是静态特征和演化特征;其次,通过贝叶斯网络核心算法来构建克隆有害性预测模型;最后,预测有害克隆代码发生的可能性。在5款C语言开源软件共99个版本上对克隆有害性预测模型的性能进行评估,实验结果表明该方法能够有效地实现对克隆代码有害性的预测,降低有害克隆代码对软件的威胁,提高软件质量。     

Prediction of faults-slip-through in large software projects: an empirical evaluation

A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization–based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.     

A systematic review of software fault prediction studies

This paper provides a systematic review of previous software fault prediction studies with a specific focus on metrics, methods, and datasets. The review uses 74 software fault prediction papers in 11 journals and several conference proceedings. According to the review results, the usage percentage of public datasets increased significantly and the usage percentage of machine learning algorithms increased slightly since 2005. In addition, method-level metrics are still the most dominant metrics in fault prediction research area and machine learning algorithms are still the most popular methods for fault prediction. Researchers working on software fault prediction area should continue to use public datasets and machine learning algorithms to build better fault predictors. The usage percentage of class-level is beyond acceptable levels and they should be used much more than they are now in order to predict the faults earlier in design phase of software life cycle.     

A fuzzy-filtered neuro-fuzzy framework for software fault prediction for inter-version and inter-project evaluation

Fault Prediction is the most required measure to estimate the software quality and reliability. Several methods, measures, aspects and testing methodologies are available to evaluate the software fault. In this paper, a fuzzy-filtered neuro-fuzzy framework is introduced to predict the software faults for internal and external software projects. The suggested framework is split into three primary phases. At the earlier phase, the effective metrics or measures are identified, which can derive the accurate decision on prediction of software fault. In this phase, the composite analytical observation of each software attribute is calculated using Information Gain and Gain Ratio measures. In the second phase, these fuzzy rules are applied on these measures for selection of effective and high-impact features. In the last phase, the Neuro-fuzzy classifier is applied on fuzzy-filtered training and testing sets. The proposed framework is applied to identify the software faults based on inter-version and inter-project evaluation. In this framework, the earlier projects or project-versions are considered as training sets and the new projects or versions are taken as testing sets. The experimentation is conducted on nine open source projects taken from PROMISE repository as well as on PDE and JDT projects. The approximation is applied on internal version-specific fault prediction and external software projects evaluation. The comparative analysis is performed against Decision Tree, Random Tree, Random Forest, Naive Bayes and Multilevel Perceptron classifiers. This prediction result signifies that the proposed framework has gained the higher accuracy, lesser error rate and significant AUC and GM for inter-project and inter-version evaluations.     

An analysis of static metrics and faults in C software

In this empirical study, we evaluate the extent to which a set of software measures are correlated with the number of faults and the total estimated repair effort for a large software system. The measures we use are basic counts reflecting program size and structure and metrics proposed by McCabe and Halstead. The effect of program size has a major influence on these metrics, and we present a suitable method of adjusting the metrics for size. In modeling faults or repair effort as a function of one variable, a number of measures individually explain approximately one-quarter of the variation observed in the fault data. No one measure does significantly better than size in explaining the variation in faults found across software units, and thus multiple variable models are necessary to find metrics of importance in addition to program size. The “best” multivariate model explains approximately one-half the variation in the fault data. The metrics included in this model (in addition to size) are: the ratio of block comments to total lines of code, the number of decisions per function, and the relative vocabulary of program variables and operators. These metrics have potential for future use in the quality control of software.     

Measuring behavioral dependency for improving change-proneness prediction in UML-based design models

Several studies have explored the relationship between the metrics of the object-oriented software and the change-proneness of the classes. This knowledge can be used to help decision-making among design alternatives or assess software quality such as maintainability. Despite the increasing use of complex inheritance relationships and polymorphism in object-oriented software, there has been less emphasis on developing metrics that capture the aspect of dynamic behavior. Considering dynamic behavior metrics in conjunction with existing metrics may go a long way toward obtaining more accurate predictions of change-proneness. To address this need, we provide the behavioral dependency measure using structural and behavioral information taken from UML 2.0 design models. Model-based change-proneness prediction helps to make high-quality software by exploiting design models from the earlier phase of the software development process. The behavioral dependency measure has been evaluated on a multi-version medium size open-source project called JFlex. The results obtained show that the proposed measure is a useful indicator and can be complementary to existing object-oriented metrics for improving the accuracy of change-proneness prediction when the system contains high degree of inheritance relationships and polymorphism.     

Analyzing software measurement data with clustering techniques

For software quality estimation, software development practitioners typically construct quality-classification or fault prediction models using software metrics and fault data from a previous system release or a similar software project. Engineers then use these models to predict the fault proneness of software modules in development. Software quality estimation using supervised-learning approaches is difficult without software fault measurement data from similar projects or earlier system releases. Cluster analysis with expert input is a viable unsupervised-learning solution for predicting software modules' fault proneness and potential noisy modules. Data analysts and software engineering experts can collaborate more closely to construct and collect more informative software metrics.     

Towards an ensemble based system for predicting the number of software faults

Software fault prediction using different techniques has been done by various researchers previously. It is observed that the performance of these techniques varied from dataset to dataset, which make them inconsistent for fault prediction in the unknown software project. On the other hand, use of ensemble method for software fault prediction can be very effective, as it takes the advantage of different techniques for the given dataset to come up with better prediction results compared to individual technique. Many works are available on binary class software fault prediction (faulty or non-faulty prediction) using ensemble methods, but the use of ensemble methods for the prediction of number of faults has not been explored so far. The objective of this work is to present a system using the ensemble of various learning techniques for predicting the number of faults in given software modules. We present a heterogeneous ensemble method for the prediction of number of faults and use a linear combination rule and a non-linear combination rule based approaches for the ensemble. The study is designed and conducted for different software fault datasets accumulated from the publicly available data repositories. The results indicate that the presented system predicted number of faults with higher accuracy. The results are consistent across all the datasets. We also use prediction at level l (Pred(l)), and measure of completeness to evaluate the results. Pred(l) shows the number of modules in a dataset for which average relative error value is less than or equal to a threshold value l. The results of prediction at level l analysis and measure of completeness analysis have also confirmed the effectiveness of the presented system for the prediction of number of faults. Compared to the single fault prediction technique, ensemble methods produced improved performance for the prediction of number of software faults. Main impact of this work is to allow better utilization of testing resources helping in early and quick identification of most of the faults in the software system.     

贝叶斯网络在软件项目风险评估中的应用

在软件项目生存周期早期或创新型项目的研发过程中,可用的案例数据很少或很不完整,项目风险多由专家经验进行主观评估,给风险的客观度量带来了很大的困难。提出了一种基于贝叶斯网络的软件项目风险评估方法,不仅可度量风险影响程度的风险当量,还能度量出多种风险对某种风险后果的组合影响以及单个风险对整体后果的综合影响,从而增强了软件项目风险的预测和应变能力,为有效地降低风险发生概率、提高软件开发成功率提供了一种新的途径。     

The optimization of success probability for software projects using genetic algorithms

The software development process is usually affected by many risk factors that may cause the loss of control and failure, thus which need to be identified and mitigated by project managers. Software development companies are currently improving their process by adopting internationally accepted practices, with the aim of avoiding risks and demonstrating the quality of their work.This paper aims to develop a method to identify which risk factors are more influential in determining project outcome. This method must also propose a cost effective investment of project resources to improve the probability of project success.To achieve these aims, we use the probability of success relative to cost to calculate the efficiency of the probable project outcome. The definition of efficiency used in this paper was proposed by researchers in the field of education. We then use this efficiency as the fitness function in an optimization technique based on genetic algorithms. This method maximizes the success probability output of a prediction model relative to cost.The optimization method was tested with several software risk prediction models that have been developed based on the literature and using data from a survey which collected information from in-house and outsourced software development projects in the Chilean software industry. These models predict the probability of success of a project based on the activities undertaken by the project manager and development team. The results show that the proposed method is very useful to identify those activities needing greater allocation of resources, and which of these will have a higher impact on the projects success probability.Therefore using the measure of efficiency has allowed a modular approach to identify those activities in software development on which to focus the project's limited resources to improve its probability of success. The genetic algorithm and the measure of efficiency presented in this paper permit model independence, in both prediction of success and cost evaluation.     

Gompertz software reliability model: Estimation algorithm and empirical validation

Gompertz curve has been used to estimate the number of residual faults in testing phases of software development, especially by Japanese software development companies. Since the Gompertz curve is a deterministic function, the curve cannot be applied to estimating software reliability which is the probability that software system does not fail in a prefixed time period. In this article, we propose a stochastic model called the Gompertz software reliability model based on non-homogeneous Poisson processes. The proposed model can be derived from the statistical theory of extreme-value, and has a similar asymptotic property to the deterministic Gompertz curve. Also, we develop an EM algorithm to determine the model parameters effectively. In numerical examples with software failure data observed in real software development projects, we evaluate performance of the Gompertz software reliability model in terms of reliability assessment and failure prediction.     

The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process

Many empirical studies have found that software metrics can predict class error proneness and the prediction can be used to accurately group error-prone classes. Recent empirical studies have used open source systems. These studies, however, focused on the relationship between software metrics and class error proneness during the development phase of software projects. Whether software metrics can still predict class error proneness in a system’s post-release evolution is still a question to be answered. This study examined three releases of the Eclipse project and found that although some metrics can still predict class error proneness in three error-severity categories, the accuracy of the prediction decreased from release to release. Furthermore, we found that the prediction cannot be used to build a metrics model to identify error-prone classes with acceptable accuracy. These findings suggest that as a system evolves, the use of some commonly used metrics to identify which classes are more prone to errors becomes increasingly difficult and we should seek alternative methods (to the metric-prediction models) to locate error-prone classes if we want high accuracy.     

Combining the requirement information for software defect estimation in design time

This paper analyzes the ability of requirement metrics for software defect prediction. Statistical significance tests are used to compare six machine learning algorithms on the requirement metrics, design metrics, and combination of both metrics in our analysis. The experimental results show the effectiveness of the predictor built on the combination of the requirement and design metrics in the early phase of the software development process.     

Empirical analysis of network measures for predicting high severity software faults

Network measures are useful for predicting fault-prone modules. However, existing work has not distinguished faults according to their severity. In practice, high severity faults cause serious problems and require further attention. In this study, we explored the utility of network measures in high severity faultproneness prediction. We constructed software source code networks for four open-source projects by extracting the dependencies between modules. We then used univariate logistic regression to investigate the associations between each network measure and fault-proneness at a high severity level. We built multivariate prediction models to examine their explanatory ability for fault-proneness, as well as evaluated their predictive effectiveness compared to code metrics under forward-release and cross-project predictions. The results revealed the following: (1) most network measures are significantly related to high severity fault-proneness; (2) network measures generally have comparable explanatory abilities and predictive powers to those of code metrics; and (3) network measures are very unstable for cross-project predictions. These results indicate that network measures are of practical value in high severity fault-proneness prediction.