Handling imprecision and uncertainty in software development effort prediction: A type-2 fuzzy logic based framework

Traditional approaches for software projects effort prediction such as the use of mathematical formulae derived from historical data, or the use of experts judgments are plagued with issues pertaining to effectiveness and robustness in their results. These issues are more pronounced when these effort prediction approaches are used during the early phases of the software development lifecycle, for example requirements development, whose effort predictors along with their relationships to effort are characterized as being even more imprecise and uncertain than those of later development phases, for example design. Recent works have demonstrated promising results using approaches based on fuzzy logic. Effort prediction systems that use fuzzy logic can deal with imprecision; they, however, can not deal with uncertainty. This paper presents an effort prediction framework that is based on type-2 fuzzy logic to allow handling imprecision and uncertainty inherent in the information available for effort prediction. Evaluation experiments have shown the framework to be promising.     

Software development effort prediction: A study on the factors impacting the accuracy of fuzzy logic systems

Reliable effort prediction remains an ongoing challenge to software engineers. Traditional approaches to effort prediction such as the use of models derived from historical data, or the use of expert opinion are plagued with issues pertaining to their effectiveness and robustness. These issues are more pronounced when the effort prediction is used during the early phases of the software development lifecycle. Recent works have demonstrated promising results obtained with the use of fuzzy logic. Fuzzy logic based effort prediction systems can deal better with imprecision, which characterizes the early phases of most software development projects, for example requirements development, whose effort predictors along with their relationships to effort are characterized as being even more imprecise and uncertain than those of later development phases, for example design. Fuzzy logic based prediction systems could produce further better estimates provided that various parameters and factors pertaining to fuzzy logic are carefully set. In this paper, we present an empirical study, which shows that the prediction accuracy of a fuzzy logic based effort prediction system is highly dependent on the system architecture, the corresponding parameters, and the training algorithms.     

Probabilistic size proxy for software effort prediction: A framework

Software effort prediction is an important and challenging activity that takes place during the early stages of software development, where costing is needed. Software size estimate is one of the most popular inputs for software effort prediction models. Accordingly, providing a size estimate with good accuracy early in the lifecycle is very important; it is equally challenging too. Estimates that are computed early in the development lifecycle, when it is needed the most, are typically associated with uncertainty. However, none of the prominent software effort prediction techniques or software size metrics addresses this issue satisfactorily. In this paper, we propose a framework for developing probabilistic size proxies for software effort prediction using information from conceptual UML models created early in the software development lifecycle. The framework accounts for uncertainty in software size and effort prediction by providing the estimate as a probability density function instead of a certain value. We conducted a case study using open source datasets and the results were encouraging.     

Predictive accuracy comparison between neural networks and statistical regression for development effort of software projects

To get a better prediction of costs, schedule, and the risks of a software project, it is necessary to have a more accurate prediction of its development effort. Among the main prediction techniques are those based on mathematical models, such as statistical regressions or machine learning (ML). The ML models applied to predicting the development effort have mainly based their conclusions on the following weaknesses: (1) using an accuracy criterion which leads to asymmetry, (2) applying a validation method that causes a conclusion instability by randomly selecting the samples for training and testing the models, (3) omitting the explanation of how the parameters for the neural networks were determined, (4) generating conclusions from models that were not trained and tested from mutually exclusive data sets, (5) omitting an analysis of the dependence, variance and normality of data for selecting the suitable statistical test for comparing the accuracies among models, and (6) reporting results without showing a statistically significant difference. In this study, these six issues are addressed when comparing the prediction accuracy of a radial Basis Function Neural Network (RBFNN) with that of a regression statistical (the model most frequently compared with ML models), to feedforward multilayer perceptron (MLP, the most commonly used in the effort prediction of software projects), and to general regression neural network (GRNN, a RBFNN variant). The hypothesis tested is the following: the accuracy of effort prediction for RBFNN is statistically better than the accuracy obtained from a simple linear regression (SLR), MLP and GRNN when adjusted function points data, obtained from software projects, is used as the independent variable. Samples obtained from the International Software Benchmarking Standards Group (ISBSG) Release 11 related to new and enhanced projects were used. The models were trained and tested from a leave-one-out cross-validation method. The criteria for evaluating the models were based on Absolute Residuals and by a Friedman statistical test. The results showed that there was a statistically significant difference in the accuracy among the four models for new projects, but not for enhanced projects. Regarding new projects, the accuracy for RBFNN was better than for a SLR at the 99% confidence level, whereas the MLP and GRNN were better than for a SLR at the 90% confidence level.     

A fuzzy-filtered neuro-fuzzy framework for software fault prediction for inter-version and inter-project evaluation

Fault Prediction is the most required measure to estimate the software quality and reliability. Several methods, measures, aspects and testing methodologies are available to evaluate the software fault. In this paper, a fuzzy-filtered neuro-fuzzy framework is introduced to predict the software faults for internal and external software projects. The suggested framework is split into three primary phases. At the earlier phase, the effective metrics or measures are identified, which can derive the accurate decision on prediction of software fault. In this phase, the composite analytical observation of each software attribute is calculated using Information Gain and Gain Ratio measures. In the second phase, these fuzzy rules are applied on these measures for selection of effective and high-impact features. In the last phase, the Neuro-fuzzy classifier is applied on fuzzy-filtered training and testing sets. The proposed framework is applied to identify the software faults based on inter-version and inter-project evaluation. In this framework, the earlier projects or project-versions are considered as training sets and the new projects or versions are taken as testing sets. The experimentation is conducted on nine open source projects taken from PROMISE repository as well as on PDE and JDT projects. The approximation is applied on internal version-specific fault prediction and external software projects evaluation. The comparative analysis is performed against Decision Tree, Random Tree, Random Forest, Naive Bayes and Multilevel Perceptron classifiers. This prediction result signifies that the proposed framework has gained the higher accuracy, lesser error rate and significant AUC and GM for inter-project and inter-version evaluations.     

Estimation and prediction metrics for adaptive maintenance effortof object-oriented systems

Many software systems built in recent years have been developed using object-oriented technology and, in some cases, they already need adaptive maintenance in order to satisfy market and customer needs. In most cases, the estimation and prediction of maintenance effort is performed with difficulty due to the lack of metrics and suitable models. In this paper, a model and metrics for estimation/prediction of adaptive maintenance effort are presented and compared with some other solutions taken from the literature. The model proposed can be used as a general approach for adopting well-known metrics (typically used for the estimation of development effort) for the estimation/prediction of adaptive maintenance effort. The model and metrics proposed have been validated against real data by using multilinear regression analysis. The validation has shown that several well-known metrics can be profitably employed for the estimation/prediction of maintenance effort     

多氯联苯色谱保留指数QSPR建模的研究

以E-Dragon软件计算的拓扑指数和连接性指数作为变量,随机将209种多氯联苯化合物(PCBs)样本数据划分为训练集、验证集和预测集,采用微粒群-v-支持向量机(PSO-v-SVM)对其色谱保留指数建立QSPR模型,选定的最佳模型入选变量仅5个,对训练集、验证集和预测集计算结果的R2分别为0.999、0.998和0.999,预测的准确性很高.本文选定的模型较文献[16-19]的计算结果好,预测结果更可靠.     

Simulation-based workforce assignment in a multi-organizational social network for alliance-based software development

The development of alliance-based software requires the collaboration of many stakeholders. These different stakeholders across multiple organizations form a complex social network. The goal of this paper is to develop a novel modeling framework, which will help task managers devise optimal workforce assignments considering both short-term and long-term aspects of the software development process. The proposed framework is composed of an assignment module and a prediction module. For a given task, the assignment module first selects a candidate workforce mix. Based on the candidate workforce mix, the prediction module then predicts the short-term performance (productivity) as well as the long-term performance (workforce training and robustness of the organization) of the organization. Then, the assignment module selects another candidate mix, and this iteration continues until an optimal workforce mix is found. The prediction module and the assignment module are based on an agent-based simulation method and a multi-objective optimization model, respectively. The proposed modeling framework is illustrated with a software enhancement request process in Kuali, an alliance-based open source software development project involving 12 organizations. The constructed framework is executed with varying parameters to demonstrate its use and benefit in the software enhancement process.     

Using Bayesian regression and EM algorithm with missing handling for software effort prediction

ContextAlthough independent imputation techniques are comprehensively studied in software effort prediction, there are few studies on embedded methods in dealing with missing data in software effort prediction.ObjectiveWe propose BREM (Bayesian Regression and Expectation Maximization) algorithm for software effort prediction and two embedded strategies to handle missing data.MethodThe MDT (Missing Data Toleration) strategy ignores the missing data when using BREM for software effort prediction and the MDI (Missing Data Imputation) strategy uses observed data to impute missing data in an iterative manner while elaborating the predictive model.ResultsExperiments on the ISBSG and CSBSG datasets demonstrate that when there are no missing values in historical dataset, BREM outperforms LR (Linear Regression), BR (Bayesian Regression), SVR (Support Vector Regression) and M5′ regression tree in software effort prediction on the condition that the test set is not greater than 30% of the whole historical dataset for ISBSG dataset and 25% of the whole historical dataset for CSBSG dataset. When there are missing values in historical datasets, BREM with the MDT and MDI strategies significantly outperforms those independent imputation techniques, including MI, BMI, CMI, MINI and M5′. Moreover, the MDI strategy provides BREM with more accurate imputation for the missing values than those given by the independent missing imputation techniques on the condition that the level of missing data in training set is not larger than 10% for both ISBSG and CSBSG datasets.ConclusionThe experimental results suggest that BREM is promising in software effort prediction. When there are missing values, the MDI strategy is preferred to be embedded with BREM.     

Functional networks as a novel data mining paradigm in forecasting software development efforts

This paper proposes a new intelligence paradigm scheme to forecast that emphasizes on numerous software development elements based on functional networks forecasting framework. The most common methods for estimating software development efforts that have been proposed in literature are: line of code (LOC)-based constructive cost model (COCOMO), function point (FP) based on neural networks, regression, and case-based reasoning (CBR). Unfortunately, such forecasting models have numerous of drawbacks, namely, their inability to deal with uncertainties and imprecision present in software projects early in the development life-cycle. The main benefit of this study is to utilize both function points and development environments of recent software development cases prominent, which have high impact on the success of software development projects. Both implementation and learning process are briefly proposed. We investigate the efficiency of the new framework for predicting the software development efforts using both simulation and COCOMO real-life databases. Prediction accuracy of the functional networks framework is evaluated and compared with the commonly used regression and neural networks-based models. The results show that the new intelligence paradigm predicts the required efforts of the initial stage of software development with reliable performance and outperforms both regression and neural networks-based models.     

A new perspective on data homogeneity in software cost estimation: a study in the embedded systems domain

Cost estimation and effort allocation are the key challenges for successful project planning and management in software development. Therefore, both industry and the research community have been working on various models and techniques to accurately predict the cost of projects. Recently, researchers have started debating whether the prediction performance depends on the structure of data rather than the models used. In this article, we focus on a new aspect of data homogeneity, “cross- versus within-application domain”, and investigate what kind of training data should be used for software cost estimation in the embedded systems domain. In addition, we try to find out the effect of training dataset size on the prediction performance. Based on our empirical results, we conclude that it is better to use cross-domain data for embedded software cost estimation and the optimum training data size depends on the method used.     

Investigating the use of moving windows to improve software effort prediction: a replicated study

To date most research in software effort estimation has not taken chronology into account when selecting projects for training and validation sets. A chronological split represents the use of a project’s starting and completion dates, such that any model that estimates effort for a new project p only uses as its training set projects that have been completed prior to p’s starting date. A study in 2009 (“S3”) investigated the use of chronological split taking into account a project’s age. The research question investigated was whether the use of a training set containing only the most recent past projects (a “moving window” of recent projects) would lead to more accurate estimates when compared to using the entire history of past projects completed prior to the starting date of a new project. S3 found that moving windows could improve the accuracy of estimates. The study described herein replicates S3 using three different and independent data sets. Estimation models were built using regression, and accuracy was measured using absolute residuals. The results contradict S3, as they do not show any gain in estimation accuracy when using windows for effort estimation. This is a surprising result: the intuition that recent data should be more helpful than old data for effort estimation is not supported. Several factors, which are discussed in this paper, might have contributed to such contradicting results. Some of our future work entails replicating this work using other datasets, to understand better when using windows is a suitable choice for software companies.     

不完美排错软件可靠性增长模型效用量化研究

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合和预测性能,首先从故障总数增长趋势角度对不完美排错模型进行深入研究,提出两个一般性不完美排错框架模型,分别考虑了总故障数量函数与累计检测故障函数间的线性关系与微分关系,并求得累计检测的故障数量与软件中总故障数量函数表达式;其次,在六组真实的失效数据集上对比了提出的两种一般性不完美排错模型和六种不完美排错模型拟合预测性能表现。实例验证结果表明,提出的一般性不完美排错框架模型在大多数失效数据集上都具有优秀的拟合和预测性能,证明了新建模型的有效性和实用性;通过对提出的模型与其他不完美排错模型在数据集上的性能的深入分析,为实际应用中不完美排错模型的选择提出了建议。     

考虑测试工作量与覆盖率的软件可靠性模型

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合与预计精度,首先,提出一个同时考虑测试工作量与测试覆盖率的NHPP类软件可靠性建模框架.在此基础上,将变形S型测试工作量函数(IS-TEF)以及Logistic测试覆盖率函数(LO-TCF)带入该建模框架,建立了一个新的软件可靠性增长模型,即IS-LO-SRGM.同时,还对利用该框架进行建模过程中的两个重要问题进行了描述与分析,即如何确定具体的TEF和TCF以及模型参数估计.然后,在两组真实的失效数据集上,利用该建模框架建立了最为合适的增长模型,即IS-LO-SRGM,并将该模型与8种经典NHPP模型进行对比.实例验证结果表明,所提出的IS-LO-SRGM模型具有最为优秀的拟合与预计性能,从而证明新建模框架的有效性和实用性.最后,对不完美排错情况进行了初步的讨论与建模分析.     

A vector-based approach to software size measurement and effortestimation

Software size is a fundamental product measure that can be used for assessment, prediction and improvement purposes. However, existing software size measures, such as function points, do not address the underlying problem complexity of software systems adequately. This can result in disproportional measures of software size for different types of systems. We propose a vector size measure (VSM) that incorporates both functionality and problem complexity in a balanced and orthogonal manner. The VSM is used as the input to a vector prediction model (VPM) which can be used to estimate development effort early in the software life-cycle. We theoretically validate the approach against a formal framework. We also empirically validate the approach with a pilot study. The results indicate that the approach provides a mechanism to measure the size of software systems, classify software systems, and estimate development effort early in the software life-cycle to within ±20% across a range of application types     

Software Reliability Assessment Using Hybrid Neuro-Fuzzy Model

Software reliability is the primary concern of software development organizations, and the exponentially increasing demand for reliable software requires modeling techniques to be developed in the present era. Small unnoticeable drifts in the software can culminate into a disaster. Early removal of these errors helps the organization improve and enhance the software’s reliability and save money, time, and effort. Many soft computing techniques are available to get solutions for critical problems but selecting the appropriate technique is a big challenge. This paper proposed an efficient algorithm that can be used for the prediction of software reliability. The proposed algorithm is implemented using a hybrid approach named Neuro-Fuzzy Inference System and has also been applied to test data. In this work, a comparison among different techniques of soft computing has been performed. After testing and training the real time data with the reliability prediction in terms of mean relative error and mean absolute relative error as 0.0060 and 0.0121, respectively, the claim has been verified. The results claim that the proposed algorithm predicts attractive outcomes in terms of mean absolute relative error plus mean relative error compared to the other existing models that justify the reliability prediction of the proposed model. Thus, this novel technique intends to make this model as simple as possible to improve the software reliability.     

Advanced quality prediction model for software architectural knowledge sharing

In the field of software architecture, a paradigm shift is occurring from describing the outcome of architecting process to describing the Architectural Knowledge (AK) created and used during architecting. Many AK models have been defined to represent domain concepts and their relationships, and they can be used for sharing and reusing AK across organizations, especially in geographically distributed contexts. However, different AK domain models can represent concepts that are different, thereby making effective AK sharing challenging. In order to understand the mapping quality from one AK model to another when more than one AK model coexists, AK sharing quality prediction based on the concept differences across AK models is necessary. Previous works in this area lack validation in the actual practice of AK sharing. In this paper, we carry out validation using four AK sharing case studies. We also improve the previous prediction models. We developed a new advanced mapping quality prediction model, this model (i) improves the prediction accuracy of the recall rate of AK sharing quality; (ii) provides a better balance between prediction effort and accuracy for AK sharing quality.     

Neural network based models for software effort estimation: a review

Prediction of software development effort is the key task for the effective management of any software industry. The accuracy and reliability of prediction mechanisms is also important. Neural network based models are competitive to traditional regression and statistical models for software effort estimation. This comprehensive article, covers various neural network based models for software estimation as presented by various researchers. The review of twenty-one articles covers a range of features used for effort prediction. This survey aims to support the research for effort prediction and to emphasize capabilities of neural network based model in effort prediction.     

基于多源域适应的缺陷类别预测方法

随着规模和复杂性的迅猛膨胀,软件系统中不可避免地存在缺陷.近年来,基于深度学习的缺陷预测技术成为软件工程领域的研究热点.该类技术可以在不运行代码的情况下发现其中潜藏的缺陷,因而在工业界和学术界受到了广泛的关注.然而,已有方法大多关注方法级的源代码中是否存在缺陷,无法精确识别具体的缺陷类别,从而降低了开发人员进行缺陷定位及修复工作的效率.此外,在实际软件开发实践中,新的项目通常缺乏足够的缺陷数据来训练高精度的深度学习模型,而利用已有项目的历史数据训练好的模型往往在新项目上无法达到良好的泛化性能.因此,本文首先将传统的二分类缺陷预测任务表述为多标签分类问题,即使用CWE(common weakness enumeration)中描述的缺陷类别作为细粒度的模型预测标签.为了提高跨项目场景下的模型性能,本文提出一种融合对抗训练和注意力机制的多源域适应框架.具体而言,该框架通过对抗训练来减少域(即软件项目)差异,并进一步利用域不变特征来获得每个源域和目标域之间的特征相关性.同时,该框架还利用加权最大均值差异作为注意力机制以最小化源域和目标域特征之间的表示距离,从而使模型可以学习到更多的域无关特征.最后在八个真实世界的开源项目上与最先进的基线方法进行大量对比实验验证了所提方法的有效性.     

Software development effort prediction of industrial projects applying a general regression neural network

An important factor for planning, budgeting and bidding a software project is prediction of the development effort required to complete it. This prediction can be obtained from models related to neural networks. The hypothesis of this research was the following: effort prediction accuracy of a general regression neural network (GRNN) model is statistically equal or better than that obtained by a statistical regression model, using data obtained from industrial environments. Each model was generated from a separate dataset obtained from the International Software Benchmarking Standards Group (ISBSG) software projects repository. Each of the two models was then validated using a new dataset from the same ISBSG repository. Results obtained from a variance analysis of accuracies of the models suggest that a GRNN could be an alternative for predicting development effort of software projects that have been developed in industrial environments.