Kernel methods for software effort estimation

Analogy based estimation (ABE) generates an effort estimate for a new software project through adaptation of similar past projects (a.k.a. analogies). Majority of ABE methods follow uniform weighting in adaptation procedure. In this research we investigated non-uniform weighting through kernel density estimation. After an extensive experimentation of 19 datasets, 3 evaluation criteria, 5 kernels, 5 bandwidth values and a total of 2090 ABE variants, we found that: (1) non-uniform weighting through kernel methods cannot outperform uniform weighting ABE and (2) kernel type and bandwidth parameters do not produce a definite effect on estimation performance. In summary simple ABE approaches are able to perform better than much more complex approaches. Hence,—provided that similar experimental settings are adopted—we discourage the use of kernel methods as a weighting strategy in ABE.     

Practical guidelines for expert-judgment-based software effort estimation

This article presents seven guidelines for producing realistic software development effort estimates. The guidelines derive from industrial experience and empirical studies. While many other guidelines exist for software effort estimation, these guidelines differ from them in three ways: 1) They base estimates on expert judgments rather than models. 2) They are easy to implement. 3) They use the most recent findings regarding judgment-based effort estimation. Estimating effort on the basis of expert judgment is the most common approach today, and the decision to use such processes instead of formal estimation models shouldn't be surprising. Simple process changes such as reframing questions can lead to more realistic estimates of software development efforts.     

A soft computing framework for software effort estimation

Accurate software estimation such as cost estimation, quality estimation and risk analysis is a major issue in software project management. In this paper, we present a soft computing framework to tackle this challenging problem. We first use a preprocessing neuro-fuzzy inference system to handle the dependencies among contributing factors and decouple the effects of the contributing factors into individuals. Then we use a neuro-fuzzy bank to calibrate the parameters of contributing factors. In order to extend our framework into fields that lack of an appropriate algorithmic model of their own, we propose a default algorithmic model that can be replaced when a better model is available. One feature of this framework is that the architecture is inherently independent of the choice of algorithmic models or the nature of the estimation problems. By integrating neural networks, fuzzy logic and algorithmic models into one scheme, this framework has learning ability, integration capability of both expert knowledge and project data, good interpretability, and robustness to imprecise and uncertain inputs. Validation using industry project data shows that the framework produces good results when used to predict software cost.     

Fuzzy grey relational analysis for software effort estimation

Accurate and credible software effort estimation is a challenge for academic research and software industry. From many software effort estimation models in existence, Estimation by Analogy (EA) is still one of the preferred techniques by software engineering practitioners because it mimics the human problem solving approach. Accuracy of such a model depends on the characteristics of the dataset, which is subject to considerable uncertainty. The inherent uncertainty in software attribute measurement has significant impact on estimation accuracy because these attributes are measured based on human judgment and are often vague and imprecise. To overcome this challenge we propose a new formal EA model based on the integration of Fuzzy set theory with Grey Relational Analysis (GRA). Fuzzy set theory is employed to reduce uncertainty in distance measure between two tuples at the k ^th continuous feature ( | ( x_o(k) - x_i(k) | ) \left( {\left| {\left( {{x_o}(k) - {x_i}(k)} \right.} \right|} \right) .GRA is a problem solving method that is used to assess the similarity between two tuples with M features. Since some of these features are not necessary to be continuous and may have nominal and ordinal scale type, aggregating different forms of similarity measures will increase uncertainty in the similarity degree. Thus the GRA is mainly used to reduce uncertainty in the distance measure between two software projects for both continuous and categorical features. Both techniques are suitable when relationship between effort and other effort drivers is complex. Experimental results showed that using integration of GRA with FL produced credible estimates when compared with the results obtained using Case-Based Reasoning, Multiple Linear Regression and Artificial Neural Networks methods.     

Debugging effort estimation using software metrics

Measurements of 23 style characteristics, and the program metrics LOC, V(g), VARS, and PARS were collected from student Cobol programs by a program analyzer. These measurements, together with debugging time (syntax and logic) data, were analyzed using several statistical procedures of SAS (statistical analysis system), including linear, quadratic, and multiple regressions. Some of the characteristics shown to correlate significantly with debug time are GOTO usage, structuring of the IF-ELSE construct, level 88 item usage, paragraph invocation pattern, and data name length. Among the observed characteristic measures which are associated with lowest debug times are: 17% blank lines in the data division, 12% blank lines in the procedure division, and 13-character-long data items. A debugging effort estimator, DEST, was developed to estimate debug times     

A model for software development effort and cost estimation

Several algorithmic models have been proposed to estimate software costs and other management parameters. Early prediction of completion time is absolutely essential for proper advance planning and aversion of the possible ruin of a project. L.H. Putnam's (1978) SLIM (Software LIfecycle Management) model offers a fairly reliable method that is used extensively to predict project completion times and manpower requirements as the project evolves. However, the nature of the Norden/Rayleigh curve used by Putnam renders it unreliable during the initial phases of the project, especially in projects involving a fast manpower buildup, as is the case with most software projects. In this paper, we propose the use of a model that improves early prediction considerably over the Putnam model. An analytic proof of the model's improved performance is also demonstrated on simulated data     

A flexible method for software effort estimation by analogy

Effort estimation by analogy uses information from former similar projects to predict the effort for a new project. Existing analogy-based methods are limited by their inability to handle non-quantitative data and missing values. The accuracy of predictions needs improvement as well. In this paper, we propose a new flexible method called AQUA that is able to overcome the limitations of former methods. AQUA combines ideas from two known analogy-based estimation techniques: case-based reasoning and collaborative filtering. The method is applicable to predict effort related to any object at the requirement, feature, or project levels. Which are the main contributions of AQUA when compared to other methods? First, AQUA supports non-quantitative data by defining similarity measures for different data types. Second, it is able to tolerate missing values. Third, the results from an explorative study in this paper shows that the prediction accuracy is sensitive to both the number N of analogies (similar objects) taken for adaptation and the threshold T for the degree of similarity, which is true especially for larger data sets. A fixed and small number of analogies, as assumed in existing analogy-based methods, may not produce the best accuracy of prediction. Fourth, a flexible mechanism based on learning of existing data is proposed for determining the appropriate values of N and T likely to offer the best accuracy of prediction. New criteria to measure the quality of prediction are proposed. AQUA was validated against two internal and one public domain data sets with non-quantitative attributes and missing values. The obtained results are encouraging. In addition, acomparative analysis with existing analogy-based estimation methods was conducted using three publicly available data sets that were used by these methods. Intwo of the three cases, AQUA outperformed all other methods.     

软件项目工作量动态估计方法研究

为解决目前软件行业工作量估计准确率低的问题,提出动态估计软件项目工作量的方法.首先,在项目执行前,针对工作量与规模之间的线性和非线性关系,采用基于规模的工作量估计模型对工作量进行初步估计;其次,在项目执行过程中,依据不断完善的信息对工作量估计进行调整;最后,在项目完成后,对工作量估计方法与估计结果进行评价,提出利用工作量与进度之间的幂指函数关系作为估计结果的验证指南,以提高估计结果的准确性.该方法将工作量估计作为贯穿项目始终的任务,并且将其作为一个动态的过程加以管理,为企业提高估计精度提供了一种简单、有效的方法,实现了估计方法的持续改进.     

用于软件开发工作量估算的IOP模型

软件开发工作量估算可以为多项与组织决策和项目管理相关的任务提供有效的支持.根据工作量估算的不同目标,通过对COCOMO Ⅱ成本驱动因子进行扩充和对国内外最新软件项目数据进行回归分析,建立了一个用于工作量估算的IOP模型.该模型采用统一框架,分别从行业水平、组织水平和项目特征3个层次实现基于规模的软件开发工作量估算,以满足针对软件行业、软件组织和特定软件项目的不同的估算目标,例如项目招标、软件组织不同项目的管理和具体软件项目的管理等.最后,给出了IOP模型应用的若干实例.     

Analogy-based software effort estimation using Fuzzy numbers

Background

Early stage software effort estimation is a crucial task for project bedding and feasibility studies. Since collected data during the early stages of a software development lifecycle is always imprecise and uncertain, it is very hard to deliver accurate estimates. Analogy-based estimation, which is one of the popular estimation methods, is rarely used during the early stage of a project because of uncertainty associated with attribute measurement and data availability.

Aims

We have integrated analogy-based estimation with Fuzzy numbers in order to improve the performance of software project effort estimation during the early stages of a software development lifecycle, using all available early data. Particularly, this paper proposes a new software project similarity measure and a new adaptation technique based on Fuzzy numbers.

Method

Empirical evaluations with Jack-knifing procedure have been carried out using five benchmark data sets of software projects, namely, ISBSG, Desharnais, Kemerer, Albrecht and COCOMO, and results are reported. The results are compared to those obtained by methods employed in the literature using case-based reasoning and stepwise regression.

Results

In all data sets the empirical evaluations have shown that the proposed similarity measure and adaptation techniques method were able to significantly improve the performance of analogy-based estimation during the early stages of software development. The results have also shown that the proposed method outperforms some well know estimation techniques such as case-based reasoning and stepwise regression.

Conclusions

It is concluded that the proposed estimation model could form a useful approach for early stage estimation especially when data is almost uncertain.     

Finding conclusion stability for selecting the best effort predictor in software effort estimation

Background: Conclusion Instability in software effort estimation (SEE) refers to the inconsistent results produced by a diversity of predictors using different datasets. This is largely due to the “ranking instability” problem, which is highly related to the evaluation criteria and the subset of the data being used. Aim: To determine stable rankings of different predictors. Method: 90 predictors are used with 20 datasets and evaluated using 7 performance measures, whose results are subject to Wilcoxon rank test (95 %). These results are called the “aggregate results”. The aggregate results are challenged by a sanity check, which focuses on a single error measure (MRE) and uses a newly developed evaluation algorithm called CLUSTER. These results are called the “specific results.” Results: Aggregate results show that: (1) It is now possible to draw stable conclusions about the relative performance of SEE predictors; (2) Regression trees or analogy-based methods are the best performers. The aggregate results are also confirmed by the specific results of the sanity check. Conclusion: This study offers means to address the conclusion instability issue in SEE, which is an important finding for empirical software engineering.     

A pragmatic ensemble learning approach for effective software effort estimation

Innovations in Systems and Software Engineering - The immense increase in software technology has resulted in the convolution of software projects. Software effort estimation is fundamental to...     

Comparison of weighted grey relational analysis for software effort estimation

In recent years, grey relational analysis (GRA), a similarity-based method, has been proposed and used in many applications. However, we found that most traditional GRA methods only consider nonweighted similarity for predicting software development effort. In fact, nonweighted similarity may cause biased predictions, because each feature of a project may have a different degree of relevance to the development effort. Therefore, this paper proposes six weighted methods, including nonweighted, distance-based, correlative, linear, nonlinear, and maximal weights, to be integrated into GRA for software effort estimation. Numerical examples and sensitivity analyses based on four public datasets are used to show the performance of the proposed methods. The experimental results indicate that the weighted GRA can improve estimation accuracy and reliability from the nonweighted GRA. The results also demonstrate that the weighted GRA performs better than other estimation techniques and published results. In summary, we can conclude that weighted GRA can be a viable and alternative method for predicting software development effort.     

Neural network based models for software effort estimation: a review

Prediction of software development effort is the key task for the effective management of any software industry. The accuracy and reliability of prediction mechanisms is also important. Neural network based models are competitive to traditional regression and statistical models for software effort estimation. This comprehensive article, covers various neural network based models for software estimation as presented by various researchers. The review of twenty-one articles covers a range of features used for effort prediction. This survey aims to support the research for effort prediction and to emphasize capabilities of neural network based model in effort prediction.     

Optimization of analogy weights by genetic algorithm for software effort estimation

A reliable and accurate estimate of software development effort has always been a challenge for both the software industry and academia. Analogy is a widely adopted problem solving technique that has been evaluated and confirmed in software effort or cost estimation domains. Similarity measures between pairs of effort drivers play a central role in analogy-based estimation models. However, hardly any research has addressed the issue of how to decide on suitable weighted similarity measures for software effort drivers. The present paper investigates the effect on estimation accuracy of the adoption of genetic algorithm (GA) to determine the appropriate weighted similarity measures of effort drivers in analogy-based software effort estimation models. Three weighted analogy methods, namely, the unequally weighted, the linearly weighted and the nonlinearly weighted methods are investigated in the present paper. We illustrate our approaches with data obtained from the International Software Benchmarking Standards Group (ISBSG) repository and the IBM DP services database. The experimental results show that applying GA to determine suitable weighted similarity measures of software effort drivers in analogy-based software effort estimation models is a feasible approach to improving the accuracy of software effort estimates. It also demonstrates that the nonlinearly weighted analogy method presents better estimate accuracy over the results obtained using the other methods.     

Application of data mining methods for effort estimation of software projects

Information technology companies currently use data mining techniques in different areas with the goal of increasing the quality of decision-making and to improve their business performance. The study described in this paper uses a data mining approach to produce an effort estimation of a software development process. It is based on data collected in a Croatian information technology company. The study examined 27 software projects with a total effort exceeding 42 000 work hours. The presented model employs a modified Cross-Industry Standard Process for Data Mining, where prior to model creation, additional clustering of projects is performed. The results generated by the proposed approach generally had a smaller effort estimation error than the results of human experts. The applied approach has proved that sound results can be gained through the use of data mining within the studied area. As a result, it would be wise to use such estimates as additional input in the decision-making process.     

A stability assessment of solution adaptation techniques for analogy-based software effort estimation

Among numerous possible choices of effort estimation methods, analogy-based software effort estimation based on Case-based reasoning is one of the most adopted methods in both the industry and research communities. Solution adaptation is the final step of analogy-based estimation, employed to aggregate and adapt to solutions derived during the case-based reasoning process. Variants of solution adaptation techniques have been proposed in previous studies; however, the ranking of these techniques is not conclusive and shows conflicting results, since different studies rank these techniques in different ways. This paper aims to find a stable ranking of solution adaptation techniques for analogy-based estimation. Compared with the existing studies, we evaluate 8 commonly adopted solution techniques with more datasets (12), more feature selection techniques included (4), and more stable error measures (5) to a robust statistical test method based on the Brunner test. This comprehensive experimental procedure allows us to discover a stable ranking of the techniques applied, and to observe similar behaviors from techniques with similar adaptation mechanisms. In general, the linear adaptation techniques based on the functions of size and productivity (e.g., regression towards the mean technique) outperform the other techniques in a more robust experimental setting adopted in this study. Our empirical results show that project features with strong correlation to effort, such as software size or productivity, should be utilized in the solution adaptation step to achieve desirable performance. Designing a solution adaptation strategy in analogy-based software effort estimation requires careful consideration of those influential features to ensure its prediction is of relevant and accurate.     

A distributed problem‐solving framework for probabilistic software effort estimation

Distributed problem‐solving (DPS) systems use a framework of human organizational notions and principles of intelligent systems to solve complex problems. Human organizational notions are used to decompose a complex problem into sub‐problems that can be solved using intelligent systems. The solutions of these sub‐problems are combined to solve the original complex problem. In this paper, we propose a DPS system for probabilistic estimation of software development effort. Using a real‐world software engineering dataset, we compare the performance of the DPS system with a neural network (NN) and show that the performance of the DPS system is equal to or better than that of the NN with the additional benefits of modularity, probabilistic estimates, greater interpretability, flexibility and capability to handle incomplete input data.     

A class of hybrid multilayer perceptrons for software development effort estimation problems

The dilation-erosion-linear perceptron is a hybrid morphological neuron which has been recently proposed in the literature to solve some prediction problems. However, a drawback arises from such model for building mappings to solve tasks with complex input-output nonlinear relationships within effort estimation problems. In this sense, to overcome this limitation, we present a particular class of hybrid multilayer perceptrons, called the multilayer dilation-erosion-linear perceptron (MDELP), to deal with software development effort estimation problems. Each processing unit of the proposed model is composed of a mix between a hybrid morphological operator (given by a balanced combination between dilation and erosion operators) and a linear operator. According to Pessoa and Maragos’s ideas, we propose a descending gradient-based learning process to train the proposed model. Besides, we conduct an experimental analysis using relevant datasets of software development effort estimation and the achieved results are discussed and compared, according to MMRE and PRED25 measures, to those obtained by classical and state of the art models presented in the literature.     

Neural network models for software development effort estimation: a comparative study

Software development effort estimation (SDEE) is one of the main tasks in software project management. It is crucial for a project manager to efficiently predict the effort or cost of a software project in a bidding process, since overestimation will lead to bidding loss and underestimation will cause the company to lose money. Several SDEE models exist; machine learning models, especially neural network models, are among the most prominent in the field. In this study, four different neural network models—multilayer perceptron, general regression neural network, radial basis function neural network, and cascade correlation neural network—are compared with each other based on: (1) predictive accuracy centred on the mean absolute error criterion, (2) whether such a model tends to overestimate or underestimate, and (3) how each model classifies the importance of its inputs. Industrial datasets from the International Software Benchmarking Standards Group (ISBSG) are used to train and validate the four models. The main ISBSG dataset was filtered and then divided into five datasets based on the productivity value of each project. Results show that the four models tend to overestimate in 80 % of the datasets, and the significance of the model inputs varies based on the selected model. Furthermore, the cascade correlation neural network outperforms the other three models in the majority of the datasets constructed on the mean absolute residual criterion.