Stable rankings for different effort models |
| |
Authors: | Tim Menzies Omid Jalali Jairus Hihn Dan Baker Karen Lum |
| |
Affiliation: | (1) De Montfort University, Leicester, UK;(2) Dalian University of Technology, Dalian, China; |
| |
Abstract: | There exists a large and growing number of proposed estimation methods but little conclusive evidence ranking one method over
another. Prior effort estimation studies suffered from “conclusion instability”, where the rankings offered to different methods
were not stable across (a) different evaluation criteria; (b) different data sources; or (c) different random selections of
that data. This paper reports a study of 158 effort estimation methods on data sets based on COCOMO features. Four “best”
methods were detected that were consistently better than the “rest” of the other 154 methods. These rankings of “best” and
“rest” methods were stable across (a) three different evaluation criteria applied to (b) multiple data sets from two different
sources that were (c) divided into hundreds of randomly selected subsets using four different random seeds. Hence, while there
exists no single universal “best” effort estimation method, there appears to exist a small number (four) of most useful methods.
This result both complicates and simplifies effort estimation research. The complication is that any future effort estimation
analysis should be preceded by a “selection study” that finds the best local estimator. However, the simplification is that
such a study need not be labor intensive, at least for COCOMO style data sets. |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|