An Examination of Determinants of Software Testing and Project Management Effort

Software estimation research has primarily focused on software effort involved in direct software development. As more and more organizations buy instead of building software, more effort is spent on software testing and project management. In this empirical study, the effect of program duration, computer platform, and software development tool (SDT) on program testing effort and project management effort is studied. The study results point to program duration and software tool as significant determinants of testing and management effort. Computer platform, however, does not have an effect on testing and management effort. Furthermore, the mean testing effort for third generation (3G) development environment was significantly higher than the mean testing effort for fourth generation (4G) environments that used IDE. In addition, the management effort for 4G environment projects without the use of IDE was lower than nonprogramming report generation projects.     

Software Project Duration and Effort: An Empirical Study

Software projects frequently finish late and over budget. Much of the research to date has characterized this problem in terms of inadequate project estimation or incomplete requirements determination. In this study, we concentrate instead on understanding the relationship between project duration and project effort. Over time, a dynamic environment contributes to the expansion of project requirements, thus increasing the scope and effort required to complete the project, irrespective of initial requirements and anticipated project size. Further, frequent delays and interruptions in a project contribute to greater effort each time work is resumed. We develop and empirically evaluate a two-stage model to relate project duration and effort. Our results indicate a significant and positive relationship between project duration and effort, controlling for anticipated project size and other project characteristics. Our model also provides an estimate for the rate of environmental change while projects are in progress. We demonstrate the practical implications of our model by showing how it can be used in conjunction with time boxing techniques and new development methodologies to better scope software projects.     

测算在软件开发中的实践①

软件测算是软件开发工程化管理的重要技术。但软件本身的复杂性、历史经验的缺乏、估算工具不完备性以及人为错误,导致软件项目的估算结果往往和实际情况相差甚远。为了改进测算结果,根据软件生命周期理论,结合COCOMOⅡ模型,指出了软件开发模型中的关键测算点及相应可采用的测算方法,并且明确了软件测算人员应具备的素质要求及在典型的软件组织中的地位。最后通过实例,使用USC提供的一个免费软件测算工具对一个具体的软件项目进行测算,较好地控制开发过程。     

Estimating software project effort using analogies

Accurate project effort prediction is an important goal for the software engineering community. To date most work has focused upon building algorithmic models of effort, for example COCOMO. These can be calibrated to local environments. We describe an alternative approach to estimation based upon the use of analogies. The underlying principle is to characterize projects in terms of features (for example, the number of interfaces, the development method or the size of the functional requirements document). Completed projects are stored and then the problem becomes one of finding the most similar projects to the one for which a prediction is required. Similarity is defined as Euclidean distance in n-dimensional space where n is the number of project features. Each dimension is standardized so all dimensions have equal weight. The known effort values of the nearest neighbors to the new project are then used as the basis for the prediction. The process is automated using a PC-based tool known as ANGEL. The method is validated on nine different industrial datasets (a total of 275 projects) and in all cases analogy outperforms algorithmic models based upon stepwise regression. From this work we argue that estimation by analogy is a viable technique that, at the very least, can be used by project managers to complement current estimation techniques     

Schedule estimation and uncertainty surrounding the cone of uncertainty

Software development project schedule estimation has long been a difficult problem. The Standish CHAOS Report indicates that only 20 percent of projects finish on time relative to their original plan. Conventional wisdom proposes that estimation gets better as a project progresses. This concept is sometimes called the cone of uncertainty, a term popularized by Steve McConnell (1996). The idea that uncertainty decreases significantly as one obtains new knowledge seems intuitive. Metrics collected from Landmark's projects show that the estimation accuracy of project duration followed a lognormal distribution, and the uncertainty range was nearly identical throughout the project, in conflict with popular interpretation of the "cone of uncertainty".     

Reasons for software effort estimation error: impact of respondent role, information collection approach, and data analysis method

This study aims to improve analyses of why errors occur in software effort estimation. Within one software development company, we collected information about estimation errors through: 1) interviews with employees in different roles who are responsible for estimation, 2) estimation experience reports from 68 completed projects, and 3) statistical analysis of relations between characteristics of the 68 completed projects and estimation error. We found that the role of the respondents, the data collection approach, and the type of analysis had an important impact on the reasons given for estimation error. We found, for example, a strong tendency to perceive factors outside the respondents' own control as important reasons for inaccurate estimates. Reasons given for accurate estimates, on the other hand, typically cited factors that were within the respondents' own control and were determined by the estimators' skill or experience. This bias in types of reason means that the collection only of project managers' viewpoints will not yield balanced models of reasons for estimation error. Unfortunately, previous studies on reasons for estimation error have tended to collect information from project managers only. We recommend that software companies combine estimation error information from in-depth interviews with stakeholders in all relevant roles, estimation experience reports, and results from statistical analyses of project characteristics     

A flexible method to estimate the software development effort based on the classification of projects and localization of comparisons

The estimation of software development effort has been centralized mostly on the accuracy of estimates through dealing with heterogeneous datasets regardless of the fact that the software projects are inherently complex and uncertain. In particular, Analogy Based Estimation (ABE), as a widely accepted estimation method, suffers a great deal from the problem of inconsistent and non-normal datasets because it is a comparison-based method and the quality of comparisons strongly depends on the consistency of projects. In order to overcome this problem, prior studies have suggested the use of weighting methods, outlier elimination techniques and various types of soft computing methods. However the proposed methods have reduced the complexity and uncertainty of projects, the results are not still convincing and the methods are limited to a special domain of software projects, which causes the generalization of methods to be impossible. Localization of comparison and weighting processes through clustering of projects is the main idea behind this paper. A hybrid model is proposed in which the software projects are divided into several clusters based on key attributes (development type, organization type and development platform). A combination of ABE and Particle Swarm Optimization (PSO) algorithm is used to design a weighting system in which the project attributes of different clusters are given different weights. Instead of comparing a new project with all the historical projects, it is only compared with the projects located in the related clusters based on the common attributes. The proposed method was evaluated through three real datasets that include a total of 505 software projects. The performance of the proposed model was compared with other well-known estimation methods and the promising results showed that the proposed localization can considerably improve the accuracy of estimates. Besides the increase in accuracy, the results also certified that the proposed method is flexible enough to be used in a wide range of software projects.     

LMES: A localized multi-estimator model to estimate software development effort

Accurate estimation of software development effort is strongly associated with the success or failure of software projects. The clear lack of convincing accuracy and flexibility in this area has attracted the attention of researchers over the past few years. Despite improvements achieved in effort estimating, there is no strong agreement as to which individual model is the best. Recent studies have found that an accurate estimation of development effort in software projects is unreachable in global space, meaning that proposing a high performance estimation model for use in different types of software projects is likely impossible. In this paper, a localized multi-estimator model, called LMES, is proposed in which software projects are classified based on underlying attributes. Different clusters of projects are then locally investigated so that the most accurate estimators are selected for each cluster. Unlike prior models, LMES does not rely on only one individual estimator in a cluster of projects. Rather, an exhaustive investigation is conducted to find the best combination of estimators to assign to each cluster. The investigation domain includes 10 estimators combined using four combination methods, which results in 4017 different combinations. ISBSG, Maxwell and COCOMO datasets are utilized for evaluation purposes, which include a total of 573 real software projects. The promising results show that the estimate accuracy is improved through localization of estimation process and allocation of appropriate estimators. Besides increased accuracy, the significant contribution of LMES is its adaptability and flexibility to deal with the complexity and uncertainty that exist in the field of software development effort estimation.     

Managing uncertainty in project portfolio cost estimation

Although typically a software development organisation is involved in more than one project simultaneously, the available tools in the area of software cost estimation deal mostly with single software projects. In order to calculate the possible cost of the entire project portfolio, one must combine the single project estimates taking into account the uncertainty involved. In this paper, statistical simulation techniques are used to calculate confidence intervals for the effort needed for a project portfolio. The overall approach is illustrated through the adaptation of the analogy-based method for software cost estimation to cover multiple projects.     

A PSO-based model to increase the accuracy of software development effort estimation

Development effort is one of the most important metrics that must be estimated in order to design the plan of a project. The uncertainty and complexity of software projects make the process of effort estimation difficult and ambiguous. Analogy-based estimation (ABE) is the most common method in this area because it is quite straightforward and practical, relying on comparison between new projects and completed projects to estimate the development effort. Despite many advantages, ABE is unable to produce accurate estimates when the importance level of project features is not the same or the relationship among features is difficult to determine. In such situations, efficient feature weighting can be a solution to improve the performance of ABE. This paper proposes a hybrid estimation model based on a combination of a particle swarm optimization (PSO) algorithm and ABE to increase the accuracy of software development effort estimation. This combination leads to accurate identification of projects that are similar, based on optimizing the performance of the similarity function in ABE. A framework is presented in which the appropriate weights are allocated to project features so that the most accurate estimates are achieved. The suggested model is flexible enough to be used in different datasets including categorical and non-categorical project features. Three real data sets are employed to evaluate the proposed model, and the results are compared with other estimation models. The promising results show that a combination of PSO and ABE could significantly improve the performance of existing estimation models.     

A comparison of software project overruns - flexible versus sequential development models

Flexible software development models, e.g., evolutionary and incremental models, have become increasingly popular. Advocates claim that among the benefits of using these models is reduced overruns, which is one of the main challenges of software project management. This paper describes an in-depth survey of software development projects. The results support the claim that projects which employ a flexible development model experience less effort overruns than do those which employ a sequential model. The reason for the difference is not obvious. We found, for example, no variation in project size, estimation process, or delivered proportion of planned functionality between projects applying different types of development model. When the managers were asked to provide reasons for software overruns and/or estimation accuracy, the largest difference was that more of flexible projects than sequential projects cited good requirement specifications-and good collaboration/communication with clients as contributing to accurate estimates.     

Algorithmic optimization method for effort estimation

This paper presents an improvement of an effort estimation method that can be used to predict the level of effort for software development projects. A new estimation approach based on a two-phase algorithm is used. In the first phase, we apply a calculation based on use case points (UCPs). In the second phase, we add correction values (a ₁, a ₂) obtained via least squares regression. This approach employs historical project data to refine the estimate. By applying the least squares regression approach, the algorithm filters out estimation errors caused by human factors and company practice.     

Investigating the use of duration-based moving windows to improve software effort prediction: A replicated study

ContextMost research in software effort estimation has not considered chronology when selecting projects for training and testing sets. A chronological split represents the use of a projects starting and completion dates, such that any model that estimates effort for a new project p only uses as training data projects that were completed prior to p’s start. Four recent studies investigated the use of chronological splits, using moving windows wherein only the most recent projects completed prior to a projects starting date were used as training data. The first three studies (S1–S3) found some evidence in favor of using windows; they all defined window sizes as being fixed numbers of recent projects. In practice, we suggest that estimators think in terms of elapsed time rather than the size of the data set, when deciding which projects to include in a training set. In the fourth study (S4) we showed that the use of windows based on duration can also improve estimation accuracy.ObjectiveThis papers contribution is to extend S4 using an additional dataset, and to also investigate the effect on accuracy when using moving windows of various durations.MethodStepwise multivariate regression was used to build prediction models, using all available training data, and also using windows of various durations to select training data. Accuracy was compared based on absolute residuals and MREs; the Wilcoxon test was used to check statistical significances between results. Accuracy was also compared against estimates derived from windows containing fixed numbers of projects.ResultsNeither fixed size nor fixed duration windows provided superior estimation accuracy in the new data set.ConclusionsContrary to intuition, our results suggest that it is not always beneficial to exclude old data when estimating effort for new projects. When windows are helpful, windows based on duration are effective.     

The impact of global dispersion on coordination,team performance and software quality – A systematic literature review

ContextGlobal software development (GSD) contains different context setting dimensions, which are essential for effective teamwork and success of projects. Although considerable research effort has been made in this area, as yet, no agreement has been reached about the impact of these dispersion dimensions on team coordination and project outcomes.ObjectiveThis paper summarizes empirical evidence on the impact of global dispersion dimensions on coordination, team performance and project outcomes.MethodWe performed a systematic literature review of 46 publications from 25 journals and 19 conference and workshop proceedings, which were published between 2001 and 2013. Thematic analysis was used to identify global dimensions and their measures. Vote counting was used to decide on the impact trends of dispersion dimensions on team performance and software quality.ResultsGlobal dispersion dimensions are consistently conceptualized, but quantified in many different ways. Different dispersion dimensions are associated with a distinct set of coordination challenges. Overall, geographical dispersion tends to have a negative impact on team performance and software quality. Temporal dispersion tends to have a negative impact on software quality, but its impact on team performance is inconsistent and can be explained by type of performance.ConclusionFor researchers, we reveal several opportunities for future research, such as coordination challenges in inter-organizational software projects, impact of processes and practices mismatches on project outcomes, evolution of coordination needs and mechanism over time and impact of dispersion dimensions on open source project outcomes. For practitioners, they should consider the tradeoff between cost and benefits while dispersing tasks, alignment impact of dispersion dimensions with individual and organizational objectives, coordination mechanisms as situational approaches and collocation of development activities of high quality demand components in GSD projects.     

Interactive and diagnostic uses of management control systems in IS projects: Antecedents and their impact on performance

We attempted to determine how formal management control systems (MCS) are used by project managers in IS development (ISD) contexts. This involved investigating the antecedents of two types of project MCS use (interactive and diagnostic), and their direct and moderated impact on project performance. PLS analysis of data collected in a survey of 93 projects indicated that project managers’ level of discretion positively affected their level of interactive use of project MCS but did not influence their diagnostic use. Our findings also showed that interactive use of MCS enhanced performance when task uncertainty (task novelty and complexity) of an ISD was high, but worsened it when task uncertainty was low. Finally, diagnostic use of MCS apparently increased project performance when an ISD task uncertainty was low, but did not reduce it when task uncertainty was high. Overall, these results were stable across different size projects.     

Impact of effort estimates on software project work

This paper presents results from two case studies and two experiments on how effort estimates impact software project work. The studies indicate that a meaningful interpretation of effort estimation accuracy requires knowledge about the size and nature of the impact of the effort estimates on the software work. For example, we found that projects with high priority on costs and incomplete requirements specifications were prone to adjust the work to fit the estimate when the estimates were too optimistic, while too optimistic estimates led to effort overruns for projects with high priority on quality and well specified requirements.

Two hypotheses were derived from the case studies and tested experimentally. The experiments indicate that: (1) effort estimates can be strongly impacted by anchor values, e.g. early indications on the required effort. This impact is present even when the estimators are told that the anchor values are irrelevant as estimation information; (2) too optimistic effort estimates lead to less use of effort and more errors compared with more realistic effort estimates on programming tasks.     

Exploiting organizational knowledge in developing IS project cost and schedule estimates: An empirical study

Project management is vital to the effective application of organizational resources to competing demands within and across projects. The effective application of project management, however, is predicated upon accurate estimates of the project budget and schedule. This study assesses primary and supporting activities that exploit knowledge within an organization's memory to develop project schedule durations and budgets. The study further assesses the subsequent impact of predictability on project success. Two hundred and sixteen survey responses from IT professionals with project management responsibilities were analyzed. Results found use of the primary activities of using parametric estimating techniques (use of formal models), bottom-up estimating techniques (formulating estimates at the task level), and the support activities of team reliance, realistic targets, and professional experience all impact the predictability of estimates for project cost and duration. Predictability in turn was found to directly impact project success with regards to meeting cost and duration objectives. While use of analogous estimating techniques (using similar previous projects) was not found to be useful for project managers with more experience, it was used by project managers with less experience in determining predictability.     

A statistical analysis of the effects of Scrum and Kanban on software development projects

Traditionally, software development processes have relied on the use of the “Waterfall” and “Vee” models. Later, Agile methodologies were used to handle the challenges of managing complex projects during the development phase. Agile methodologies are a group of incremental and iterative methods that are more effective, and have been used in project management. Kanban and Scrum are two powerful Agile project management approaches in software development. The objective of Scrum and Kanban is achieved by optimizing the development process by identifying the tasks, managing time more effectively, and setting-up teams. A review of the literature reveals that there is a lack of statistical evidence to conclude which methodology is more effective in dealing with the traditional project management factors of budget handling, risk control, quality of the project, available resources, having clear project scope, and schedule handling. This research statistically compares the effectiveness of the Scrum and Kanban methods in terms of their effects on the project management factors for software development projects. Numerical analysis is performed based on survey responses from those with experience in the Scrum and Kanban methods. Results suggest that both Scrum and Kanban lead to the development of successful projects, and that the Kanban method can be better than the Scrum method in terms of managing project schedule.     

Estimating Maintenance Effort by Analogy

Effort estimation is a key step of any software project. This paper presents a method to estimate project effort using an improved version of analogy. Unlike estimation methods based on case-based reasoning, our method makes use of two nearest neighbors of the target project for estimation. An additional refinement based on the relative location of the target project is then applied to generate the effort estimate. We first identify the relationships between cost drivers and project effort, and then determine the number of past project data that should be used in the estimation to provide the best result. Our method is then applied to a set of maintenance projects. Based on a comparison of the estimation results from our estimation method and those of other estimation methods, we conclude that our method can provide more accurate results.