The value of software sizing

Context

One of the difficulties faced by software development Project Managers is estimating the cost and schedule for new projects. Previous industry surveys have concluded that software size and cost estimation is a significant technical area of concern. In order to estimate cost and schedule it is important to have a good understanding of the size of the software product to be developed. There are a number of techniques used to derive software size, with function points being amongst the most documented.

Objective

In this paper we explore the utility of function point software sizing techniques when applied to two levels of software requirements documentation in a commercial software development organisation. The goal of the research is to appraise the value (cost/benefit) which functional sizing techniques can bring to the project planning and management of software projects within a small-to-medium sized software development enterprise (SME).

Method

Functional counts were made at the bid and detailed functional specification stages for each of five commercial projects used in the research. Three variants of the NESMA method were used to determine these function counts. Through a structured interview session, feedback on the sizing results was obtained to evaluate its feasibility and potential future contribution to the company.

Results

The results of our research suggest there is value in performing size estimates at two appropriate stages in the software development lifecycle, with simplified methods providing the optimal return on effort expended.

Conclusion

The ‘Estimated NESMA’ is the most appropriate tool for use in size estimation for the company studied. The use of software sizing provides a valuable contribution which would augment, but not replace, the company’s existing cost estimation approach.     

Partnering effects on user-developer conflict and role ambiguity in information system projects

Context

Information system development (ISD) has been plagued with high failure rates. This is partially due to the activities being a combination of both a technical and social processes involving stakeholders with conflicting interests.

Objective

Existing software risk management theories and frameworks offer limited suggestions for actions that can be taken to reduce the chance of failure of ISD projects. Our objective is to examine the connections among some of the more important user related risks in order to shed light on how specific strategies enhance the chance of project success.

Method

We conducted a sample of information systems project managers to test a multivariate model to explain the impact of pursuing a partnership with users on the conflicts that arise between users and developers, role ambiguity, and subsequent impact on project performance.

Results

The proposed model was supported, suggesting that user-developer conflict and role ambiguity have a negative impact on performance estimation difficulty, which negatively affects project performance.

Conclusion

Pursuit of project partnering yields a number of significant relationships in the model indicating an organization can implement practices that reduce risks associated with role ambiguity and conflict in system development projects.     

Code churn estimation using organisational and code metrics: An experimental comparison

Context

Source code revision control systems contain vast amounts of data that can be exploited for various purposes. For example, the data can be used as a base for estimating future code maintenance effort in order to plan software maintenance activities. Previous work has extensively studied the use of metrics extracted from object-oriented source code to estimate future coding effort. In comparison, the use of other types of metrics for this purpose has received significantly less attention.

Objective

This paper applies machine learning techniques to unveil predictors of yearly cumulative code churn of software projects on the basis of metrics extracted from revision control systems.

Method

The study is based on a collection of object-oriented code metrics, XML code metrics, and organisational metrics. Several models are constructed with different subsets of these metrics. The predictive power of these models is analysed based on a dataset extracted from eight open-source projects.

Results

The study shows that a code churn estimation model built purely with organisational metrics is superior to one built purely with code metrics. However, a combined model provides the highest predictive power.

Conclusion

The results suggest that code metrics in general, and XML metrics in particular, are complementary to organisational metrics for the purpose of estimating code churn.     

The longitudinal, chronological case study research strategy: A definition, and an example from IBM Hursley Park

Context

There is surprisingly little empirical software engineering research (ESER) that has analysed and reported the rich, fine-grained behaviour of phenomena over time using qualitative and quantitative data. The ESER community also increasingly recognises the need to develop theories of software engineering phenomena e.g. theories of the actual behaviour of software projects at the level of the project and over time.

Objective

To examine the use of the longitudinal, chronological case study (LCCS) as a research strategy for investigating the rich, fine-grained behaviour of phenomena over time using qualitative and quantitative data.

Method

Review the methodological literature on longitudinal case study. Define the LCCS and demonstrate the development and application of the LCCS research strategy to the investigation of Project C, a software development project at IBM Hursley Park. Use the study to consider prospects for LCCSs, and to make progress on a theory of software project behaviour.

Results

LCCSs appear to provide insights that are hard to achieve using existing research strategies, such as the survey study. The LCCS strategy has basic requirements that data is time-indexed, relatively fine-grained and collected contemporaneous to the events to which the data refer. Preliminary progress is made on a theory of software project behaviour.

Conclusion

LCCS appears well suited to analysing and reporting rich, fine-grained behaviour of phenomena over time.     

Systematic literature review of machine learning based software development effort estimation models

Context

Software development effort estimation (SDEE) is the process of predicting the effort required to develop a software system. In order to improve estimation accuracy, many researchers have proposed machine learning (ML) based SDEE models (ML models) since 1990s. However, there has been no attempt to analyze the empirical evidence on ML models in a systematic way.

Objective

This research aims to systematically analyze ML models from four aspects: type of ML technique, estimation accuracy, model comparison, and estimation context.

Method

We performed a systematic literature review of empirical studies on ML model published in the last two decades (1991-2010).

Results

We have identified 84 primary studies relevant to the objective of this research. After investigating these studies, we found that eight types of ML techniques have been employed in SDEE models. Overall speaking, the estimation accuracy of these ML models is close to the acceptable level and is better than that of non-ML models. Furthermore, different ML models have different strengths and weaknesses and thus favor different estimation contexts.

Conclusion

ML models are promising in the field of SDEE. However, the application of ML models in industry is still limited, so that more effort and incentives are needed to facilitate the application of ML models. To this end, based on the findings of this review, we provide recommendations for researchers as well as guidelines for practitioners.     

Determinants of software quality in offshore development - An empirical study of an Indian vendor

Context

Cost advantage has been one of the primary drivers of successful offshoring engagements of Indian software and services companies. However, the emphasis has shifted to the ability of the vendors to provide high quality over cost advantage in delivering software products and services. Meeting high quality requirements of the clients is a challenge due to the very nature of development and delivery of software through offshoring.

Objective

The objective of this research paper is to identify and evaluate the key determinants of quality in the case of software projects delivered through offshoring model.

Method

A detailed survey was conducted among project managers/project leaders (leads) of a leading midsize Indian IT services company to evaluate the relationship of the determinants on the attributes of quality.

Results

Out of six determinants, our research reveals requirements uncertainty has significant association with all the attributes of quality. While process maturity and trained personnel have moderate association, communication and control, knowledge transfer and integration and technical infrastructure have relatively low association on software quality attributes in the case of offshoring.

Conclusion

It is concluded that the complexities in offshoring necessitates proper capturing of requirements. In addition high level of process maturity and availability of trained personnel to the project will help vendors to achieve software quality. The paper provides a set of implications for practice and directions for further research.     

A controlled experiment in assessing and estimating software maintenance tasks

Context

Software maintenance is an important software engineering activity that has been reported to account for the majority of the software total cost. Thus, understanding the factors that influence the cost of software maintenance tasks helps maintainers to make informed decisions about their work.

Objective

This paper describes a controlled experiment of student programmers performing maintenance tasks on a C++ program. The objective of the study is to assess the maintenance size, effort, and effort distributions of three different maintenance types and to describe estimation models to predict the programmer’s effort spent on maintenance tasks.

Method

Twenty-three graduate students and a senior majoring in computer science participated in the experiment. Each student was asked to perform maintenance tasks required for one of the three task groups. The impact of different LOC metrics on maintenance effort was also evaluated by fitting the data collected into four estimation models.

Results

The results indicate that corrective maintenance is much less productive than enhancive and reductive maintenance and program comprehension activities require as much as 50% of the total effort in corrective maintenance. Moreover, the best software effort model can estimate the time of 79% of the programmers with the error of or less than 30%.

Conclusion

Our study suggests that the LOC added, modified, and deleted metrics are good predictors for estimating the cost of software maintenance. Effort estimation models for maintenance work may use the LOC added, modified, deleted metrics as the independent parameters instead of the simple sum of the three. Another implication is that reducing business rules of the software requires a sizable proportion of the software maintenance effort. Finally, the differences in effort distribution among the maintenance types suggest that assigning maintenance tasks properly is important to effectively and efficiently utilize human resources.     

A systematic literature review of actionable alert identification techniques for automated static code analysis

Context

Automated static analysis (ASA) identifies potential source code anomalies early in the software development lifecycle that could lead to field failures. Excessive alert generation and a large proportion of unimportant or incorrect alerts (unactionable alerts) may cause developers to reject the use of ASA. Techniques that identify anomalies important enough for developers to fix (actionable alerts) may increase the usefulness of ASA in practice.

Objective

The goal of this work is to synthesize available research results to inform evidence-based selection of actionable alert identification techniques (AAIT).

Method

Relevant studies about AAITs were gathered via a systematic literature review.

Results

We selected 21 peer-reviewed studies of AAITs. The techniques use alert type selection; contextual information; data fusion; graph theory; machine learning; mathematical and statistical models; or dynamic detection to classify and prioritize actionable alerts. All of the AAITs are evaluated via an example with a variety of evaluation metrics.

Conclusion

The selected studies support (with varying strength), the premise that the effective use of ASA is improved by supplementing ASA with an AAIT. Seven of the 21 selected studies reported the precision of the proposed AAITs. The two studies with the highest precision built models using the subject program’s history. Precision measures how well a technique identifies true actionable alerts out of all predicted actionable alerts. Precision does not measure the number of actionable alerts missed by an AAIT or how well an AAIT identifies unactionable alerts. Inconsistent use of evaluation metrics, subject programs, and ASAs in the selected studies preclude meta-analysis and prevent the current results from informing evidence-based selection of an AAIT. We propose building on an actionable alert identification benchmark for comparison and evaluation of AAIT from literature on a standard set of subjects and utilizing a common set of evaluation metrics.     

A systematic review of software architecture evolution research

Context

Software evolvability describes a software system’s ability to easily accommodate future changes. It is a fundamental characteristic for making strategic decisions, and increasing economic value of software. For long-lived systems, there is a need to address evolvability explicitly during the entire software lifecycle in order to prolong the productive lifetime of software systems. For this reason, many research studies have been proposed in this area both by researchers and industry practitioners. These studies comprise a spectrum of particular techniques and practices, covering various activities in software lifecycle. However, no systematic review has been conducted previously to provide an extensive overview of software architecture evolvability research.

Objective

In this work, we present such a systematic review of architecting for software evolvability. The objective of this review is to obtain an overview of the existing approaches in analyzing and improving software evolvability at architectural level, and investigate impacts on research and practice.

Method

The identification of the primary studies in this review was based on a pre-defined search strategy and a multi-step selection process.

Results

Based on research topics in these studies, we have identified five main categories of themes: (i) techniques supporting quality consideration during software architecture design, (ii) architectural quality evaluation, (iii) economic valuation, (iv) architectural knowledge management, and (v) modeling techniques. A comprehensive overview of these categories and related studies is presented.

Conclusion

The findings of this review also reveal suggestions for further research and practice, such as (i) it is necessary to establish a theoretical foundation for software evolution research due to the fact that the expertise in this area is still built on the basis of case studies instead of generalized knowledge; (ii) it is necessary to combine appropriate techniques to address the multifaceted perspectives of software evolvability due to the fact that each technique has its specific focus and context for which it is appropriate in the entire software lifecycle.     

Software engineering projects may fail before they are started: Post-mortem analysis of five cancelled projects

Context

Software project cancellations are often caused by mistakes made during the project, and such cancellations make a strong economic impact. We analyzed five cancelled software engineering projects. One case was an internal product development project of a company that sells products to its customers. The other four cases were different software engineering projects, and outcomes of these projects were planned to be delivered to external customers.

Objective

This study reports a post-mortem analysis of five software engineering projects with the aim of providing more knowledge about the reasons for cancellation decisions and the causes behind those reasons.

Methods

The research method is case study. A method for a document-based post-mortem analysis was developed and post-mortem analysis was performed. All project documentation was available for analysis.

Results

The reasons for the cancellation decisions were well-known ones. In four cases of five, the outcome of the project was to be delivered to an external customer, but in these cases the causes of the cancellation reasons were not found from the normal project documentation. In these cases the cause of the cancellation originated in a phase before the start of the project and therefore the project was doomed before it was started.

Conclusion

It is reasonable to suggest that a remarkable portion of project cancellations are due to mistakes made before the project is started in the case of contract-based software engineering projects.     

Reconciling software development models: A quasi-systematic review

Purpose

The purpose of this paper is to characterize reconciliation among the plan-driven, agile, and free/open source software models of software development.

Design/methodology/approach

An automated quasi-systematic review identified 42 papers, which were then analyzed.

Findings

The main findings are: there exist distinct - organization, group and process - levels of reconciliation; few studies deal with reconciliation among the three models of development; a significant amount of work addresses reconciliation between plan-driven and agile development; several large organizations (such as Microsoft, Motorola, and Philips) are interested in trying to combine these models; and reconciliation among software development models is still an open issue, since it is an emerging area and research on most proposals is at an early stage.

Research limitations

Automated searches may not capture relevant papers in publications that are not indexed. Other data sources not amenable to execution of the protocol were not used. Data extraction was performed by only one researcher, which may increase the risk of threats to internal validity.

Implications

This characterization is important for practitioners wanting to be current with the state of research. This review will also assist the scientific community working with software development processes to build a common understanding of the challenges that must be faced, and to identify areas where research is lacking. Finally, the results will be useful to software industry that is calling for solutions in this area.

Originality/value

There is no other systematic review on this subject, and reconciliation among software development models is an emerging area. This study helps to identify and consolidate the work done so far and to guide future research. The conclusions are an important step towards expanding the body of knowledge in the field.     

The impact of inadequate customer collaboration on self-organizing Agile teams

Context

Customer collaboration is a vital feature of Agile software development.

Objective

This article addresses the importance of adequate customer involvement on Agile projects, and the impact of different levels of customer involvement on real-life Agile projects.

Method

We conducted a Grounded Theory study involving 30 Agile practitioners from 16 software development organizations in New Zealand and India, over a period of 3 years.

Results

We discovered that Lack of Customer Involvement was one of the biggest challenges faced by Agile teams. Customers were not as involved on these Agile projects as Agile methods demand. We describe the causes of inadequate customer collaboration, its adverse consequences on self-organizing Agile teams, and Agile Undercover — a set of strategies used by the teams to practice Agile despite insufficient or ineffective customer involvement.

Conclusion

Customer involvement is important on Agile projects. Inadequate customer involvement causes adverse problems for Agile teams. The Agile Undercover strategies we’ve identified can assist Agile teams facing similar lack of customer involvement.     

Towards understanding the underlying structure of motivational factors for software engineers to guide the definition of motivational programs

Aim

In this article, factors influencing the motivation of software engineers is studied with the goal of guiding the definition of motivational programs.

Method

Using a set of 20 motivational factors compiled in a systematic literature review and a general theory of motivation, a survey questionnaire was created to evaluate the influence of these factors on individual motivation. Then, the questionnaire was applied on a semi-random sample of 176 software engineers from 20 software companies located in Recife-PE, Brazil.

Results

The survey results show the actual level of motivation for each motivator in the target population. Using principal component analysis on the values of all motivators, a five factor structure was identified and used to propose a guideline for the creation of motivational programs for software engineers.

Conclusions

The five factor structure provides an intuitive categorization for the set of variables and can be used to explain other motivational models presented in the literature. This contributes to a better understanding of motivation in software engineering.     

Simplifying effort estimation based on Use Case Points

Context

The Use Case Points (UCP) method can be used to estimate software development effort based on a use-case model and two sets of adjustment factors relating to the environmental and technical complexity of a project. The question arises whether all of these components are important from the effort estimation point of view.

Objective

This paper investigates the construction of UCP in order to find possible ways of simplifying it.

Method

The cross-validation procedure was used to compare the accuracy of the different variants of UCP (with and without the investigated simplifications). The analysis was based on data derived from a set of 14 projects for which effort ranged from 277 to 3593 man-hours. In addition, the factor analysis was performed to investigate the possibility of reducing the number of adjustment factors.

Results

The two variants of UCP - with and without unadjusted actor weights (UAW) provided similar prediction accuracy. In addition, a minor influence of the adjustment factors on the accuracy of UCP was observed. The results of the factor analysis indicated that the number of adjustment factors could be reduced from 21 to 6 (2 environmental factors and 4 technical complexity factors). Another observation was made that the variants of UCP calculated based on steps were slightly more accurate than the variants calculated based on transactions. Finally, a recently proposed use-case-based size metric TTPoints provided better accuracy than any of the investigated variants of UCP.

Conclusion

The observation in this study was that the UCP method could be simplified by rejecting UAW; calculating UCP based on steps instead of transactions; or just counting the total number of steps in use cases. Moreover, two recently proposed use-case-based size metrics Transactions and TTPoints could be used as an alternative to UCP to estimate effort at the early stages of software development.     

MC Sandbox: Devising a tool for method-user-centered method configuration

Context

Method engineering approaches are often based on the assumption that method users are able to explicitly express their situational method requirements. Similar to systems requirements, method requirements are often vague and hard to explicate. In this paper we address the issue of involving method users early in method configuration. This is done through borrowing ideas from user-centered design and prototyping, and implementing them on the method engineering layer.

Objective

We design a computerized tool, MC Sandbox, to capture method requirements through the use of method-user-centered method configuration, hence bridging the gap between systems developers’ and method engineers’ understanding of and expectations on a situational method.

Method

The research method adopted can be characterized as multi-grounded action research. Our implementation of multi-grounded action research follows the traditional ‘canonical’ action research method, which has cycles of diagnosing, action planning, action taking, evaluating, and specifying learning. The research project comprised three such action research cycles where 10 action cases were performed.

Results

MC Sandbox has proven useful in eliciting and negotiating method requirements in a continuously ongoing dialog between the method users and the method engineers during configuration workshops. The results also show that the method engineer role rotated among the systems developers and that they were indeed committed to the negotiated methods during the systems development projects.

Conclusion

It is possible for method users to actively participate and construct suitable situational methods if they are provided with appropriate high-level modelling concepts, such as method components, configuration packages and configuration templates. This way, the project members’ understanding of the current development practice develops incrementally, both in terms of understanding the needs and available method support. In addition, both method requirements and commitments are made explicit, which are important aspects when working with method configuration from a collaboration point of view.     

Conceptual scheduling model and optimized release scheduling for agile environments

Context

Release scheduling deals with the selection and assignment of deliverable features to a sequence of consecutive product deliveries while several constraints are fulfilled. Although agile software development represents a major approach to software engineering, there is no well-established conceptual definition and sound methodological support of agile release scheduling.

Objective

To propose a solution, we present, (1) a conceptual model for agile scheduling, and (2) a novel multiple knapsack-based optimization model with (3) a branch-and-bound optimization algorithm for agile release scheduling.

Method

To evaluate our model simulations were carried out seven real life and several generated data sets.

Results

The developed algorithm strives to prevent resource overload and resource underload, and mitigates risks of delivery slippage.

Conclusion

The results of the experiment suggest that this approach can provide optimized semi-automatic release schedule generations and more informed and established decisions utilizing what-if-analysis on the fly to tailor the best schedule for the specific project context.     

Software process improvement success factors for small and medium Web companies: A qualitative study

Context

The context of this research is software process improvement (SPI) in small and medium Web companies.

Objective

The primary objective of this paper is to identify software process improvement (SPI) success factors for small and medium Web companies.

Method

To achieve this goal, we conducted semi-structured, open-ended interviews with 21 participants representing 11 different companies in Pakistan, and analyzed the data qualitatively using the Glaserian strand of grounded theory research procedures. The key steps of these procedures that were employed in this research included open coding, focused coding, theoretical coding, theoretical sampling, constant comparison, and scaling up.

Results

An initial framework of key SPI success factors for small and medium Web companies was proposed, which can be of use for small and medium Web companies engaged in SPI. The paper also differentiates between small and medium Web companies and analyzes crucial SPI requirements for companies operating in the Web development domain.

Conclusion

The results of this work, in particular the use of qualitative techniques - allowed us to obtain rich insight into SPI success factors for small and medium Web companies. Future work comprises the validation of the SPI success factors with small and medium Web companies.     

Transfer learning for cross-company software defect prediction

Context

Software defect prediction studies usually built models using within-company data, but very few focused on the prediction models trained with cross-company data. It is difficult to employ these models which are built on the within-company data in practice, because of the lack of these local data repositories. Recently, transfer learning has attracted more and more attention for building classifier in target domain using the data from related source domain. It is very useful in cases when distributions of training and test instances differ, but is it appropriate for cross-company software defect prediction?

Objective

In this paper, we consider the cross-company defect prediction scenario where source and target data are drawn from different companies. In order to harness cross company data, we try to exploit the transfer learning method to build faster and highly effective prediction model.

Method

Unlike the prior works selecting training data which are similar from the test data, we proposed a novel algorithm called Transfer Naive Bayes (TNB), by using the information of all the proper features in training data. Our solution estimates the distribution of the test data, and transfers cross-company data information into the weights of the training data. On these weighted data, the defect prediction model is built.

Results

This article presents a theoretical analysis for the comparative methods, and shows the experiment results on the data sets from different organizations. It indicates that TNB is more accurate in terms of AUC (The area under the receiver operating characteristic curve), within less runtime than the state of the art methods.

Conclusion

It is concluded that when there are too few local training data to train good classifiers, the useful knowledge from different-distribution training data on feature level may help. We are optimistic that our transfer learning method can guide optimal resource allocation strategies, which may reduce software testing cost and increase effectiveness of software testing process.     

A framework for analysis and design of software reference architectures

Context

A software reference architecture is a generic architecture for a class of systems that is used as a foundation for the design of concrete architectures from this class. The generic nature of reference architectures leads to a less defined architecture design and application contexts, which makes the architecture goal definition and architecture design non-trivial steps, rooted in uncertainty.

Objective

The paper presents a structured and comprehensive study on the congruence between context, goals, and design of software reference architectures. It proposes a tool for the design of congruent reference architectures and for the analysis of the level of congruence of existing reference architectures.

Method

We define a framework for congruent reference architectures. The framework is based on state of the art results from literature and practice. We validate our framework and its quality as analytical tool by applying it for the analysis of 24 reference architectures. The conclusions from our analysis are compared to the opinions of experts on these reference architectures documented in literature and dedicated communication.

Results

Our framework consists of a multi-dimensional classification space and of five types of reference architectures that are formed by combining specific values from the multi-dimensional classification space. Reference architectures that can be classified in one of these types have better chances to become a success. The validation of our framework confirms its quality as a tool for the analysis of the congruence of software reference architectures.

Conclusion

This paper facilitates software architects and scientists in the inception, design, and application of congruent software reference architectures. The application of the tool improves the chance for success of a reference architecture.     

Measuring and predicting software productivity: A systematic map and review

Context

Software productivity measurement is essential in order to control and improve the performance of software development. For example, by identifying role models (e.g. projects, individuals, tasks) when comparing productivity data. The prediction is of relevance to determine whether corrective actions are needed, and to discover which alternative improvement action would yield the best results.

Objective

In this study we identify studies for software productivity prediction and measurement. Based on the identified studies we first create a classification scheme and map the studies into the scheme (systematic map). Thereafter, a detailed analysis and synthesis of the studies is conducted.

Method

As a research method for systematically identifying and aggregating the evidence of productivity measurement and prediction approaches systematic mapping and systematic review have been used.

Results

In total 38 studies have been identified, resulting in a classification scheme for empirical research on software productivity. The mapping allowed to identify the rigor of the evidence with respect to the different productivity approaches. In the detailed analysis the results were tabulated and synthesized to provide recommendations to practitioners.

Conclusion

Risks with simple ratio-based measurement approaches were shown. In response to the problems data envelopment analysis seems to be a strong approach to capture multivariate productivity measures, and allows to identify reference projects to which inefficient projects should be compared. Regarding simulation no general prediction model can be identified. Simulation and statistical process control are promising methods for software productivity prediction. Overall, further evidence is needed to make stronger claims and recommendations. In particular, the discussion of validity threats should become standard, and models need to be compared with each other.