Engaging the net generation with evidence-based software engineering through a community-driven web database

Software engineering faculty face the challenge of educating future researchers and industry practitioners regarding the generation of empirical software engineering studies and their use in evidence-based software engineering. In order to engage the Net generation with this topic, we propose development and population of a community-driven Web database containing summaries of empirical software engineering studies. We also present our experience with integrating these activities into a graduate software engineering course. These efforts resulted in the creation of “SEEDS: Software Engineering Evidence Database System”. Graduate students initially populated SEEDS with 216 summaries of empirical software engineering studies. The summaries were randomly sampled and reviewed by industry professionals who found the student-written summaries to be at least as useful as professional-written summaries. In fact, 30% more of the respondents found the student-written summaries to be “very useful”. Motivations, student and instructor-developed prototypes, and assessments of the resulting artifacts will be discussed.     

A case study comparing defect profiles of a reused framework and of applications reusing it

The benefits of software reuse have been studied for many years. Several previous studies have observed that reused software has a lower defect density than newly built software. However, few studies have investigated empirically the reasons for this phenomenon. To date, we have only the common sense observation that as software is reused over time, the fixed defects will accumulate and will result in high-quality software. This paper reports on an industrial case study in a large Norwegian Oil and Gas company, involving a reused Java class framework and two applications that use that framework. We analyzed all trouble reports from the use of the framework and the applications according to the Orthogonal Defect Classification (ODC), followed by a qualitative Root Cause Analysis (RCA). The results reveal that the framework has a much lower defect density in total than one application and a slightly higher defect density than the other. In addition, the defect densities of the most severe defects of the reused framework are similar to those of the applications that are reusing it. The results of the ODC and RCA analyses reveal that systematic reuse (i.e. clearly defined and stable requirements, better design, hesitance to change, and solid testing) lead to lower defect densities of the functional-type defects in the reused framework than in applications that are reusing it. However, the different “nature” of the framework and the applications (e.g. interaction with other software, number and complexity of business logic, and functionality of the software) may confound the causal relationship between systematic reuse and the lower defect density of the reused software. Using the results of the study as a basis, we present an improved overall cause–effect model between systematic reuse and lower defect density that will facilitate further studies and implementations of software reuse.

An approach to optimizing software development team size

This paper explores the relationship between software size, development effort and team size. We propose an approach aimed at finding the team size where the project effort has its minimum. The approach was applied to the ISBSG repository containing nearly 4000 software projects. Based on the results we provide our recommendation for the optimal or near-optimal team size in seven project groups defined by four project properties.     

Towards understanding the underlying structure of motivational factors for software engineers to guide the definition of motivational programs

Aim

In this article, factors influencing the motivation of software engineers is studied with the goal of guiding the definition of motivational programs.

Method

Using a set of 20 motivational factors compiled in a systematic literature review and a general theory of motivation, a survey questionnaire was created to evaluate the influence of these factors on individual motivation. Then, the questionnaire was applied on a semi-random sample of 176 software engineers from 20 software companies located in Recife-PE, Brazil.

Results

The survey results show the actual level of motivation for each motivator in the target population. Using principal component analysis on the values of all motivators, a five factor structure was identified and used to propose a guideline for the creation of motivational programs for software engineers.

Conclusions

The five factor structure provides an intuitive categorization for the set of variables and can be used to explain other motivational models presented in the literature. This contributes to a better understanding of motivation in software engineering.     

An empirical study of the effect of complexity, platform, and program type on software development effort of business applications

Several popular cost estimation models like COCOMO and function points use adjustment variables, such as software complexity and platform, to modify original estimates and arrive at final estimates. Using data on 666 programs from 15 software projects, this study empirically tests a research model that studies the influence of three adjustment variables—software complexity, computer platform, and program type (batch or online programs) on software effort. The results confirm that all the three adjustment variables have a significant effect on effort. Further, multiple comparison of means also points to two other results for the data examined. Batch programs involve significantly higher software effort than online programs. Programs rated as complex have significantly higher effort than programs rated as average.     

Identification of defect-prone classes in telecommunication software systems using design metrics

The goal of this paper is to investigate the relation between object-oriented design choices and defects in software systems, with focus on a real-time telecommunication domain. The design choices are measured using the widely accepted metrics suite proposed by Chidamber and Kemerer for object oriented languages [S.R. Chidamber, C.F. Kemerer, A metrics suite for object oriented design, IEEE Transactions on Software Engineering 20 (6) (1994) 476-493].This paper reports the results of an extensive case study, which strongly reinforces earlier, mainly anecdotal, evidence that design aspects related to communication between classes can be used as indicators of the most defect-prone classes.Statistical models applicable for the non-normally distributed count data are used, such as Poisson regression, negative binomial regression, and zero-inflated negative binomial regression. The performances of the models are assessed using correlations, dispersion coefficients and Alberg diagrams.The zero-inflated negative binomial regression model based on response for a class shows the best overall ability to describe the variability of the number of defects in classes.     

An empirical study on the implementation and evaluation of a goal-driven software development risk management model

ContextBuilding a quality software product in the shortest possible time to satisfy the global market demand gives an enterprise a competitive advantage. However, uncertainties and risks exist at every stage of a software development project. These can have an extremely high influence on the success of the final software product. Early risk management practice is effective to manage such risks and contributes effectively towards the project success.ObjectiveDespite risk management approaches, a detailed guideline that explains where to integrate risk management activities into the project is still missing. Little effort has been directed towards the evaluation of the overall impact of a risk management method. We present a Goal-driven Software Development Risk Management Model (GSRM) and its explicit integration into the requirements engineering phase and an empirical investigation result of applying GSRM into a project.MethodWe combine the case study method with action research so that the results from the case study directly contribute to manage the studied project risks and to identify ways to improve the proposed methodology. The data is collected from multiple sources and analysed both in a qualitative and quantitative way.ResultsWhen risk factors are beyond the control of the project manager and project environment, it is difficult to control these risks. The project scope affects all the dimensions of risk. GSRM is a reasonable risk management method that can be employed in an industrial context. The study results have been compared against other study results in order to generalise findings and identify contextual factors.ConclusionA formal early stage risk management practice provides early warning related to the problems that exists in a project, and it contributes to the overall project success. It is not necessary to always consider budget and schedule constraints as top priority. There exist issues such as requirements, change management, and user satisfaction which can influence these constraints.     

An empirical study on students’ ability to comprehend design patterns

Design patterns have become a widely acknowledged software engineering practice and therefore have been incorporated in the curricula of most computer science departments. This paper presents an observational study on students’ ability to understand and apply design patterns. Within the context of a postgraduate software engineering course, students had to deliver two versions of a software system; one without and one with design patterns. The former served as a poorly designed system suffering from architectural problems, while the latter served as an improved system where design problems had been solved by appropriate patterns. The experiment allowed the quantitative evaluation of students’ preference to patterns. Moreover, it was possible to assess students’ ability in relating design problems with patterns and interpreting the impact of patterns on software metrics. The overall goal was to empirically identify ways in which a course on design patterns could be improved.     

A study of software reuse in NASA legacy systems

Software reuse is regarded as a highly important factor in reducing development overheads for new software projects; however, much of the literature is concerned with cost and labor savings that reuse brings to industrial software development and little is known about the inherent risks associated with reuse, particularly in the case of mission and safety-critical software systems. We present the preliminary findings of a research project geared toward assessing the impact of risk in National Aeronautics and Space Administration (NASA) legacy software in flight control systems. We introduce the concept of context variables and the impact they have on reuse within these legacy systems as well as the genealogy classification models, which provide a simple, concise method of mapping reuse between families of software projects. This research was conducted at Global Science and Technology, Inc. under NASA grant number NCC0NNG06GI57G.     

Experimenting with software testbeds for evaluating new technologies

The evolution of a new technology depends upon a good theoretical basis for developing the technology, as well as upon its experimental validation. In order to provide for this experimentation, we have investigated the creation of a software testbed and the feasibility of using the same testbed for experimenting with a broad set of technologies. The testbed is a set of programs, data, and supporting documentation that allows researchers to test their new technology on a standard software platform. An important component of this testbed is the Unified Model of Dependability (UMD), which was used to elicit dependability requirements for the testbed software. With a collection of seeded faults and known issues of the target system, we are able to determine if a new technology is adept at uncovering defects or providing other aids proposed by its developers. In this paper, we present the Tactical Separation Assisted Flight Environment (TSAFE) testbed environment for which we modeled and evaluated dependability requirements and defined faults to be seeded for experimentation. We describe two completed experiments that we conducted on the testbed. The first experiment studies a technology that identifies architectural violations and evaluates its ability to detect the violations. The second experiment studies model checking as part of design for verification. We conclude by describing ongoing experimental work studying testing, using the same testbed. Our conclusion is that even though these three experiments are very different in terms of the studied technology, using and re-using the same testbed is beneficial and cost effective.

Research synthesis in software engineering: A tertiary study

Context

Comparing and contrasting evidence from multiple studies is necessary to build knowledge and reach conclusions about the empirical support for a phenomenon. Therefore, research synthesis is at the center of the scientific enterprise in the software engineering discipline.

Objective

The objective of this article is to contribute to a better understanding of the challenges in synthesizing software engineering research and their implications for the progress of research and practice.

Method

A tertiary study of journal articles and full proceedings papers from the inception of evidence-based software engineering was performed to assess the types and methods of research synthesis in systematic reviews in software engineering.

Results

As many as half of the 49 reviews included in the study did not contain any synthesis. Of the studies that did contain synthesis, two thirds performed a narrative or a thematic synthesis. Only a few studies adequately demonstrated a robust, academic approach to research synthesis.

Conclusion

We concluded that, despite the focus on systematic reviews, there is limited attention paid to research synthesis in software engineering. This trend needs to change and a repertoire of synthesis methods needs to be an integral part of systematic reviews to increase their significance and utility for research and practice.     

The lean gap: A review of lean approaches to large-scale software systems development

Lean approaches to product development (LPD) have had a strong influence on many industries and in recent years there have been many proponents for lean in software development as it can support the increasing industry need of scaling agile software development. With it's roots in industrial manufacturing and, later, industrial product development, it would seem natural that LPD would adapt well to large-scale development projects of increasingly software-intensive products, such as in the automotive industry. However, it is not clear what kind of experience and results have been reported on the actual use of lean principles and practices in software development for such large-scale industrial contexts. This was the motivation for this study as the context was an ongoing industry process improvement project at Volvo Car Corporation and Volvo Truck Corporation.     

An experimental evaluation of capture-recapture in software inspections

The use of capture-recapture to estimate the residual faults in a software artifact has evolved as a promising method. However, the assumptions needed to make the estimates are not completely fulfilled in software development, leading to an underestimation of the residual fault content. Therefore, a method employing a filtering technique with an experience factor to improve the estimate of the residual faults is proposed in this paper. An experimental study of the capture-recapture method with this correction method has been conducted. It is concluded that the correction method improves the capture-recapture estimate of the number of residual defects in the inspected document.     

A multi-level analysis of factors affecting software developers’ intention to reuse software assets: An empirical investigation

Reuse of software assets in application development has held promise but faced challenges. In addressing these challenges, research has focused on organizational- and project-level factors while neglecting grass-root level adoption of reusable assets. Our research investigated factors associated with individual software developers’ intention to reuse software assets and integrated them in TAM. Towards that end, 13 project managers were interviewed and 207 software developers were surveyed in India. Results revealed that the technological-level (infrastructure), and individual-level factors (reuse-related experience and self-efficacy) were major determinants. Implications are discussed.     

Requirements engineering challenges in market-driven software development – An interview study with practitioners

Requirements engineering for market-driven software development entails special challenges. This paper presents results from an empirical study that investigates these challenges, taking a qualitative approach using interviews with fourteen employees at eight software companies and a focus group meeting with practitioners. The objective of the study is to increase the understanding of the area of market-driven requirements engineering and provide suggestions for future research by describing encountered challenges. A number of challenging issues were found, including bridging communication gaps between marketing and development, selecting the right level of process support, basing the release plan on uncertain estimates, and managing the constant flow of requirements.     

Integrated model for software component selection with simultaneous consideration of implementation and verification

One important objective of component-based software engineering is the minimization of the development cost of software products. Thus, the costs of software component implementation and verification, which may involve substantial expenses while under development, should be reduced. In addition, the costs for these processes should not be considered individually, but in an integrated manner, to further reduce development cost. In the current paper, an integrated decision model is proposed to assist decision-makers in selecting reuse scenarios for components used for implementation and in simultaneously determining the optimal number of test cases for verification. An objective of the model is the minimization of development cost, while satisfying the required system and reliability requirements. The Lagrange relaxation decomposition (LRD) method with heuristics was developed to solve integrated decision problems. Based on LRD, the nonlinear model is condensed into a 0–1 knapsack problem for the subproblem on reuse scenario selection and an integer knapsack problem for the subproblem on the determination of the optimal number of tests. Combined with the Lagrange multiplier-determined heuristic, the proposed algorithm can determine the global optimum solution. Simulations of varying sizes for problems and sensitivity analyses were conducted, and the results indicate that LRD is more effective than previous methods in determining global optimal solutions for the integrated decision problem.