Identifying high perceived value practices of CMMI level 2: An empirical study

ObjectiveIn this paper, we present findings from an empirical study that was aimed at identifying the relative “perceived value” of CMMI level 2 specific practices based on the perceptions and experiences of practitioners of small and medium size companies. The objective of this study is to identify the extent to which a particular CMMI practice is used in order to develop a finer-grained framework, which encompasses the notion of perceived value within specific practices.MethodWe used face-to-face questionnaire based survey sessions as the main approach to collecting data from 46 software development practitioners from Malaysia and Vietnam. We asked practitioners to choose and rank CMMI level 2 practices against the five types of assessments (high, medium, low, zero or do not know). From this, we have proposed the notion of ‘perceived value’ associated with each practice.ResultsWe have identified three ‘requirements management’ practices as having a ‘high perceived value’. The results also reveal the similarities and differences in the perceptions of Malaysian and Vietnamese practitioners with regard to the relative values of different practices of CMMI level 2 process areas.ConclusionsSmall and medium size companies should not be seen as being “at fault” for not adopting CMMI – instead the Software Process Improvement (SPI) implementation approaches and its transition mechanisms should be improved. We argue that research into “tailoring” existing process capability maturity models may address some of the issues of small and medium size companies.     

Towards a theoretical framework of SPI success factors for small and medium web companies

ContextThe context of this research is software process improvement (SPI) success factors for small and medium Web companies.ObjectiveThe primary objective of this paper is to propose a theoretical framework of SPI success factors for small and medium Web companies.MethodThe theoretical framework presented in this study aggregated the results of three previous research phases by applying principles of theoretical integration and comparative analysis. Those three previous phases were all empirical in nature, and comprise: a systematic review of SPI in small and medium Web companies [1], [2]; a replication study [3] and a grounded theory-based initial exploratory framework of factors in small and medium Web companies [4].ResultsThe theoretical framework includes 18 categories of SPI success factors, 148 properties of these categories and 25 corresponding relationships, which bind these categories together. With the help of these relationships, the categories and properties of SPI success factors can be directly translated into a set of guidelines, which can then be used by the practitioners of small and medium Web companies to improve the current state of SPI in their companies and achieve overall company success.ConclusionThe comprehensive theoretical framework of SPI success factors presented herein provides evidence regarding key factors for predicting SPI success for small and medium Web companies. The framework can be used as a baseline for a successful implementation of SPI initiatives in the mentioned domain.     

Software process improvement with weak management support: an analysis of the dynamics of intra-organizational alliances in IS change initiatives

Software Process Improvement (SPI) projects are large-scale, complex organization-wide change initiatives. They require considerable investments in personnel, time and money and impact just about every aspect of software firms. The group charged with conducting an SPI project has, however, little formal authority to influence or force software professionals to engage in SPI work or to define and implement changes. The SPI literature suggests that successful SPI initiatives depend on strong commitment from top management. But what should the SPI group do if management support is weak? In this paper, we present an analysis of how an SPI group can use alliances to obtain influence and succeed when management support is weak. Our study is based on a 3-year longitudinal field study of SPI change initiatives at Denmark Electronics. Our findings show that a lack of top management support is not necessarily incompatible with success. This research opens an important new area of research on intra-organizational alliances and information system (IS) implementation. It has the potential to offer new theories and practical advice on how IS implementation projects can be more effectively managed.     

Applying grounded theory to understand software process improvement implementation: a study of Brazilian software organizations

Despite the fact that organizations recognize the importance of implementing software process improvement (SPI) to enhance software development capabilities, recent studies show that many organizations still struggle to successfully implement SPI. These studies demonstrate that most SPI initiatives fail due to their incapacity to overcome social–cultural critical barriers, such as lack of motivation and higher management support. Therefore, the accurate understanding of the context where SPI occurs may facilitate the development of more efficient SPI implementation strategies aiming to overcome those difficulties. The objective of this work was to present a Grounded Theory-based study that investigated SPI implementation initiatives conducted by Brazilian software organizations. The main product of this study is a theoretical framework that tries and explains the phenomenon associated to the success of SPI implementation initiatives. The theoretical framework constituted a set of interrelated SPI domain entities supported by a set of propositions assessed by experienced Brazilian SPI consultants. We expect that such theoretical framework can facilitate SPI implementation by providing better understating of the dependency relationships among critical success factors that may jeopardize SPI initiatives.     

A systematic literature review on the barriers faced by newcomers to open source software projects

ContextNumerous open source software projects are based on volunteers collaboration and require a continuous influx of newcomers for their continuity. Newcomers face barriers that can lead them to give up. These barriers hinder both developers willing to make a single contribution and those willing to become a project member.ObjectiveThis study aims to identify and classify the barriers that newcomers face when contributing to open source software projects.MethodWe conducted a systematic literature review of papers reporting empirical evidence regarding the barriers that newcomers face when contributing to open source software (OSS) projects. We retrieved 291 studies by querying 4 digital libraries. Twenty studies were identified as primary. We performed a backward snowballing approach, and searched for other papers published by the authors of the selected papers to identify potential studies. Then, we used a coding approach inspired by open coding and axial coding procedures from Grounded Theory to categorize the barriers reported by the selected studies.ResultsWe identified 20 studies providing empirical evidence of barriers faced by newcomers to OSS projects while making a contribution. From the analysis, we identified 15 different barriers, which we grouped into five categories: social interaction, newcomers’ previous knowledge, finding a way to start, documentation, and technical hurdles. We also classified the problems with regard to their origin: newcomers, community, or product.ConclusionThe results are useful to researchers and OSS practitioners willing to investigate or to implement tools to support newcomers. We mapped technical and non-technical barriers that hinder newcomers’ first contributions. The most evidenced barriers are related to socialization, appearing in 75% (15 out of 20) of the studies analyzed, with a high focus on interactions in mailing lists (receiving answers and socialization with other members). There is a lack of in-depth studies on technical issues, such as code issues. We also noticed that the majority of the studies relied on historical data gathered from software repositories and that there was a lack of experiments and qualitative studies in this area.     

Bridges and barriers to hardware-dependent software ecosystem participation – A case study

BackgroundSoftware ecosystems emerged as means for several actors to jointly provide more value to the market than any of them can do on its own. Recently, software ecosystems are more often used to support the development of hardware-dependent solutions.ObjectivesThis work aims at studying barriers and bridges to participation in an ecosystem with substantial hardware dependencies.MethodWe conducted an interview-based case study of an ecosystem around Axis’ network video surveillance systems, interviewing 10 internal experts and 8 external representatives of 6 companies, complemented by document studies at Axis.ResultsMajor bridges to the ecosystem include end customer demands, open and transparent communication and relationship, as well as internal and external standardizations. Barriers include the two-tier business model, entry barriers and execution performance issues. Approximately half of the identified bridges and barriers could be considered hardware-dependent ecosystems specific.ConclusionOur results suggest that ecosystem leaders should share their sales channels with the ecosystem participants and focus on good communication and relationships as the dominant factors for the ecosystem participation. Moreover, we report that internal and external standardization can play a dual role, not only ease the development but also enable additional sales channels and new opportunities for the ecosystem participants. At the same time, the business model selected by the ecosystem leaders and performance, are identified as the main barriers to ecosystem participation. We believe that the business model barrier may be much more important for similar hardware-dependent software ecosystems.     

Barriers in the selection of offshore software development outsourcing vendors: An exploratory study using a systematic literature review

Context

Software development outsourcing is a contract-based relationship between client and vendor organisations in which a client contracts out all or part of its software development activities to a vendor, who provides agreed services for remuneration.

Objective

The objective is to identify various barriers that have a negative impact on software outsourcing clients in the selection process of offshore software development outsourcing vendors.

Method

We have performed a systematic literature review (SLR) process for the identification of barriers. We have performed all the SLR steps such as the protocol development, initial selection, final selection, quality assessment, data extraction and data synthesis.

Results

We have identified barriers such as ‘language and cultural barriers’, ‘country instability’, ‘lack of project management’, ‘lack of protection for intellectual property rights’ and ‘lack of technical capability’ that generally have a negative impact on outsourcing clients. We have identified only one common frequently cited barrier in three types of organisations (i.e. small, medium and large) which is ‘language and cultural barriers’. We did not identify any common frequently cited barrier in three continents (Asia, North America and Europe) and in two decades (1990-1999 and 2000-mid 2008). The results also reveal the similarities and differences in the barriers identified through different study strategies.

Conclusions

Vendors should address frequently cited barriers such as ‘language and cultural barriers’, ‘country instability’, ‘lack of project management’, ‘lack of protection for intellectual property rights’ and ‘lack of technical capability’ in order to compete in the offshore outsourcing business.     

Defining a Requirements Process Improvement Model

Both software organisations and the academic community are aware that the requirements phase of software development is in need of further support. We address this problem by creating a specialised Requirements Capability Maturity Model (R-CMM¹). The model focuses on the requirements engineering process as defined within the established Software Engineering Institute’s (SEI’s) software process improvement framework. Our empirical work with software practitioners is a primary motivation for creating this requirements engineering process improvement model. Although all organisations in our study were involved in software process improvement (SPI), they all showed a lack of control over many requirement engineering activities.This paper describes how the requirements engineering (RE) process is decomposed and prioritised in accordance with maturity goals set by the SEI’s Software Capability Maturity Model (SW CMM). Our R-CMM builds on the SEI’s framework by identifying and defining recommended RE sub-processes that meet maturity goals. This new focus will help practitioners to define their RE process with a view to setting realistic goals for improvement.Sarah Beecham is a research fellow in the Department of Maths and Computing in The Open University in the UK. She is currently working on the EPSRC funded CRESTES project () looking into modelling resource estimation for long-lived software. She has recently completed her PhD for a program of work entitled “A Requirements-based Software Process Maturity Model”. Current research interests are in estimation for software evolution and maintenance and in the general areas of software process improvement. Her particular research interests are in empirical methods in software engineering and requirements engineering.Tracy Hall leads the Systems & Software Research Group in the Department of Computer Science at the University of Hertfordshire. She specialises in the empirical investigation of technical and non-technical issues within software engineering. During the past ten years Tracy has successfully collaborated with many companies on a variety of research projects. She is very active in the Empirical Software Engineering community and is regularly invited to talk about empirical methods both in the UK and abroad. Tracy is an accomplished researcher having published over twenty high quality journal papers.Austen Rainer Austen Rainer is a senior lecturer at the University of Hertfordshire. He studied for his PhD at Bournemouth University, in conjunction with IBM Hursley Park. His current research interests include open source software development, longitudinal case study research, and the credibility of empirical evidence for researchers and software practitioners.     

Software evolution visualization: A systematic mapping study

BackgroundSoftware evolution is an important topic in software engineering. It generally deals with large amounts of data, as one must look at whole project histories as opposed to their current snapshot. Software visualization is the field of software engineering that aims to help people to understand software through the use of visual resources. It can be effectively used to analyze and understand the large amount of data produced during software evolution.ObjectiveThis study investigates Software Evolution Visualization (SEV) approaches, collecting evidence about how SEV research is structured, synthesizing current evidence on the goals of the proposed approaches and identifying key challenges for its use in practice.MethodsA mapping study was conducted to analyze how the SEV area is structured. Selected primary studies were classified and analyzed with respect to nine research questions.ResultsSEV has been used for many different purposes, especially for change comprehension, change prediction and contribution analysis. The analysis identified gaps in the studies with respect to their goals, strategies and approaches. It also pointed out to a widespread lack of empirical studies in the area.ConclusionResearchers have proposed many SEV approaches during the past years, but some have failed to clearly state their goals, tie them back to concrete problems, or formally validate their usefulness. The identified gaps indicate that there still are many opportunities to be explored in the area.     

Software development in startup companies: A systematic mapping study

ContextSoftware startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present a unique combination of characteristics which pose several challenges to software development activities.ObjectiveThis study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers.MethodWe conducted a systematic mapping study, developing a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups.ResultsA total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed.ConclusionThis mapping study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.     

SPI Models: What Characteristics are Required for Small Software Development Companies?

There is a need for small indigenous software companies to improve their software process. Consequently, much has been written highlighting the deficiencies in the more popular Software Process Improvement (SPI) models where the small company is concerned. However, there has been little discussion about the characteristics that should be included in SPI models to make them useful for the small company. In this paper, the author proposes an SPI model for use in small software development companies. The eight characteristics that were required to exist in the model are examined and reasons for their successful inclusion presented.     

Testing scientific software: A systematic literature review

ContextScientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract publications due to errors caused by software faults. Systematic testing can identify such faults in code.ObjectiveThis study aims to identify specific challenges, proposed solutions, and unsolved problems faced when testing scientific software.MethodWe conducted a systematic literature survey to identify and analyze relevant literature. We identified 62 studies that provided relevant information about testing scientific software.ResultsWe found that challenges faced when testing scientific software fall into two main categories: (1) testing challenges that occur due to characteristics of scientific software such as oracle problems and (2) testing challenges that occur due to cultural differences between scientists and the software engineering community such as viewing the code and the model that it implements as inseparable entities. In addition, we identified methods to potentially overcome these challenges and their limitations. Finally we describe unsolved challenges and how software engineering researchers and practitioners can help to overcome them.ConclusionsScientific software presents special challenges for testing. Specifically, cultural differences between scientist developers and software engineers, along with the characteristics of the scientific software make testing more difficult. Existing techniques such as code clone detection can help to improve the testing process. Software engineers should consider special challenges posed by scientific software such as oracle problems when developing testing techniques.     

Toward the characterization of software testing practices in South America: looking at Brazil and Uruguay

Software testing is an important activity in the software development life cycle. Several previous studies reported the results of surveys on software testing practices among practitioners from different countries. In this paper, we analyze these surveys aiming to get their main questions, and replicate a survey with practitioners from Brazil and Uruguay, two emerging South American software development scenarios. This survey was previously conducted in Manaus/Brazil in 2006 and Buenos Aires/Argentina in 2013. The replication’s scope includes three regions (Northern Brazil, Southern Brazil, and Uruguay). A total of 150 software testing practitioners responded to the survey. Its results are compared with the previous executions and other software testing surveys identified in the technical literature, strengthening previous findings. The Brazilian and Uruguayan participants indicate that: (1) documentation of test artifacts (plan, cases, procedures, results) are useful and important for software testing practitioners; (2) system and regression testing are the two test types deemed most useful and important; (3) tools for monitoring and management of test process tasks and bug reports are considered useful and important; (4) it is usual for software companies to have a definition of a testing process and to have a dedicated testing team; (5) there is a lack of measurement of test tasks and coverage in the industry; and (6) tools to support automation of test case generation and execution or code coverage are still poorly used in their organizations.     

A practitioner’s guide to light weight software process assessment and improvement planning

Software process improvement (SPI) is challenging, particularly for small and medium sized enterprises. Most existing SPI frameworks are either too expensive to deploy, or do not take an organizations’ specific needs into consideration. There is a need for light weight SPI frameworks that enable practitioners to base improvement efforts on the issues that are the most critical for the specific organization.

This paper presents a step-by-step guide to process assessment and improvement planning using improvement framework utilizing light weight assessment and improvement planning (iFLAP), aimed at practitioners undertaking SPI initiatives. In addition to the guide itself the industrial application of iFLAP is shown through two industrial cases. iFLAP is a packaged improvement framework, containing both assessment and improvement planning capabilities, explicitly developed to be light weight in nature. Assessment is performed by eliciting improvements issues based on the organization’s experience and knowledge. The findings are validated through triangulation utilizing multiple data sources. iFLAP actively involves practitioners in prioritizing improvement issues and identifying dependencies between them in order to package improvements, and thus establish a, for the organization, realistic improvement plan. The two cases of iFLAP application in industry are presented together with lessons learned in order to exemplify actual use of the framework as well as challenges encountered.     

CIP-UQIM: A unified model for quality improvement in software SME's based on CMMI level 2 and 3

ContextSoftware Process Improvement (SPI) is among the most effective ways to improve the quality of software products and services. Despite many research and industrial reports, SPI planning and implementation in Multi-Model Environments (MMEs) still face various issues and difficulties which increase the risk of failure especially for Small and Medium size Enterprises (SMEs). As a popular approach, harmonization techniques are used to meet the MME's specific issues through recognizing the common or similar areas among the implemented models and developing an integrated solution for quality improvement throughout the organization.ObjectiveThis paper has two main objectives: (1) recognizing the set of most popular models used in worldwide harmonization projects and implemented or requested by Iranian software SMEs. (2) Comparing and integrating these models in Process Activity level to develop a unified quality improvement model.MethodA combination of literature review and questionnaire methods was used to identify the set of the most popular models. Then, based on harmonization techniques the common or similar areas among these models are recognized and then an initial version of the Unified Quality Improvement Model (UQIM) is developed, named CIP-UQIM. Next, this initial model was presented in a two-day workshop for SPI experts and it was refined and finalized by incorporating their feedbacks.ResultsInvestigations showed that the set {CMMI-DEV, ISO 9001, and PMBOK} was the most popular. Thus in CIP-UQIM, for the first time, the last version of these models was unified at the activity level details. Finally, the experience of using CIP-UQIM in an Iranian SME has been reported which demonstrate its applicability and advantages.ConclusionConsidering the high similarity among the three models and CIP-UQIM level of details, it can be concluded that CIP-UQIM can be beneficial to resolve or reduce SPI issues in MME's, especially in case of software SMEs.     

Managing change in software process improvement

Software process improvement has become the primary approach to improving software quality and reliability, employee and customer satisfaction, and return on investment. Although the literature acknowledges that SPI implementation faces various problems, most published cases report success, detailing dramatic improvements. Such best-practice cases are a great benefit when learning how to effectively implement SPI. On the basis of experiences from SPI initiatives and insights into organizational-change management, we offer the following advice for successful SPI implementation: software managers must appreciate that each SPI initiative is unique and carefully negotiate the context of change. Managers must also understand the elements of change involved. SPI can't succeed without managerial commitment and a mastery of appropriate change tactics.     

A systematic review of machine learning techniques for software fault prediction

BackgroundSoftware fault prediction is the process of developing models that can be used by the software practitioners in the early phases of software development life cycle for detecting faulty constructs such as modules or classes. There are various machine learning techniques used in the past for predicting faults.MethodIn this study we perform a systematic review of studies from January 1991 to October 2013 in the literature that use the machine learning techniques for software fault prediction. We assess the performance capability of the machine learning techniques in existing research for software fault prediction. We also compare the performance of the machine learning techniques with the statistical techniques and other machine learning techniques. Further the strengths and weaknesses of machine learning techniques are summarized.ResultsIn this paper we have identified 64 primary studies and seven categories of the machine learning techniques. The results prove the prediction capability of the machine learning techniques for classifying module/class as fault prone or not fault prone. The models using the machine learning techniques for estimating software fault proneness outperform the traditional statistical models.ConclusionBased on the results obtained from the systematic review, we conclude that the machine learning techniques have the ability for predicting software fault proneness and can be used by software practitioners and researchers. However, the application of the machine learning techniques in software fault prediction is still limited and more number of studies should be carried out in order to obtain well formed and generalizable results. We provide future guidelines to practitioners and researchers based on the results obtained in this work.     

IT Service Management Process Improvement based on ISO/IEC 15504: A systematic review

Context

In recent years, many software companies have considered Software Process Improvement (SPI) as essential for successful software development. These companies have also shown special interest in IT Service Management (ITSM). SPI standards have evolved to incorporate ITSM best practices.

Objective

This paper presents a systematic literature review of ITSM Process Improvement initiatives based on the ISO/IEC 15504 standard for process assessment and improvement.

Method

A systematic literature review based on the guidelines proposed by Kitchenham and the review protocol template developed by Biolchini et al. is performed.

Results

Twenty-eight relevant studies related to ITSM Process Improvement have been found. From the analysis of these studies, nine different ITSM Process Improvement initiatives have been detected. Seven of these initiatives use ISO/IEC 15504 conformant process assessment methods.

Conclusion

During the last decade, in order to satisfy the on-going demand of mature software development companies for assessing and improving ITSM processes, different models which use the measurement framework of ISO/IEC 15504 have been developed. However, it is still necessary to define a method with the necessary guidelines to implement both software development processes and ITSM processes reducing the amount of effort, especially because some processes of both categories are overlapped.     

Knowledge management initiatives in software testing: A mapping study

ContextSoftware testing is a knowledge intensive process, and, thus, Knowledge Management (KM) principles and techniques should be applied to manage software testing knowledge.ObjectiveThis study conducts a survey on existing research on KM initiatives in software testing, in order to identify the state of the art in the area as well as the future research. Aspects such as purposes, types of knowledge, technologies and research type are investigated.MethodThe mapping study was performed by searching seven electronic databases. We considered studies published until December 2013. The initial resulting set was comprised of 562 studies. From this set, a total of 13 studies were selected. For these 13, we performed snowballing and direct search to publications of researchers and research groups that accomplished these studies.ResultsFrom the mapping study, we identified 15 studies addressing KM initiatives in software testing that have been reviewed in order to extract relevant information on a set of research questions.ConclusionsAlthough only a few studies were found that addressed KM initiatives in software testing, the mapping shows an increasing interest in the topic in the recent years. Reuse of test cases is the perspective that has received more attention. From the KM point of view, most of the studies discuss aspects related to providing automated support for managing testing knowledge by means of a KM system. Moreover, as a main conclusion, the results show that KM is pointed out as an important strategy for increasing test effectiveness, as well as for improving the selection and application of suited techniques, methods and test cases. On the other hand, inadequacy of existing KM systems appears as the most cited problem related to applying KM in software testing.