What Do We Know about Agile Software Development?

From 1,996 studies available in the agile software development literature, the authors identified 36 research studies of acceptable rigor, credibility, and relevance to include in a systematic review of empirical evidence for agile development's benefits and limitations. The review results suggest that despite some limitations, agile development can improve job satisfaction, project productivity, and customer satisfaction.     

持续集成测试用例集优化综述研究

基于互联网的软件开发要求产品快速迭代,同时保持产品的质量,其关键的环节就是持续集成.持续集成通过自动化测试来保证集成到主干的代码质量.持续集成时的测试用例选择是一个非常有挑战性的问题.如果运行所有的测试用例,需要消耗大量的计算资源,并造成测试反馈周期过长.如果选择的测试用例集不合适,又不足以覆盖必要的测试代码以保证待集成代码的质量.持续集成测试用例集优化的目的是平衡测试资源和测试质量,在尽可能不影响测试质量的情况下,减少持续集成的资源需求.对近年来国内外学者在该领域的研究工作进行了系统的分析、提炼和总结.为此,首先从研究主题、影响因子、研究方法、研究对象以及性能评价等方面提出5个研究问题;然后从电子文献数据库搜索最近10年的研究工作,经过仔细审查和筛选后选择39篇文献作为研究对象;最后,从选择文献中收集数据,通过定量分析和可视化展示来回答提出的研究问题.总的来说,回顾了持续集成测试用例集优化的研究进展,为该领域的研究者提供了一些有用的发现,并总结了面临的问题和挑战.     

Analyzing an automotive testing process with evidence-based software engineering

ContextEvidence-based software engineering (EBSE) provides a process for solving practical problems based on a rigorous research approach. The primary focus so far was on mapping and aggregating evidence through systematic reviews.ObjectivesWe extend existing work on evidence-based software engineering by using the EBSE process in an industrial case to help an organization to improve its automotive testing process. With this we contribute in (1) providing experiences on using evidence based processes to analyze a real world automotive test process and (2) provide evidence of challenges and related solutions for automotive software testing processes.MethodsIn this study we perform an in-depth investigation of an automotive test process using an extended EBSE process including case study research (gain an understanding of practical questions to define a research scope), systematic literature review (identify solutions through systematic literature), and value stream mapping (map out an improved automotive test process based on the current situation and improvement suggestions identified). These are followed by reflections on the EBSE process used.ResultsIn the first step of the EBSE process we identified 10 challenge areas with a total of 26 individual challenges. For 15 out of those 26 challenges our domain specific systematic literature review identified solutions. Based on the input from the challenges and the solutions, we created a value stream map of the current and future process.ConclusionsOverall, we found that the evidence-based process as presented in this study helps in technology transfer of research results to industry, but at the same time some challenges lie ahead (e.g. scoping systematic reviews to focus more on concrete industry problems, and understanding strategies of conducting EBSE with respect to effort and quality of the evidence).     

Exploring principles of user-centered agile software development: A literature review

ContextIn the last decade, software development has been characterized by two major approaches: agile software development, which aims to achieve increased velocity and flexibility during the development process, and user-centered design, which places the goals and needs of the system’s end-users at the center of software development in order to deliver software with appropriate usability. Hybrid development models, referred to as user-centered agile software development (UCASD) in this article, propose to combine the merits of both approaches in order to design software that is both useful and usable.ObjectiveThis paper aims to capture the current state of the art in UCASD approaches and to derive generic principles from these approaches. More specifically, we investigate the following research question: Which principles constitute a user-centered agile software development approach?MethodWe conduct a systematic review of the literature on UCASD. Identified works are analyzed using a coding scheme that differentiates four levels of UCASD: the process, practices, people/social and technology dimensions. Through subsequent synthesis, we derive generic principles of UCASD.ResultsWe identified and analyzed 83 relevant publications. The analysis resulted in a comprehensive coding system and five principles for UCASD: (1) separate product discovery and product creation, (2) iterative and incremental design and development, (3) parallel interwoven creation tracks, (4) continuous stakeholder involvement, and (5) artifact-mediated communication.ConclusionOur paper contributes to the software development body of knowledge by (1) providing a broad overview of existing works in the area of UCASD, (2) deriving an analysis framework (in form a coding system) for works in this area, going beyond former classifications, and (3) identifying generic principles of UCASD and associating them with specific practices and processes.     

Influences on regression testing strategies in agile software development environments

Regression testing is a well-established practice in software development, but in recent years it has seen a change of status and emphasis with the increasing popularity of agile methods, which stress the central role of regression testing in maintaining software quality. The objectives of this article are to investigate regression testing strategies in agile development teams and identify the factors that can influence the adoption and implementation of this practice. We have used a mixed methods approach to our research, beginning with an analysis of the literature to identify research themes related to the adoption of regression testing techniques under agile methodologies, from which we developed an analytical framework for the study. This was followed by three exploratory case studies that we used to exercise the main elements of the framework, develop some key themes of interest, and devise a questionnaire for the final stage of the study, an on-line survey to explore the main issues identified in the case studies across different contexts. Within our specific sample, our results suggest that organizational maturity is a key factor in effective regression testing practices and that the adoption of such practices is helped by a coherent testing philosophy and change management processes. We also found that the return on investment in automated regression testing was positive for our respondents and that adopting these practices in the context of agile methods had been a relatively painless process for the organizations in our survey. We conclude that investing in regression testing tools and processes is likely to be beneficial for organizations. However, further work is needed in assessing how organizational culture impacts on the quality process and the financial outcomes for commercial software development organizations.     

嵌入式系统开发中敏捷方法的应用研究综述

伴随着计算机技术的迅速发展,嵌入式系统软件的应用领域得以不断拓宽,这使得嵌入式系统开发面临着日益严峻的质量、成本以及项目周期等方面的压力.另一方面,敏捷方法已在传统的软件项目当中得到越来越多的应用.很多研究都表明,敏捷方法在适应需求变更、提升生产效率和最终产品的质量方面都发挥出显著的作用.因此,在嵌入式系统软件开发中应用敏捷方法,自然也得到研究者和实践者的日益关注.应用系统评价(systematicreview)方法,试图尽可能系统地了解嵌入式系统开发过程中敏捷方法的应用状况和研究进展.通过对敏捷宣言提出以来12年间49篇相关文献的概况和分析,试图回答如下3 个问题：1) 在不同类型的嵌入式系统开发中,敏捷方法的总体应用情况如何? 2) 敏捷方法或实践是如何解决各类嵌入式软件开发中的挑战的? 3) 敏捷方法(实践)该如何通过扩展和改进,以更好地适应嵌入式系统开发?研究表明,尽管应用程度存在一定的差异,但敏捷方法已在不同类型的嵌入式开发中得到了应用.传统的敏捷方法也需要进行多种改变,以适应这些不同类型的嵌入式开发项目的特征.     

A Systematic Mapping Study of HCI Practice Research

ABSTRACT

Human–computer interaction (HCI) practice has emerged as a research domain in the HCI field and is growing. The need to transfer HCI practices to the industry began significantly with the works of Nielsen on usability engineering. To date, methods and techniques for designing, evaluating, and implementing interactive systems for human use have continued to emerge. It is, therefore, justified to conduct a systematic mapping study to determine the landscape of HCI practice research. A Systematic Mapping Study method was used to map 142 studies according to research type, topic, and contribution. These were then analyzed to determine an overview of HCI practice research. The objective was to analyze studies on HCI practice and present prominent issues that characterize the HCI practice research landscape. Second, to identify pressing challenges regarding HCI practices in software/systems development companies. The results show that HCI practice research has steadily increased since 2012. The majority of the studies explored focused on evaluation research that largely contributed to the evaluation methods or processes. Most of the studies were on design tools and techniques, design methods and contexts, design work and organizational culture, and collaboration and team communication. Interviews, case studies, and survey methods have been prominently used as research methods. HCI techniques are mostly used during the initial phase of development and during evaluation. HCI practice challenges in companies are mostly process-related and on performance of usability and user experience activities. The major challenge seems to be to find a way to collect and incorporate user feedback in a timely manner, especially in agile processes. There are areas identified in this study as needing more research.     

A systematic mapping study of API usability evaluation methods

An Application Programming Interface (API) provides a programmatic interface to a software component that is often offered publicly and may be used by programmers who are not the API’s original designers. APIs play a key role in software reuse. By reusing high quality components and services, developers can increase their productivity and avoid costly defects. The usability of an API is a qualitative characteristic that evaluates how easy it is to use an API. Recent years have seen a considerable increase in research efforts aiming at evaluating the usability of APIs. An API usability evaluation can identify problem areas and provide recommendations for improving the API. In this systematic mapping study, we focus on 47 primary studies to identify the aim and the method of the API usability studies. We investigate which API usability factors are evaluated, at which phases of API development is the usability of API evaluated and what are the current limitations and open issues in API usability evaluation. We believe that the results of this literature review would be useful for both researchers and industry practitioners interested in investigating the usability of API and new API usability evaluation methods.     

System requirements-OSS components: matching and mismatch resolution practices – an empirical study

Developing systems by integrating Open Source Software (OSS) is increasingly gaining importance in the software industry. Although the literature claims that this approach highly impacts Requirements Engineering (RE) practices, there is a lack of empirical evidence to demonstrate this statement. To explore and understand problems and challenges of current system requirement–OSS component matching and mismatches resolution practices in software development projects that integrate one or more OSS components into their software products. Semi-structured in-depth interviews with 25 respondents that have performed RE activities in software development projects that integrate OSS components in 25 different software development companies in Spain, Norway, Sweden, and Denmark. The study uncovers 15 observations regarding system requirements-OSS components matching and mismatch resolution practices used in industrial projects that integrate OSS components. The assessed projects focused mainly on pre-release stages of software applications that integrate OSS components in an opportunistic way. The results also provide details of a set of previously unexplored scenarios when solving system requirement–OSS component mismatches; and clarify some challenges and related problems. For instance, although licensing issues and the potential changes in OSS components by their corresponding communities and/or changes in system requirements have been greatly discussed in the RE literature as problems for OSS component integration, they did not appear to be relevant in our assessed projects. Instead, practitioners highlighted the problem of getting suitable OSS component documentation/information.     

Agile distributed software development: enacting control through media and context

While face‐to‐face interaction is fundamental in agile software development, distributed environments must rely extensively on mediated interactions. Practicing agile principles in distributed environments therefore poses particular control challenges related to balancing fixed vs. evolving quality requirements and people vs. process‐based collaboration. To investigate these challenges, we conducted an in‐depth case study of a successful agile distributed software project with participants from a Russian firm and a Danish firm. Applying Kirsch's elements of control framework, we offer an analysis of how control was enacted through the project context and in the participants' mediated communication. The analysis reveals that formal measurement and evaluation control were persistently enacted through mediated communication. These formal control practices were, however, predominantly carried out in conjunction with informal roles and relationships such as clan‐like control inherent in agile development. Overall, the study demonstrates that, if appropriately applied, communication technologies can significantly support distributed, agile practices by allowing concurrent enactment of both formal and informal controls. The paper discusses these findings as they relate to previous research and concludes with their implications for future research.     

Strengths and barriers behind the successful agile deployment—insights from the three software intensive companies in Finland

The number of success stories being reported concerning agile software development has led to an increase in interest among industries and research communities. The purpose of this paper is to identify strengths and barriers for ‘successful agile deployment’ in the software companies. This knowledge can benefit software companies planning their current strategy for agile deployment. Analysis of 57 developers, architects, project managers, customers, quality managers, and line and product managers in three case companies identifies 71 strengths and 169 barriers of agile deployment. The analysis revealed the importance of management providing the necessary goals and support for agile development. It also indicated the significance of defining a tailored process model and giving developers the freedom to improve their own agile development process continuously during agile deployment. The identified barriers, strengths and recommendations can be used as a checklist for planning and/or monitoring the effectiveness of agile deployment in software companies. By identifying the barriers and strengths of agile deployment, the paper deepens understanding of this highly relevant but relatively under-researched phenomenon and contributes to the literature on agile deployment and software process improvement.     

A comparative study of challenges in integrating Open Source Software and Inner Source Software

Context

Several large software-developing organizations have adopted Open Source Software development (OSSD) practices to develop in-house components that are subsequently integrated into products. This phenomenon is also known as “Inner Source”. While there have been several reports of successful cases of this phenomenon, little is known about the challenges that practitioners face when integrating software that is developed in such a setting.

Objective

The objective of this study was to shed light on challenges related to building products with components that have been developed within an Inner Source development environment.

Method

Following an initial systematic literature review to generate seed category data constructs, we performed an in-depth exploratory case study in an organization that has a significant track record in the implementation of Inner Source. Data was gathered through semi-structured interviews with participants from a range of divisions across the organization. Interviews were transcribed and analyzed using qualitative data analysis techniques.

Results

We have identified a number of challenges and approaches to address them, and compared the findings to challenges related to development with OSS products reported in the literature. We found that many challenges identified in the case study could be mapped to challenges related to integration of OSS.

Conclusion

The results provide important insights into common challenges of developing with OSS and Inner Source and may help organizations to understand how to improve their software development practices by adopting certain OSSD practices. The findings also identify the areas that need further research.     

The effect of moving from a plan-driven to an incremental software development approach with agile practices

So far, only few in-depth studies focused on the direct comparison of process models in general, and between plan-driven and incremental/agile approaches in particular. That is, it is not made explicit what the effect is of moving from one model to another model. Furthermore, there is limited evidence on advantages and issues encountered in agile software development, this is specifically true in the context of large-scale development. The objective of the paper is to investigate how the perception of bottlenecks, unnecessary work, and rework (from hereon referred to as issues) changes when migrating from a plan-driven to an incremental software development approach with agile practices (flexible product backlog, face-to-face interaction, and frequent integration), and how commonly perceived these practices are across different systems and development roles. The context in which the objective should be achieved is large-scale development with a market-driven focus. The selection of the context was based on the observation in related work that mostly small software development projects were investigated and that the investigation was focused on one agile model (eXtreme programming). A case study was conducted at a development site of Ericsson AB, located in Sweden in the end of 2007. In total 33 interviews were conducted in order to investigate the perceived change when migrating from plan-driven to incremental and agile software development, the interviews being the primary source of evidence. For triangulation purposes measurements collected by Ericsson were considered, the measurements relating to unnecessary work (amount of discarded requirements) and rework (data on testing efficiency and maintenance effort). Triangulation in this context means that the measurements were used to confirm the perceived changes with an additional data source. In total 64 issues were identified, 24 being of general nature and the remaining 40 being local and therefore unique to individual’s opinions or a specific system. The most common ones were documented and analyzed in detail. The commonality refers to how many persons in different roles and across the systems studied have mentioned the issues for each of the process models. The majority of the most common issues relates to plan-driven development. We also identified common issues remaining for agile after the migration, which were related to testing lead-time, test coverage, software release, and coordination overhead. Improvements were identified as many issues commonly raised for the plan-driven approach were not raised anymore for the incremental and agile approach. It is concluded that the recent introduction (start in 2005 with the study being conducted in the end of 2007) of incremental and agile practices brings added values in comparison to the plan-driven approach, which is evident from the absence of critical issues that are encountered in plan-driven development.     

Quality in model-driven engineering: a tertiary study

Model-driven engineering (MDE) is believed to have a significant impact in software quality. However, researchers and practitioners may have a hard time locating consolidated evidence on this impact, as the available information is scattered in several different publications. Our goal is to aggregate consolidated findings on quality in MDE, facilitating the work of researchers and practitioners in learning about the coverage and main findings of existing work as well as identifying relatively unexplored niches of research that need further attention. We performed a tertiary study on quality in MDE, in order to gain a better understanding of its most prominent findings and existing challenges, as reported in the literature. We identified 22 systematic literature reviews and mapping studies and the most relevant quality attributes addressed by each of those studies, in the context of MDE. Maintainability is clearly the most often studied and reported quality attribute impacted by MDE. Eighty out of 83 research questions in the selected secondary studies have a structure that is more often associated with mapping existing research than with answering more concrete research questions (e.g., comparing two alternative MDE approaches with respect to their impact on a specific quality attribute). We briefly outline the main contributions of each of the selected literature reviews. In the collected studies, we observed a broad coverage of software product quality, although frequently accompanied by notes on how much more empirical research is needed to further validate existing claims. Relatively, little attention seems to be devoted to the impact of MDE on the quality in use of products developed using MDE.     

On the reliability of mapping studies in software engineering

Background

Systematic literature reviews and systematic mapping studies are becoming increasingly common in software engineering, and hence it becomes even more important to better understand the reliability of such studies.

Objective

This paper presents a study of two systematic mapping studies to evaluate the reliability of mapping studies and point out some challenges related to this type of study in software engineering.

Method

The research is based on an in-depth case study of two published mapping studies on software product line testing.

Results

We found that despite the fact that the two studies are addressing the same topic, there are quite a number of differences when it comes to papers included and in terms of classification of the papers included in the two mapping studies.

Conclusions

From this we conclude that although mapping studies are important, their reliability cannot simply be taken for granted. Based on the findings we also provide four conjectures that further research has to address to make secondary studies (systematic mapping studies and systematic literature reviews) even more valuable to both researchers and practitioners.     

A systematic mapping study on best practices for domain-specific modeling

Model-driven software development comes in different styles. While standard-based approaches leverage existing language standards (e.g., UML), tooling, and development processes, domain-specific modeling (DSM) requires languages and tool support to be created prior to the actual software development. The design, implementation, and testing of languages and tool support require a wide spectrum of methods and techniques which introduce complexity and new challenges. To tackle these DSM-specific challenges, best practices have been collected from various application domains and published in literature to guide the development and application of DSM solutions. It is the goal of this paper to identify studies reporting best practices on domain-specific modeling. Moreover, a systematic and comprehensive compilation of best practices should act as a starting point to identify literature that facilitates industrial adoption of DSM. To search for literature and classify identified studies, we conduct a systematic mapping study (SMS). Furthermore, we perform an in-depth analysis of the identified studies to answer how practices overlap, complement, or contradict each other. The systematic search resulted in 21 studies reporting 321 best practices. From these 321 practices, we compiled 192 unique best practices. We found that the DSM community created a substantial corpus of best practices for DSM. The large majority of practices (75%) are only reported once. The top best practices by number of reports contain only practices that are reported at least four times. However, the frequency of reports does not necessarily imply a high importance of practice, as the application of practices is always context specific.     

A model-based framework for classifying and diagnosing usability problems

A great deal of study has been devoted to the problem of how to identify and categorize usability problems; however, there is still a lack of studies dealing with the problem of how to diagnose the causes of usability problems and how to feed them back into design process. The value of classifying usability problems can be enhanced when they are interpreted in connection with design process and activities. Thus, it is necessary to develop a systematic way of diagnosing usability problems in terms of design aspects and applying diagnosis results to improve design process and activities. With this issue in mind, this paper proposes a conceptual framework that supports a systematic classification and diagnosis of usability problems. This paper firstly reviews seven approaches to classifying usability problems. Then, we point out the needs of adopting a model-based approach to classifying and diagnosing usability problems and of developing a comprehensive framework guiding the use of model-based approaches. We then propose a conceptual framework that specifies how a model-based classification and diagnosis of usability problems should be conducted and suggests the combined use of three different types of models, each of which addresses context of use, design knowledge and design activities. Last, we explain how a sound classification scheme of usability problems can be systematically developed, and how the classification of usability problems can be connected to design process and activities on the basis of the framework.     

Software development in startup companies: A systematic mapping study

ContextSoftware startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present a unique combination of characteristics which pose several challenges to software development activities.ObjectiveThis study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers.MethodWe conducted a systematic mapping study, developing a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups.ResultsA total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed.ConclusionThis mapping study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.     

Applying empirical software engineering to software architecture: challenges and lessons learned

In the last 15 years, software architecture has emerged as an important software engineering field for managing the development and maintenance of large, software-intensive systems. Software architecture community has developed numerous methods, techniques, and tools to support the architecture process (analysis, design, and review). Historically, most advances in software architecture have been driven by talented people and industrial experience, but there is now a growing need to systematically gather empirical evidence about the advantages or otherwise of tools and methods rather than just rely on promotional anecdotes or rhetoric. The aim of this paper is to promote and facilitate the application of the empirical paradigm to software architecture. To this end, we describe the challenges and lessons learned when assessing software architecture research that used controlled experiments, replications, expert opinion, systematic literature reviews, observational studies, and surveys. Our research will support the emergence of a body of knowledge consisting of the more widely-accepted and well-formed software architecture theories.     

The IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems

Aiming at a user-oriented approach in software evaluation on the basis of ISO 9241 Part 10, we present a questionnaire (IsoMetrics) which collects usability data for summative and formative evaluation, and document its construction. The summative version of IsoMetrics shows a high reliability of its subscales and gathers valid information about differences in the usability of different software systems. Moreover, we show that the formative version of IsoMetrics is a powerful tool for supporting the identification of software weaknesses. Finally, we propose a procedure to categorize and prioritize weak points, which subsequently can be used as basic input to usability reviews.