Influences on regression testing strategies in agile software development environments

Regression testing is a well-established practice in software development, but in recent years it has seen a change of status and emphasis with the increasing popularity of agile methods, which stress the central role of regression testing in maintaining software quality. The objectives of this article are to investigate regression testing strategies in agile development teams and identify the factors that can influence the adoption and implementation of this practice. We have used a mixed methods approach to our research, beginning with an analysis of the literature to identify research themes related to the adoption of regression testing techniques under agile methodologies, from which we developed an analytical framework for the study. This was followed by three exploratory case studies that we used to exercise the main elements of the framework, develop some key themes of interest, and devise a questionnaire for the final stage of the study, an on-line survey to explore the main issues identified in the case studies across different contexts. Within our specific sample, our results suggest that organizational maturity is a key factor in effective regression testing practices and that the adoption of such practices is helped by a coherent testing philosophy and change management processes. We also found that the return on investment in automated regression testing was positive for our respondents and that adopting these practices in the context of agile methods had been a relatively painless process for the organizations in our survey. We conclude that investing in regression testing tools and processes is likely to be beneficial for organizations. However, further work is needed in assessing how organizational culture impacts on the quality process and the financial outcomes for commercial software development organizations.     

An empirical study of testing and integration strategies usingartificial software systems

There has been much discussion about the merits of various testing and integration strategies. Top-down, bottom-up, big-bang, and sandwich integration strategies are advocated by various authors. Also, some authors insist that modules be unit tested, while others believe that unit testing diverts resources from more effective verification processes. This article addresses the ability of the aforementioned integration strategies to detect defects, and produce reliable systems. It also explores the efficacy of spot unit testing, and compares phased and incremental versions of top-down and bottom-up integration strategies. Relatively large artificial software systems were constructed using a code generator with ten basic module templates. These systems were seeded with known defects and tested using the above testing and integration strategies. A number of experiments were then conducted using a simulator whose validity was established by comparing results against these artificial systems. The defect detection ability and resulting system reliability were measured for each strategy. Results indicated that top-down integration strategies are generally most effective in terms of defect correction. Top-down and big-bang strategies produced the most reliable systems. Results favored neither those strategies that incorporate spot unit testing nor those that do not; also, results favored neither phased nor incremental strategies     

A learning algorithm for optimizing continuous integration development and testing practice

Continuous integration, at its core, includes a set of practices that aim to prevent and reduce the cost of software integration issues by merging working software copies often. Regression testing is considered a good practice in software development with continuous integration, which ensures that code changes are not negatively affecting software functionality. As, nowadays, software development is carried out iteratively, with small code increments continuously developed and regression tested, it is of critical importance that continuous regression testing is time efficient. However, in practice, regression testing is often long lasting and faces scalability problems as software grows larger or as software changes are made more frequently. One contributing factor to these issues is test redundancy, which causes the same software functionality being tested multiple times across a test suite. In large-scale software, especially highly configurable software, redundancy in continuous regression testing can significantly grow the size of test suites and negatively affect the cost effectiveness of continuous integration. This paper presents a practical learning algorithm for optimizing continuous integration testing by reducing ineffective test redundancy in regression suites. The novelty of the algorithm lies in learning and predicting the fault-detection effectiveness of continuous integration tests using historical test records and combining this information with coverage-based redundancy metrics. The goal is to identify ineffective redundancy, which is maximally reduced in the resulting regression test suite, thus reducing test time and improving the performance of continuous integration. We apply and evaluate the algorithm in two industrial projects of continuous integration. The results show that the proposed algorithm can improve the efficiency of continuous integration practice in terms of decreasing test execution time by 38% on average compared to the industry practice of our case study and by 40% on average compared to the retest-all approach. The results further demonstrate no significant reduction in fault-detection effectiveness of continuous regression testing. This suggests that the proposed algorithm contributes to the state of the practice in the continuous integration development and testing of highly configurable systems.     

A dependency taxonomy for agile software development projects

Agile software development is designed to achieve collaborative software development. A supporting pillar of collaboration is effective coordination, which is necessary to manage dependencies in projects. Understanding the dependencies arising in agile software development projects can help practitioners choose appropriate coordinative practices from the large number of practices provided by the various agile methods. To achieve this understanding, this article analyses dependencies in three typical cases of co-located agile software development and presents the dependencies as a taxonomy with decision rules for allocating dependencies into categories. Findings show that knowledge, process, and resource dependencies are present, with knowledge dependencies predominant. In addition, there are agile practices with a coordinative function that address multiple dependencies in these agile software development projects. These practices would be a good choice for coordinating a project and supporting collaboration in agile software projects.     

敏捷方法在软件项目开发中的实践

目前很多企业内中小型软件项目面临开发时间紧迫、人手不足、需求不断变化的困难,传统重量级的软件开发方法无法应对这样的挑战.敏捷方法是基于实践的软件开发方法学,为解决这类轻量级项目管理和开发所碰到的问题提供了新的思路.通过分析敏捷方法的主要目标、观点和原则,并结合一个实际的管理信息系统项目开发,从项目计划、项目文档、重构的改进和项目维护的4个方面探讨了敏捷方法的实践应用.实践证明,采用敏捷方法的观点和原则进行必要的改进,能取得项目开发的成功.     

Modal testing strategies for OO software

The essential challenge in testing object-oriented software is to craft a practical test suite that exercises enough combinations of message sequence and state interactions to provide sufficient confidence. These interactions are often subtle and complex and therefore error-prone. If all classes were alike, it might be enough to devise tests based either on message sequence or state. Sometimes this suffices, but usually both views are needed. The notion of class modality characterizes class behavior differences to indicate an appropriate test strategy     

Coordination in co-located agile software development projects

Agile software development provides a way to organise the complex task of multi-participant software development while accommodating constant project change. Agile software development is well accepted in the practitioner community but there is little understanding of how such projects achieve effective coordination, which is known to be critical in successful software projects. A theoretical model of coordination in the agile software development context is presented based on empirical data from three cases of co-located agile software development. Many practices in these projects act as coordination mechanisms, which together form a coordination strategy. Coordination strategy in this context has three components: synchronisation, structure, and boundary spanning. Coordination effectiveness has two components: implicit and explicit. The theoretical model of coordination in agile software development projects proposes that an agile coordination strategy increases coordination effectiveness. This model has application for practitioners who want to select appropriate practices from agile methods to ensure they achieve coordination coverage in their project. For the field of information systems development, this theory contributes to knowledge of coordination and coordination effectiveness in the context of agile software development.     

Supporting agile software development through active documentation

Agile development processes are adaptive rather than predictive. Therefore, agile processes emphasize operational system code rather than its documentation. To overcome the absence of comprehensive documentation artifacts, agile methods require constant interaction between the system stakeholders. Ironically, however, some traditional documentation artifacts come to support this kind of interaction. In this study, we examine the relationship between software and documentation. We develop an approach that enables incorporating domain documentation to agile development, while keeping the processes adaptive. We also provide a system design that actively uses domain knowledge documentation. These ideas have been applied through the implementation and use of agile documentation support components.     

Integrating agile software development into stage-gate managed product development

Agile methods have evolved as a bottom-up approach to software development. However, as the software in embedded products is only one part of development projects, agile methods must coexist with project management models typically of the stage-gate type. This paper presents a qualitative case study of two large independent software system projects that have used eXtreme Programming (XP) for software development within contexts of stage-gate project management models. The study is comprised of open ended interviews with managers as well as practitioners, followed by a structured, fully traceable, qualitative analysis. We conclude that it is possible to integrate XP in a gate model context. Key issues for success are the interfaces towards the agile subproject and management attitudes towards the agile approach. Editor: Marvin Zelkowitz     

敏捷软件开发与计划驱动开发的概述比较

人们在设想、确定以及创建软件时,身边的环境不断在变更.敏捷是为了在动荡的业务环境中获益而创造变革和响应变革的能力.极限编程是最著名的敏捷软件开发方法.传统的开发侧重于计划和架构,计划驱动开发关注的是软件的质量和过程的可预见性.计划驱动开发最佳范例是能力成熟度模型.两种表面上有不同观点的方法在争夺着软件开发的主导权,对敏捷软件开发与计划驱动开发进行了概述,并就特征、擅长领域和关键要素等进行比较.     

Problems and strategies for software component testing standards

What does it mean to say that an item of software has been tested? Unfortunately, currently accepted standards are inadequate to give the confidence the user needs and the meaningful objective for the supplier. This paper assesses the currently available standards, mainly in the component testing area and advocates that the British Computer Society proto-standard should be taken as a basis for a formal standard. The paper is intended for those concerned with software quality.     

A Lagrangian heuristic for sprint planning in agile software development

Agile methods for software development promote iterative design and implementation. Most of them divide a project into functionalities, called user stories; at each iteration, often called a sprint, a subset of user stories are developed. The sprint planning phase is critical to ensure the project success, but it is also a difficult problem because several factors impact on the optimality of a sprint plan, e.g., the estimated complexity, business value, and affinity of the user stories to be included in each sprint. In this paper we present an approach for sprint planning based on an integer linear programming model. Given the estimates made by the project team and a set of development constraints, the optimal solution of the model is a sprint plan that maximizes the business value perceived by users. Solving to optimality the model by a general-purpose MIP solver, such as IBM Ilog Cplex, takes time and for some instances even finding a feasible solution requires too large computing times for an operational use. For this reason we propose an effective Lagrangian heuristic based on a relaxation of the proposed model and some greedy and exchange algorithms. Computational results on both real and synthetic projects show the effectiveness of the proposed approach.     

面向持续集成的回归测试优化方法

针对持续集成环境下回归测试需要进行持续优化的问题,提出一种依据回归测试目标自适应调整策略的优化方法.首先将失败标志、缺陷检测数、重要性因子、新旧功能标志作为用例属性进行标记,根据历史数据和关联关系对用例属性进行初始化;而后根据阶段测试目标,区分新功能测试、修改性测试,将需求映射为具体的用例属性指标,据此对用例进行选择;计算重要性因子,更新用例属性标签,根据用例属性进行优先级自动排序;用例执行中,按照时间、资源要求,根据用例属性选择相应规模的测试用例进行执行.最后选择开源数据集进行实验,结果表明该方法针对不同的测试目标均能够降低执行用例的规模,提高缺陷检测效率.     

一种面向数值模拟软件的持续集成平台

随着软件复杂度的增加和交付需求的变化,快速集成和自动部署成为高性能数值模拟软件推广应用的瓶颈。为满足软件构建和发布的差异化需求,简化从用户需求到软件发布的工作流程,基于Jenkins设计持续集成平台,实现软件静态审查、编译、测试和发布流程的自动化,全面记录软件构建日志和测试结果。利用Docker技术将构建环境容器化,实现环境的快速搭建和配置管理,满足多样化目标环境需求。该平台的实际应用效果显示,基于该平台的产品发布周期相比之前缩短约75%,可极大简化软件安装和部署难度。     

Test criteria and coverage measures for software integration testing

The aim of this paper is to introduce a systematic approach to integration testing of software systems. Various test data selection criteria for integration testing are presented, coverage measures are introduced, and interconnection between them are discussed. The main principle is to transfer and adapt test criteria and coverage measures which are useful for unit testing to the level of integration testing. Test criteria help the tester to organise the test process. They should be chosen in accordance with the available test effort. Test coverage measures are defined as a ratio between the test cases required for satisfying the criteria and those of these which have been executed. The measures are used to obtain information about the completeness of integration tests. The approach is described for data flow and control flow oriented criteria and measures. The intention is to enable the tester to specify integration tests in advance in terms of effort, and to evaluate the results in terms of test completeness.     

The trip-packing dilemma [agile software development]

A guiding principle behind agile software development is the idea of keeping things simple. Making life simpler is an admirable goal, but unfortunately many developers have a knack for making one of two errors: oversimplifying something that really is complex; and overcomplicating something that should be easy. The paper considers how Extreme Programming attacks over-embellishment with the YAGNI (you aren't gonna need it) principle.     

Usability in agile software development: A tertiary study

In the last years the interest in developing research on integration of usability and agile software development has been increasing. The number of systematic literature reviews, systematic mapping studies and non-systematic reviews, related to this thematic has also increased. Nevertheless, there is no analysis on the quality of these published secondary studies, nor is there a consolidated research that brings the answer of how to integrate these two areas. The goal of this paper is to categorize secondary studies related to the integration of usability and agile software development and present a critical analysis on the quality of the selected studies. To accomplish this goal a tertiary study was performed to categorize the related studies selected. Initially 3,065 papers were identified and further narrowed to 14 by applying exclusion criteria and analysis. We classified the selected studies as systematic literature reviews, systematic mapping studies and non-systematic literature reviews to report the data analysis. As a result of this study different forms to integrate usability and agile software development were detected as well as the various challenges that must be overcome for the integration success. Six main categories were identified to represent ways of integrating usability into agile development: processes, techniques, practices, recommendations, principles and different approaches. Regarding to the challenges for the integration seven main categories were also identified: issues related to tests, time, work balance, modularization, feedback, prioritization, and documentation. Although the interest in researching the integration of usability and agile software development has increased in the last years, mostly of the analyzed studies neglected the quality criteria and presented difficulties to use methods to synthetize the research results. Despite this, it has been realized that the integration of usability with agile software development is possible and is strongly aligned with user-centered design. The initial studies indicated a separation of activities and roles into specific tracks with parallel work to treat usability in agile software development, but the trend is no longer to manage and control these activities in separate ways, so new challenges are becoming to appear. Although we have identified several points of tension, the integration does not become unfeasible.     

A method of test-suite reduction for regression integration testing

This paper describes a method of test-suite reduction for regression testing of the interaction between two software modules based on a model of module interaction on a test suite and subsequent filtering of the suite using the constructed model. The interaction model is constructed in terms of sequences of interface functions of the modules being integrated called during the software run.     

面向对象软件的依赖性分析与回归测试

针对面向对象软件中类之间复杂的依赖性问题,通过分析UML类图中类之间的各种静态关系,提出一个基于UML类图的依赖性分析模型。为了实现自动化分析,引入了类依赖关系图的概念,并提出一个基于UML类图导出的XMI文件的类依赖关系图生成算法。根据这个模型,通过分析一个类或依赖关系的改变对其他类产生的影响,提出测试路径查找算法,并利用此算法得到需要进行回归测试的类集与测试序列。