Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today??s testing approaches for SPLs validate products individually using different kinds of reuse techniques for testing. Because of their reusability and adaptability capabilities, model-based approaches are suitable to describe variability and are therefore frequently used for implementation and testing purposes of SPLs. Due to the enormous number of possible products, individual product testing becomes more and more infeasible. Pairwise testing offers one possibility to test a subset of all possible products. However, according to the best of our knowledge, there is no contribution discussing and rating this approach in the SPL context. In this contribution, we provide a mapping between feature models describing the common and variable parts of an SPL and a reusable test model in the form of statecharts. Thereby, we interrelate feature model-based coverage criteria and test model-based coverage criteria such as control and data flow coverage and are therefore able to discuss the potentials and limitations of pairwise testing. We pay particular attention to test requirements for feature interactions constituting a major challenge in SPL engineering. We give a concise definition of feature dependencies and feature interactions from a testing point of view, and we discuss adequacy criteria for SPL coverage under pairwise feature interaction testing and give a generalization to the T-wise case. The concept and implementation of our approach are evaluated by means of a case study from the automotive domain.     

SPL Conqueror: Toward optimization of non-functional properties in software product lines

A software product line (SPL) is a family of related programs of a domain. The programs of an SPL are distinguished in terms of features, which are end-user visible characteristics of programs. Based on a selection of features, stakeholders can derive tailor-made programs that satisfy functional requirements. Besides functional requirements, different application scenarios raise the need for optimizing non-functional properties of a variant. The diversity of application scenarios leads to heterogeneous optimization goals with respect to non-functional properties (e.g., performance vs. footprint vs. energy optimized variants). Hence, an SPL has to satisfy different and sometimes contradicting requirements regarding non-functional properties. Usually, the actually required non-functional properties are not known before product derivation and can vary for each application scenario and customer. Allowing stakeholders to derive optimized variants requires us to measure non-functional properties after the SPL is developed. Unfortunately, the high variability provided by SPLs complicates measurement and optimization of non-functional properties due to a large variant space. With SPL Conqueror, we provide a holistic approach to optimize non-functional properties in SPL engineering. We show how non-functional properties can be qualitatively specified and quantitatively measured in the context of SPLs. Furthermore, we discuss the variant-derivation process in SPL Conqueror that reduces the effort of computing an optimal variant. We demonstrate the applicability of our approach by means of nine case studies of a broad range of application domains (e.g., database management and operating systems). Moreover, we show that SPL Conqueror is implementation and language independent by using SPLs that are implemented with different mechanisms, such as conditional compilation and feature-oriented programming.     

Intelligent software product line configurations: A literature review

A software product line (SPL) is a set of industrial software-intensive systems for configuring similar software products in which personalized feature sets are configured by different business teams. The integration of these feature sets can generate inconsistencies that are typically resolved through manual deliberation. This is a time-consuming process and leads to a potential loss of business resources. Artificial intelligence (AI) techniques can provide the best solution to address this issue autonomously through more efficient configurations, lesser inconsistencies and optimized resources. This paper presents the first literature review of both research and industrial AI applications to SPL configuration issues. Our results reveal only 19 relevant research works which employ traditional AI techniques on small feature sets with no real-life testing or application in industry. We categorize these works in a typology by identifying 8 perspectives of SPL. We also show that only 2 standard industrial SPL tools employ AI in a limited way to resolve inconsistencies. To inject more interest and application in this domain, we motivate and present future research directions. Particularly, using real-world SPL data, we demonstrate how predictive analytics (a state of the art AI technique) can separately model inconsistent and consistent patterns, and then predict inconsistencies in advance to help SPL designers during the configuration of a product.     

A model-driven traceability framework for software product lines

Software product line (SPL) engineering is a recent approach to software development where a set of software products are derived for a well defined target application domain, from a common set of core assets using analogous means of production (for instance, through Model Driven Engineering). Therefore, such family of products are built from reuse, instead of developed individually from scratch. SPL promise to lower the costs of development, increase the quality of software, give clients more flexibility and reduce time to market. These benefits come with a set of new problems and turn some older problems possibly more complex. One of these problems is traceability management. In the European AMPLE project we are creating a common traceability framework across the various activities of the SPL development. We identified four orthogonal traceability dimensions in SPL development, one of which is an extension of what is often considered as “traceability of variability”. This constitutes one of the two contributions of this paper. The second contribution is the specification of a metamodel for a repository of traceability links in the context of SPL and the implementation of a respective traceability framework. This framework enables fundamental traceability management operations, such as trace import and export, modification, query and visualization. The power of our framework is highlighted with an example scenario.     

PAxSPL: A feature retrieval process for software product line reengineering

Software product lines (SPLs) are a well-known solution to systematically create reusable software products. Among the approaches to create an SPL, the extractive approach is usually used when the organization already has a set of similar systems. These systems are analyzed to extract, categorize, and group their common and variant features throughout the SPL reengineering process. As there are different scenario variables, such as available artifacts and team experience, the activities and techniques used to perform these tasks may change. This may increase the effort and decrease the quality of retrieved features when users with low experience in SPL reengineering perform such tasks. However, there is a lack of a process supporting these tasks considering different scenarios. Therefore, we specify the P repare, A ssemble, and E x ecute Process for SPL Reengineering (PAxSPL), a process that provides support to prepare, assemble, and execute feature retrieval throughout the analysis of documentation and team experience. To initially evaluate PAxSPL, we conducted and reported an exploratory case study in a real development environment. The results indicated that our proposal helps in the assembly of a feature retrieval process according to user needs. Results were important to identify points for improvement in PAxSPL. We also could use the information gathered to improve the guidelines and provide this information to be used as basis of comparison for future users.     

Feature Nets: behavioural modelling of software product lines

Software product lines (SPLs) are diverse systems that are developed using a dual engineering process: (a) family engineering defines the commonality and variability among all members of the SPL, and (b) application engineering derives specific products based on the common foundation combined with a variable selection of features. The number of derivable products in an SPL can thus be exponential in the number of features. This inherent complexity poses two main challenges when it comes to modelling: firstly, the formalism used for modelling SPLs needs to be modular and scalable. Secondly, it should ensure that all products behave correctly by providing the ability to analyse and verify complex models efficiently. In this paper, we propose to integrate an established modelling formalism (Petri nets) with the domain of software product line engineering. To this end, we extend Petri nets to Feature Nets. While Petri nets provide a framework for formally modelling and verifying single software systems, Feature Nets offer the same sort of benefits for software product lines. We show how SPLs can be modelled in an incremental, modular fashion using Feature Nets, provide a Feature Nets variant that supports modelling dynamic SPLs, and propose an analysis method for SPL modelled as Feature Nets. By facilitating the construction of a single model that includes the various behaviours exhibited by the products in an SPL, we make a significant step towards efficient and practical quality assurance methods for software product lines.     

Plural: A decentralized business process modeling method

Top-down and centralized approaches prevail in the design and improvement of business processes. However, centralized structures pose difficulties for organizations in adapting to a rapidly changing business environment. Here we present the Plural method which can be used to guide organizations in performing process modeling in a decentralized way. Instead of a centralized group of people understanding, modeling and improving processes, our method allows individuals to model and improve their own processes to help in fulfilling their roles in the organization. An individual model depicts a set of activities performed by a role, which together result in a cohesive service within the organization. These individual models are then integrated as necessary to show the way the organization works. We applied the Plural method in a case study of a small-size software organization. We describe the method and its underlying principles and then discuss the findings of our case study, lessons learned, and limitations. The study thus provided evidence of Plural's utility and showed how an organization might exploit its strengths.     

System-specific static code analyses: a case study in the complex embedded systems domain

In this paper, we are exploring the approach to utilize system-specific static analyses of code with the goal to improve software quality for specific software systems. Specialized analyses, tailored for a particular system, make it possible to take advantage of system/domain knowledge that is not available to more generic analyses. Furthermore, analyses can be selected and/or developed in order to best meet the challenges and specific issues of the system at hand. As a result, such analyses can be used as a complement to more generic code analysis tools because they are likely to have a better impact on (business) concerns such as improving certain software quality attributes and reducing certain classes of failures. We present a case study of a large, industrial embedded system, giving examples of what kinds of analyses could be realized and demonstrate the feasibility of implementing such analyses. We synthesize lessons learned based on our case study and provide recommendations on how to realize system-specific analyses and how to get them adopted by industry.     

MobiLine: A Nested Software Product Line for the domain of mobile and context-aware applications

Mobile devices are multipurpose and multi-sensor equipments supporting applications able to adapt their behavior according to changes in the user’s context (device, location, time, etc.). Meanwhile, the development of mobile and context-aware software is not a simple task, mostly due to the peculiar characteristics of these devices. Although several solutions have been proposed to facilitate their development, reuse is not systematically used throughout the software development life-cycle. In this paper, we discuss an approach for the development of mobile and context-aware software using the Software Product Line (SPL) paradigm. Furthermore, a Nested SPL for the domain of mobile and context-aware applications is presented, lessons learned in the SPL development are discussed and a product for a context-aware visit guide is shown.     

Domain modeling as a basis for building a meshing tool software product line

Meshing tools are highly complex software for generating and managing geometrical discretizations. Due to their complexity, they have generally been developed by end users – physicists, forest engineers, mechanical engineers – with ad hoc methodologies and not by applying well established software engineering practices. Different meshing tools have been developed over the years, making them a good application domain for Software Product Lines (SPLs). This paper proposes building a domain model that captures the different domain characteristics such as features, goals, scenarios and a lexicon, and the relationships among them. The model is partly specified using a formal language. The domain model captures product commonalities and variabilities as well as the particular characteristics of different SPL products. The paper presents a rigorous process for building the domain model, where specific roles, activities and artifacts are identified. This process also clearly establishes consistency and completeness conditions. The usefulness of the model and the process are validated by using them to generate a software product line of Tree Stem Deformation (TSD) meshing tools. We also present Meshing Tool Generator, a software that follows the SPL approach for generating meshing tools belonging to the TSD SPL. We show how an end user can easily generate three different TSD meshing tools using Meshing Tool Generator.     

Incremental method evolution in global software product management: A retrospective case study

Company growth in a global setting causes challenges in the adaptation and maintenance of an organization’s methods. In this paper, we will analyze incremental method evolution in software product management in a global environment. We validate a method increment approach, based on method engineering principles, by applying it to a retrospective case study conducted at a large ERP vendor. The results show that the method increment types cover all increments that were found in the case study. Also, we identified the following lessons learned for company growth in a global software product management context: method increment drivers, such as the change of business strategy, vary during evolution; a shared infrastructure is critical for rollout; small increments facilitate gradual process improvement; and global involvement is critical. We then claim that method increments enable software companies to accommodate evolutionary adaptations of development process in agreement with the overall company expansion.     

一种基于Mealy!机的语义程序验证方法

语义验证是束缚语义软件和语义程序设计语言发展的问题之一,针对这一问题,在基于语义Web服务的语义程序设计语言SPL及其知识库业务领域本体(BDO)的基础上,提出了一种基于Mealy!机对SPL所编排的业务过程进行语义验证的方法,结合在线外汇交易平台的案例,详细描述了运用该方法进行语义验证的过程。通过案例证明,本方法有助于编写语义正确的语义程序。     

Comparing practices for reuse in integration‐oriented software product lines and large open source software projects

This article compares the organization and practices for software reuse in integration‐oriented software product lines (SPLs) and open source software projects. The main observation is that both approaches are successful regarding large variability and reuse, but differ widely in their practices and organization. To capture practices in large open source projects, we describe an open compositional model, which reflects their more decentralized organization of software development. We capture key practices and organizational forms for this and validate these by comparing four case studies of this model. Two of these studies are based on published SPL case studies, for the other two we analyze the practices in two large and successful open source projects based on their published developer documentation. Our analysis highlights key differences between the practices in the two open source organizations and the more integrational practices used in the other two cases. Finally, we discuss which practices are successful in which environment and how the current practices can move towards more open, widely scoped and distributed software development. Copyright © 2010 John Wiley & Sons, Ltd.     

Salespoint: A Java framework for teaching object-oriented software development

Teaching systematic object-oriented software development to undergraduate students is difficult: Students need to develop a lot of complex skills. These include technical skills in object-oriented software development, but also social skills—for example, how to collaborate with other developers as part of a team working towards a large and complex software system. To acquire these skills, students need hands-on development experiences—for example, through team-oriented project courses. Designing such project courses is a challenge in itself: They must be both sufficiently challenging and achievable within the limited time available. In our special situation (large numbers of students supervised by small numbers of staff) an important further requirement is scalability: Different projects should be easily comparable while allowing for different tasks for different teams to reduce the risk of plagiarism. The solution that in our experience satisfies all these requirements is to use an application framework for an everyday application domain—for example, the business domain.Since 1997, we have been using Salespoint, a Java-based framework for creating business applications, that has been jointly developed and maintained in Dresden and Munich. In this paper, we briefly recollect the educational background and aims of the courses and present in some detail Salespoint (and its most recent revision, Salespoint2010): central notions like catalogs and stocks, the functionality it offers to users (application control, data management, and much more), a technical overview of its architecture, an example application built with Salespoint, and some lessons learned so far.     

Fuzzy inference system for software product family process evaluation

When developing multiple products within a common application domain, systematic use of a software product family process can yield increased productivity in cost, quality, effort and schedule. Such a process provides the means for the reuse of software assets which can considerably reduce the development time and the cost of software products. A comprehensive strategy for the evaluating the maturity of a software product family process is needed due to growing popularity of this concept in the software industry. In this paper, we propose a five-level maturity scale for software product family process. We also present a fuzzy inference system for evaluating maturity of software product family process using the proposed maturity scale. This research is aimed at establishing a comprehensive and unified strategy for process evaluation of a software product family. Such a process evaluation strategy will enable an organization to discover and monitor the strengths and weaknesses of the various activities performed during development of multiple products within a common application domain.     

Specification and automated validation of staged reconfiguration processes for dynamic software product lines

Dynamic software product lines (DSPLs) propose elaborated design and implementation principles for engineering highly configurable runtime-adaptive systems in a sustainable and feature-oriented way. For this, DSPLs add to classical software product lines (SPL) the notions of (1) staged (pre-)configurations with dedicated binding times for each individual feature, and (2) continuous runtime reconfigurations of dynamic features throughout the entire product life cycle. Especially in the context of safety- and mission-critical systems, the design of reliable DSPLs requires capabilities for accurately specifying and validating arbitrary complex constraints among configuration parameters and/or respective reconfiguration options. Compared to classical SPL domain analysis which is usually based on Boolean constraint solving, DSPL validation, therefore, further requires capabilities for checking temporal properties of reconfiguration processes. In this article, we present a comprehensive approach for modeling and automatically verifying essential validity properties of staged reconfiguration processes with complex binding time constraints during DSPL domain engineering. The novel modeling concepts introduced are motivated by (re-)configuration constraints apparent in a real-world industrial case study from the automation engineering domain, which are not properly expressible and analyzable using state-of-the-art SPL domain modeling approaches. We present a prototypical tool implementation based on the model checker SPIN and present evaluation results obtained from our industrial case study, demonstrating the applicability of the approach.     

Feature-Oriented Nonfunctional Requirement Analysis for Software Product Line

Domain analysis in software product line (SPL) development provides a basis for core assets design and implementation by a systematic and comprehensive commonality/variability analysis. In feature-oriented SPL methods, products of the domain analysis are domain feature models and corresponding feature decision models to facilitate application-oriented customization. As in requirement analysis for a single system, the domain analysis in the SPL development should consider both functional and nonfunctional domain requirements. However, the nonfunctional requirements (NFRs) are often neglected in the existing domain analysis methods. In this paper, we propose a context-based method of the NFR analysis for the SPL development. In the method, NFRs are materialized by connecting nonfunctional goals with real-world context, thus NFR elicitation and variability analysis can be performed by context analysis for the whole domain with the assistance of NFR templates and NFR graphs. After the variability analysis, our method integrates both functional and nonfunctional perspectives by incorporating the nonfunctional goals and operationalizations into an initial functional feature model. NFR-related constraints are also elicited and integrated. Finally, a decision model with both functional and nonfunctional perspectives is constructed to facilitate application-oriented feature model customization. A computer-aided grading system (CAGS) product line is employed to demonstrate the method throughout the paper. This work is supported by the National Natural Science Foundation of China under Grant Nos. 60703092 and 90818009, the National High Technology Research and Development 863 Program of China under Grant No. 2007AA01Z125.     

BlackHorse: creating smart test cases from brittle recorded tests

Testing software that has a GUI is difficult. Manual testing is costly and error-prone, but recorded test cases frequently “break” due to changes in the GUI. Test cases intended to test business logic must therefore be converted to a less “brittle” form to lengthen their useful lifespan. In this paper, we describe BlackHorse, an approach to doing this that converts a recorded test case to Java code that bypasses the GUI. The approach was implemented within the testing environment of Research In Motion. We describe the design of the toolset and discuss lessons learned during the course of the project.     

Utilization of statistical process control (SPC) in emergent software organizations: Pitfalls and suggestions

Common wisdom in the domain of software engineering tells us that companies should be mature enough to apply Statistical Process Control (SPC) techniques. Since reaching high maturity levels (in CMM or similar models such as ISO 15504) usually takes 5–10 years, should software companies wait years to utilize Statistical Process Control techniques? To answer this question, we performed a case study of the application of SPC techniques using existing measurement data in an emergent software organization. Specifically, defect density, rework percentage and inspection performance metrics are analyzed. This paper provides a practical insight on the usability of SPC for the selected metrics in the specific processes and describes our observations on the difficulties and the benefits of applying SPC to an emergent software organization.     

Useful business cases: value creation in IS projects

Business cases have become popular as part of managing value creation in IS projects. Nevertheless, business cases are often poorly linked to value creation activities and organizations struggle to develop business cases that are useful and express more than simple cost savings. This action research study seeks to improve the usefulness of business cases in IS projects. We used collaborative action research with Danish municipalities to improve their practices when developing and using business cases and to change their perceptions of what constitutes a good business case that is useful during implementation and value creation. This article presents lessons learned from our action research, lessons that we incorporated into a business case method and subsequently evaluated with IS managers. There are three lessons on (1) improving the content of business cases, (2) how to develop business cases, and (3) the use of business cases in subsequent value creation. These lessons summarize our findings and the contributions are that we in the business case method propose to (1) include minimal contents, (2) develop social commitment, and (3) structure for dynamic use during value creation. We discuss the lessons and contributions related to research on IS business cases and value creation.