Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today??s testing approaches for SPLs validate products individually using different kinds of reuse techniques for testing. Because of their reusability and adaptability capabilities, model-based approaches are suitable to describe variability and are therefore frequently used for implementation and testing purposes of SPLs. Due to the enormous number of possible products, individual product testing becomes more and more infeasible. Pairwise testing offers one possibility to test a subset of all possible products. However, according to the best of our knowledge, there is no contribution discussing and rating this approach in the SPL context. In this contribution, we provide a mapping between feature models describing the common and variable parts of an SPL and a reusable test model in the form of statecharts. Thereby, we interrelate feature model-based coverage criteria and test model-based coverage criteria such as control and data flow coverage and are therefore able to discuss the potentials and limitations of pairwise testing. We pay particular attention to test requirements for feature interactions constituting a major challenge in SPL engineering. We give a concise definition of feature dependencies and feature interactions from a testing point of view, and we discuss adequacy criteria for SPL coverage under pairwise feature interaction testing and give a generalization to the T-wise case. The concept and implementation of our approach are evaluated by means of a case study from the automotive domain.     

Verifying feature models using OWL

Feature models are widely used in domain engineering to capture common and variant features among systems in a particular domain. However, the lack of a formal semantics and reasoning support of feature models has hindered the development of this area. Industrial experiences also show that methods and tools that can support feature model analysis are badly appreciated. Such reasoning tool should be fully automated and efficient. At the same time, the reasoning tool should scale up well since it may need to handle hundreds or even thousands of features a that modern software systems may have. This paper presents an approach to modeling and verifying feature diagrams using Semantic Web OWL ontologies. We use OWL DL ontologies to precisely capture the inter-relationships among the features in a feature diagram. OWL reasoning engines such as FaCT++ are deployed to check for the inconsistencies of feature configurations fully automatically. Furthermore, a general OWL debugger has been developed to tackle the disadvantage of lacking debugging aids for the current OWL reasoner and to complement our verification approach. We also developed a CASE tool to facilitate visual development, interchange and reasoning of feature diagrams in the Semantic Web environment.     

Automated diagnosis of feature model configurations

Software product-lines (SPLs) are software platforms that can be readily reconfigured for different project requirements. A key part of an SPL is a model that captures the rules for reconfiguring the software. SPLs commonly use feature models to capture SPL configuration rules. Each SPL configuration is represented as a selection of features from the feature model. Invalid SPL configurations can be created due to feature conflicts introduced via staged or parallel configuration or changes to the constraints in a feature model. When invalid configurations are created, a method is needed to automate the diagnosis of the errors and repair the feature selections.This paper provides two contributions to research on automated configuration of SPLs. First, it shows how configurations and feature models can be transformed into constraint satisfaction problems to automatically diagnose errors and repair invalid feature selections. Second, it presents empirical results from diagnosing configuration errors in feature models ranging in size from 100 to 5,000 features. The results of our experiments show that our CSP-based diagnostic technique can scale up to models with thousands of features.     

Visualization of variability and configuration options

When designing, constructing, and maintaining diverse and variable software systems, a key challenge is the complexity of systems. A potential approach to tackle this challenge are techniques from variability management and product line engineering to handle the diversity and variability. A key asset in variability management is a variability model, which explicitly specifies the commonalities and variability of a system and the constraints between variants. However, handling variability and configurations remains a challenge due to the complexity on a cognitive level as human engineers reach their limits in identifying, understanding, and using all relevant details. In this paper we address this issue by providing concepts for interactive visual tool support for the configuration of systems with the help of feature models. We discuss relevant principles from the area of information visualization and their application to the domain of feature model configuration. We discuss techniques for interactive configuration support based on a reasoning engine, which, e.g., ensures the validity of configurations. We illustrate our findings by a concrete tool solution called S2T2 Configurator.     

Behavioral adaptation of information systems through goal models

Customizing software to perfectly fit individual needs is becoming increasingly important in information systems engineering. Users want to be able to customize software behavior through reference to terms familiar to their diverse needs and experience. We present a requirements-driven approach to behavioral customization of software systems. Goal models are constructed to represent alternative behaviors that users can exhibit to achieve their goals. Customization information is then added to restrict the space of possibilities to those that fit specific users, contexts, or situations. Meanwhile, elements of the goal models are mapped to units of source code. This way, customization preferences posed at the requirements level are directly translated into system customizations. Our approach, which we apply to an on-line shopping cart system and an automated teller machine simulator, does not assume adoption of a particular development methodology, platform, or variability implementation technique and keeps the reasoning computation overhead from interfering with the execution of the configured application.     

Evolving feature model configurations in software product lines

The increasing complexity and cost of software-intensive systems has led developers to seek ways of reusing software components across development projects. One approach to increasing software reusability is to develop a software product-line (SPL), which is a software architecture that can be reconfigured and reused across projects. Rather than developing software from scratch for a new project, a new configuration of the SPL is produced. It is hard, however, to find a configuration of an SPL that meets an arbitrary requirement set and does not violate any configuration constraints in the SPL.Existing research has focused on techniques that produce a configuration of an SPL in a single step. Budgetary constraints or other restrictions, however, may require multi-step configuration processes. For example, an aircraft manufacturer may want to produce a series of configurations of a plane over a span of years without exceeding a yearly budget to add features.This paper provides three contributions to the study of multi-step configuration for SPLs. First, we present a formal model of multi-step SPL configuration and map this model to constraint satisfaction problems (CSPs). Second, we show how solutions to these SPL configuration problems can be automatically derived with a constraint solver by mapping them to CSPs. Moreover, we show how feature model changes can be mapped to our approach in a multi-step scenario by using feature model drift. Third, we present empirical results demonstrating that our CSP-based reasoning technique can scale to SPL models with hundreds of features and multiple configuration steps.     

The effects of visualization and interaction techniques on feature model configuration

A Software Product Line is a set of software systems of a domain, which share some common features but also have significant variability. A feature model is a variability modeling artifact which represents differences among software products with respect to variability relationships among their features. Having a feature model along with a reference model developed in the domain engineering lifecycle, a concrete product of the family is derived by selecting features in the feature model (referred to as the configuration process) and by instantiating the reference model. However, feature model configuration can be a cumbersome task because: 1) feature models may consist of a large number of features, which are hard to comprehend and maintain; and 2) many factors including technical limitations, implementation costs, stakeholders’ requirements and expectations must be considered in the configuration process. Recognizing these issues, a significant amount of research efforts has been dedicated to different aspects of feature model configuration such as automating the configuration process. Several approaches have been proposed to alleviate the feature model configuration challenges through applying visualization and interaction techniques. However, there have been limited empirical insights available into the impact of visualization and interaction techniques on the feature model configuration process. In this paper, we present a set of visualization and interaction interventions for representing and configuring feature models, which are then empirically validated to measure the impact of the proposed interventions. An empirical study was conducted by following the principles of control experiments in software engineering and by applying the well-known software quality standard ISO 9126 to operationalize the variables investigated in the experiment. The results of the empirical study revealed that the employed visualization and interaction interventions significantly improved completion time of comprehension and changing of the feature model configuration. Additionally, according to results, the proposed interventions are easy-to-use and easy-to-learn for the participants.     

Statistical prioritization for software product line testing: an experience report

Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion?, an embedded system providing helicopter landing assistance.     

Generative software product line development using variability-aware design patterns

Software Product Lines (SPLs) are an approach to reuse in-the-large that models a set of closely related software systems in terms of commonalities and variabilities. Design patterns are best practices for addressing recurring design problems in object-oriented source code. In the practice of implementing SPL, instances of certain design patterns are employed to handle variability, which makes these “variability-aware design patterns” a best practice for SPL design. However, currently there is no dedicated method for proactively developing SPLs using design patterns suitable for realizing variable functionality. In this paper, we present a method to perform generative SPL development with design patterns. We use role models to capture design patterns and their relation to a variability model. We further allow mapping of individual design pattern roles to (parts of) implementation elements to be generated (e.g., classes, methods) and check the conformance of the realization with the specification of the pattern. We provide definitions for the variability-aware versions of the design patterns Observer, Strategy, Template Method and Composite. Furthermore, we support generation of realizations in Java, C++ and UML class diagrams utilizing annotative, compositional and transformational variability realization mechanisms. Hence, we support proactive development of SPLs using design patterns to apply best practices for the realization of variability. We realize our concepts within the Eclipse IDE and demonstrate them within a case study.     

Dynamic decision models for staged software product line configuration

Software product line engineering practices offer desirable characteristics such as rapid product development, reduced time-to-market, and more affordable development costs as a result of systematic representation of the variabilities of a domain of discourse that leads to methodical reuse of software assets. The development lifecycle of a product line consists of two main phases: domain engineering, which deals with the understanding and formally modeling of the target domain, and application engineering that is concerned with the configuration of a product line into one concrete product based on the preferences and requirements of the stakeholders. The work presented in this paper focuses on the application engineering phase and builds both the theoretical and technological tools to assist the stakeholders in (a) understanding the complex interactions of the features of a product line; (b) eliciting the utility of each feature for the stakeholders and hence exposing the stakeholders’ otherwise implicit preferences in a way that they can more easily make decisions; and (c) dynamically building a decision model through interaction with the stakeholders and by considering the structural characteristics of software product line feature models, which will guide the stakeholders through the product configuration process. Initial exploratory empirical experiments that we have performed show that our proposed approach for helping stakeholders understand their feature preferences and its associated staged feature model configuration process is able to positively impact the quality of the end results of the application engineering process within the context of the limited number of participants. In addition, it has been observed that the offered tooling support is able to ease the staged feature model configuration process.     

Feature Nets: behavioural modelling of software product lines

Software product lines (SPLs) are diverse systems that are developed using a dual engineering process: (a) family engineering defines the commonality and variability among all members of the SPL, and (b) application engineering derives specific products based on the common foundation combined with a variable selection of features. The number of derivable products in an SPL can thus be exponential in the number of features. This inherent complexity poses two main challenges when it comes to modelling: firstly, the formalism used for modelling SPLs needs to be modular and scalable. Secondly, it should ensure that all products behave correctly by providing the ability to analyse and verify complex models efficiently. In this paper, we propose to integrate an established modelling formalism (Petri nets) with the domain of software product line engineering. To this end, we extend Petri nets to Feature Nets. While Petri nets provide a framework for formally modelling and verifying single software systems, Feature Nets offer the same sort of benefits for software product lines. We show how SPLs can be modelled in an incremental, modular fashion using Feature Nets, provide a Feature Nets variant that supports modelling dynamic SPLs, and propose an analysis method for SPL modelled as Feature Nets. By facilitating the construction of a single model that includes the various behaviours exhibited by the products in an SPL, we make a significant step towards efficient and practical quality assurance methods for software product lines.     

基于特征可变性建模教学服务管理系统的研究与实现

可变性建模是软件产品线领域研究热点,现阶段的研究局限在需求阶段,缺乏完整的理论体系,从领域工程到应用工程缺乏详细的指导。针对这些问题,本文改进特征模型构建方法和流程,使其支持软件全生命周期,加强各模型的映射关系,增强流程可操作性,保障模型间的一致性。最后,得到需求阶段和设计阶段的可变性模型,并成功应用到教学服务管理系统二次开发中。通过构件开发和效率比较,验证本文改进方法作为软件复用分支理论是实际可行的,并且能够提高开发效率。     

CLIPS-OWL: A framework for providing object-oriented extensional ontology queries in a production rule engine

In this paper, we define a framework, namely CLIPS-OWL, for enabling the CLIPS production rule engine to represent the extensional results of DL reasoning on OWL ontologies in the form of Object-Oriented (OO) models. The purpose of this transformation is to allow CLIPS to use these OO models as static query models that are able to answer extensional ontology queries directly by the RETE reasoning engine during the development of custom CLIPS production rule programs, without interfacing at runtime the external DL reasoner. In that way, any CLIPS-based application may enhance its functionality by incorporating ontological knowledge without modifying the architecture of the CLIPS rule engine. CLIPS-OWL has been implemented using the Pellet DL reasoner and the CLIPS Object-Oriented Language (COOL).     

Toward automated feature model configuration with optimizing non-functional requirements

ContextA software product line is a family of software systems that share some common features but also have significant variabilities. A feature model is a variability modeling artifact, which represents differences among software products with respect to the variability relationships among their features. Having a feature model along with a reference model developed in the domain engineering lifecycle, a concrete product of the family is derived by binding the variation points in the feature model (called configuration process) and by instantiating the reference model.ObjectiveIn this work we address the feature model configuration problem and propose a framework to automatically select suitable features that satisfy both the functional and non-functional preferences and constraints of stakeholders. Additionally, interdependencies between various non-functional properties are taken into account in the framework.MethodThe proposed framework combines Analytical Hierarchy Process (AHP) and Fuzzy Cognitive Maps (FCM) to compute the non-functional properties weights based on stakeholders’ preferences and interdependencies between non-functional properties. Afterwards, Hierarchical Task Network (HTN) planning is applied to find the optimal feature model configuration.ResultOur approach improves state-of-art of feature model configuration by considering positive or negative impacts of the features on non-functional properties, the stakeholders’ preferences, and non-functional interdependencies. The approach presented in this paper extends earlier work presented in [1] from several distinct perspectives including mechanisms handling interdependencies between non-functional properties, proposing a novel tooling architecture, and offering visualization and interaction techniques for representing functional and non-functional aspects of feature models.Conclusionour experiments show the scalability of our configuration approach when considering both functional and non-functional requirements of stakeholders.     

A holistic approach to feature modeling for product line requirements engineering

Requirements engineering (RE) offers the means to discover, model, and manage the requirements of the products that comprise a product line, while software product line engineering (SPLE) offers the means of realizing the products’ requirements from a common base of software assets. In practice, however, RE and SPLE have proven to be less complementary than they should. While some RE techniques, particularly goal modeling, support the exploration of alternative solutions, the appropriate solution is typically conditional on context and a large product line may have many product-defining contexts. Thus, scalability and traceability through into product line features are key challenges for RE. Feature modeling, by contrast, has been widely accepted as a way of modeling commonality and variability of products of a product line that may be very complex. In this paper, we propose a goal-driven feature modeling approach that separates a feature space in terms of problem space and solution space features, and establish explicit mappings between them. This approach contributes to reducing the inherent complexity of a mixed-view feature model, deriving key engineering drivers for developing core assets of a product line, and facilitating the quality-based product configuration.     

Defining variability in activity diagrams and Petri nets

Control flow models, such as UML activity diagrams or Petri nets, are widely accepted modeling languages used to support quality assurance activities in single system engineering as well as software product line (SPL) engineering. Quality assurance in product line engineering is a challenging task since a defect in a domain artifact may affect several products of the product line. Thus, proper quality assurance approaches need to pay special attention to the product line variability. Automation is essential to support quality assurance approaches. A prerequisite for automation is a profound formalization of the underlying control flow models and, in the context of SPLs, of the variability therein.     

Lifting transformational models of product lines: a case study

Model driven engineering (MDE) of software product lines (SPLs) merges two increasing important paradigms that synthesize programs by transformation. MDE creates programs by transforming models, and SPLs elaborate programs by applying transformations called features. In this paper, we present the design and implementation of a transformational model of a product line of scalar vector graphics and JavaScript applications. We explain how we simplified our implementation by lifting selected features and their compositions from our original product line (whose implementations were complex) to features and their compositions of another product line (whose specifications were simple). We used operators to map higher-level features and their compositions to their lower-level counterparts. Doing so exposed commuting relationships among feature compositions in both product lines that helped validate our model and implementation.