Is lines of code a good measure of effort in effort-aware models?

ContextEffort-aware models, e.g., effort-aware bug prediction models aim to help practitioners identify and prioritize buggy software locations according to the effort involved with fixing the bugs. Since the effort of current bugs is not yet known and the effort of past bugs is typically not explicitly recorded, effort-aware bug prediction models are forced to use approximations, such as the number of lines of code (LOC) of the predicted files.ObjectiveAlthough the choice of these approximations is critical for the performance of the prediction models, there is no empirical evidence on whether LOC is actually a good approximation. Therefore, in this paper, we investigate the question: is LOC a good measure of effort for use in effort-aware models?MethodWe perform an empirical study on four open source projects, for which we obtain explicitly-recorded effort data, and compare the use of LOC to various complexity, size and churn metrics as measures of effort.ResultsWe find that using a combination of complexity, size and churn metrics are a better measure of effort than using LOC alone. Furthermore, we examine the impact of our findings on previous effort-aware bug prediction work and find that using LOC as a measure for effort does not significantly affect the list of files being flagged, however, using LOC under-estimates the amount of effort required compared to our best effort predictor by approximately 66%.ConclusionStudies using effort-aware models should not assume that LOC is a good measure of effort. For the case of effort-aware bug prediction, using LOC provides results that are similar to combining complexity, churn, size and LOC as a proxy for effort when prioritizing the most risky files. However, we find that for the purpose of effort-estimation, using LOC may under-estimate the amount of effort required.     

Managing complexity and variability of a model-based embedded software product line

This paper presents a framework for model-based product lines of embedded systems. We show how to integrate model-based product line techniques into a consistent framework that can deal with large product lines as they are common in industry. The framework demonstrates the strengths of model-based techniques like abstraction, support for customised representations, and a high degree of automation. In particular, we provide the following contributions: (1) to shift existing product lines towards a model-based approach, we support the (semi-) automated extraction of models from existing requirement, test, and implementation artefacts; (2) to cope with the complexity of artefacts and their interrelations in industrial product lines, we support the generation of context-specific views. These views support developers, e.g., in analysing complex dependencies between different artefacts; (3) finally, we support automated product derivation based on an integrated hardware abstraction layer. Most of the presented concepts have been inspired by challenges arising in the industrial application of product line techniques in the model-based engineering of embedded systems. We report on experiences gathered during the application of the techniques to a prototypical product line (on a rapid prototyping platform in the university lab) and to industrial sample cases (at the industry partner).     

Automatically propagating changes from reference implementations to code generation templates

ContextCode generators can automatically perform some tedious and error-prone implementation tasks, increasing productivity and quality in the software development process. Most code generators are based on templates, which are fundamentally composed of text expansion statements. To build templates, the code of an existing, tested and validated implementation may serve as reference, in a process known as templatization. With the dynamics of software evolution/maintenance and the need for performing changes in the code generation templates, there is a loss of synchronism between the templates and this reference code. Additional effort is required to keep them synchronized.ObjectiveThis paper proposes automation as a way to reduce the extra effort needed to keep templates and reference code synchronized.MethodA mechanism was developed to semi-automatically detect and propagate changes from reference code to templates, keeping them synchronized with less effort. The mechanism was also submitted to an empirical evaluation to analyze its effects in terms of effort reduction during maintenance/evolution templatization.ResultsIt was observed that the developed mechanism can lead to a 50% reduction in the effort needed to perform maintenance/evolution templatization, when compared to a manual approach. It was also observed that this effect depends on the nature of the evolution/maintenance task, since for one of the tasks there was no observable advantage in using the mechanism. However, further studies are needed to better characterize these tasks.ConclusionAlthough there is still room for improvement, the results indicate that automation can be used to reduce effort and cost in the maintenance and evolution of a template-based code generation infrastructure.     

Facilitating the evolution of products in product line engineering by capturing and replaying configuration decisions

Software product lines rely on developing reusable artifacts and defining their variability in models to support and accelerate the derivation of individual products. A major challenge in product lines is the continuous evolution of both the reusable artifacts and derived products. Products that have been derived from a product line have to be updated regularly, e.g., after bugfixes or the development of new features. Changes to reusable artifacts and variability models have to be propagated to derived products. The aim of our research is to provide automated support for the evolution of products derived from product lines by capturing and replaying configuration decisions. Our PUPLE (Product Updates in Product Line Engineering) approach supports updating derived products after changes to the product line they have been derived from. It exploits the structure of variability models and uses change-tracking data to minimize user intervention. The paper first explores how different types of product line changes influence the derived products. It then presents extensions to our decision-oriented product line approach DOPLER to support product line evolution. We evaluate the feasibility of the PUPLE approach with evolution tasks that were performed by engineers of an industry partner on a product line of an Eclipse-based tool suite with six derived products. We conclude with lessons learned and limitations of our approach.     

An expert system for determining candidate software classes for refactoring

In the lifetime of a software product, development costs are only the tip of the iceberg. Nearly 90% of the cost is maintenance due to error correction, adaptation and mainly enhancements. As Lehman and Belady [Lehman, M. M., & Belady, L. A. (1985). Program evolution: Processes of software change. Academic Press Professional.] state that software will become increasingly unstructured as it is changed. One way to overcome this problem is refactoring. Refactoring is an approach which reduces the software complexity by incrementally improving internal software quality. Our motivation in this research is to detect the classes that need to be rafactored by analyzing the code complexity. We propose a machine learning based model to predict classes to be refactored. We use Weighted Naïve Bayes with InfoGain heuristic as the learner and we conducted experiments with metric data that we collected from the largest GSM operator in Turkey. Our results showed that we can predict 82% of the classes that need refactoring with 13% of manual inspection effort on the average.     

Quantifying structural attributes of system decompositions in 28 feature-oriented software product lines

A key idea of feature orientation is to decompose a software product line along the features it provides. Feature decomposition is orthogonal to object-oriented decomposition—it crosscuts the underlying package and class structure. It has been argued often that feature decomposition improves system structure by reducing coupling and by increasing cohesion. However, recent empirical findings suggest that this is not necessarily the case. In this exploratory, observational study, we investigate the decompositions of 28 feature-oriented software product lines into classes, features, and feature-specific class fragments. The product lines under investigation are implemented using the feature-oriented programming language Fuji. In particular, we quantify and compare the internal attributes import coupling and cohesion of the different product-line decompositions in a systematic, reproducible manner. For this purpose, we adopt three established software measures (e.g., coupling between units, CBU; internal-ratio unit dependency, IUD) as well as standard concentration statistics (e.g., Gini coefficient). In our study, we found that feature decomposition can be associated with higher levels of structural coupling in a product line than a decomposition into classes. Although coupling can be concentrated in very few features in most feature decompositions, there are not necessarily hot-spot features in all product lines. Interestingly, feature cohesion is not necessarily higher than class cohesion, whereas features are more equal in serving dependencies internally than classes of a product line. Our empirical study raises critical questions about alleged advantages of feature decomposition. At the same time, we demonstrate how our measurement approach of coupling and cohesion has potential to support static and dynamic analyses of software product lines (i.e., type checking and feature-interaction detection) by facilitating product sampling.     

A study of several metrics for programming effort

As the cost of programming becomes a major component of the cost of computer systems, it becomes imperative that program development and maintenance be better managed. One measurement a manager could use is programming complexity. Such a measure can be very useful if the manager is confident that the higher the complexity measure is for a programming project, the more effort it takes to complete the project and perhaps to maintain it. Until recently most measures of complexity were based only on intuition and experience. In the past 3 years two objective metrics have been introduced, McCabe's cyclomatic number v(G) and Halstead's effort measure E. This paper reports an empirical study designed to compare these two metrics with a classic size measure, lines of code. A fourth metric based on a model of programming is introduced and shown to be better than the previously known metrics for some experimental data.     

The DOPLER meta-tool for decision-oriented variability modeling: a multiple case study

The variability of a product line is typically defined in models. However, many existing variability modeling approaches are rigid and don’t allow sufficient domain-specific adaptations. We have thus been developing a flexible and extensible approach for defining product line variability models. Its main purposes are to guide stakeholders through product derivation and to automatically generate product configurations. Our approach is supported by the DOPLER (Decision-Oriented Product Line Engineering for effective Reuse) meta-tool that allows modelers to specify the types of reusable assets, their attributes, and dependencies for their specific system and context. The aim of this paper is to investigate the suitability of our approach for different domains. More specifically, we explored two research questions regarding the implementation of variability and the utility of DOPLER for variability modeling in different domains. We conducted a multiple case study consisting of four cases in the domains of industrial automation systems and business software. In each of these case studies we analyzed variability implementation techniques. Experts from our industry partners then developed domain-specific meta-models, tool extensions, and variability models for their product lines using DOPLER. The four cases demonstrate the flexibility of the DOPLER approach and the extensibility and adaptability of the supporting meta tool.     

Predicting failure-proneness in an evolving software product line

ContextPrevious work by researchers on 3 years of early data for an Eclipse product has identified some predictors of failure-prone files that work well. Eclipse has also been used previously by researchers to study characteristics of product line software.ObjectiveThe work reported here investigates whether classification-based prediction of failure-prone files improves as the product line evolves.MethodThis investigation first repeats, to the extent possible, the previous study and then extends it by including four more recent years of data, comparing the prominent predictors with the previous results. The research then looks at the data for three additional Eclipse products as they evolve over time. The analysis compares results from three different types of datasets with alternative data collection and prediction periods.ResultsOur experiments with a variety of learners show that the difference between the performance of J48, used in this work, and the other top learners is not statistically significant. Furthermore, new results show that the effectiveness of classification significantly depends on the data collection period and prediction period. The study identifies change metrics that are prominent predictors across all four releases of all four products in the product line for the three different types of datasets. From the product line perspective, prediction of failure-prone files for the four products studied in the Eclipse product line shows statistically significant improvement in accuracy but not in recall across releases.ConclusionAs the product line matures, the learner performance improves significantly for two of the three datasets, but not for prediction of post-release failure-prone files using only pre-release change data. This suggests that it may be difficult to detect failure-prone files in the evolving product line. At least in part, this may be due to the continuous change, even for commonalities and high-reuse variation components, which we previously have shown to exist.     

Combining service-orientation and software product line engineering: A systematic mapping study

ContextService-Orientation (SO) is a rapidly emerging paradigm for the design and development of adaptive and dynamic software systems. Software Product Line Engineering (SPLE) has also gained attention as a promising and successful software reuse development paradigm over the last decade and proven to provide effective solutions to deal with managing the growing complexity of software systems.ObjectiveThis study aims at characterizing and identifying the existing research on employing and leveraging SO and SPLE.MethodWe conducted a systematic mapping study to identify and analyze related literature. We identified 81 primary studies, dated from 2000–2011 and classified them with respect to research focus, types of research and contribution.ResultThe mapping synthesizes the available evidence about combining the synergy points and integration of SO and SPLE. The analysis shows that the majority of studies focus on service variability modeling and adaptive systems by employing SPLE principles and approaches.In particular, SPLE approaches, especially feature-oriented approaches for variability modeling, have been applied to the design and development of service-oriented systems. While SO is employed in software product line contexts for the realization of product lines to reconcile the flexibility, scalability and dynamism in product derivations thereby creating dynamic software product lines.ConclusionOur study summarizes and characterizes the SO and SPLE topics researchers have investigated over the past decade and identifies promising research directions as due to the synergy generated by integrating methods and techniques from these two areas.     

Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption

ContextA software product line is a family of related software products, typically created from a set of common assets. Users select features to derive a product that fulfills their needs. Users often expect a product to have specific non-functional properties, such as a small footprint or a bounded response time. Because a product line may have an exponential number of products with respect to its features, it is usually not feasible to generate and measure non-functional properties for each possible product.ObjectiveOur overall goal is to derive optimal products with respect to non-functional requirements by showing customers which features must be selected.MethodWe propose an approach to predict a product’s non-functional properties based on the product’s feature selection. We aggregate the influence of each selected feature on a non-functional property to predict a product’s properties. We generate and measure a small set of products and, by comparing measurements, we approximate each feature’s influence on the non-functional property in question. As a research method, we conducted controlled experiments and evaluated prediction accuracy for the non-functional properties footprint and main-memory consumption. But, in principle, our approach is applicable for all quantifiable non-functional properties.ResultsWith nine software product lines, we demonstrate that our approach predicts the footprint with an average accuracy of 94%, and an accuracy of over 99% on average if feature interactions are known. In a further series of experiments, we predicted main memory consumption of six customizable programs and achieved an accuracy of 89% on average.ConclusionOur experiments suggest that, with only few measurements, it is possible to accurately predict non-functional properties of products of a product line. Furthermore, we show how already little domain knowledge can improve predictions and discuss trade-offs between accuracy and required number of measurements. With this technique, we provide a basis for many reasoning and product-derivation approaches.     

Structuring the modeling space and supporting evolution in software product line engineering

The scale and complexity of product lines means that it is practically infeasible to develop a single model of the entire system, regardless of the languages or notations used. The dynamic nature of real-world systems means that product line models need to evolve continuously to meet new customer requirements and to reflect changes of product line artifacts. To address these challenges, product line engineers need to apply different strategies for structuring the modeling space to ease the creation and maintenance of models. This paper presents an approach that aims at reducing the maintenance effort by organizing product lines as a set of interrelated model fragments defining the variability of particular parts of the system. We provide support to semi-automatically merge fragments into complete product line models. We also provide support to automatically detect inconsistencies between product line artifacts and the models representing these artifacts after changes. Furthermore, our approach supports the co-evolution of models and their respective meta-models. We discuss strategies for structuring the modeling space and show the usefulness of our approach using real-world examples from our ongoing industry collaboration.     

Supporting distributed product configuration by integrating heterogeneous variability modeling approaches

ContextIn industrial settings products are developed by more than one organization. Software vendors and suppliers commonly typically maintain their own product lines, which contribute to a larger (multi) product line or software ecosystem. It is unrealistic to assume that the participating organizations will agree on using a specific variability modeling technique—they will rather use different approaches and tools to manage the variability of their systems.ObjectiveWe aim to support product configuration in software ecosystems based on several variability models with different semantics that have been created using different notations.MethodWe present an integrative approach that provides a unified perspective to users configuring products in multi product line environments, regardless of the different modeling methods and tools used internally. We also present a technical infrastructure and a prototype implementation based on web services.ResultsWe show the feasibility of the approach and its implementation by using it with the three most widespread types of variability modeling approaches in the product line community, i.e., feature-based, OVM-style, and decision-oriented modeling. To demonstrate the feasibility and flexibility of our approach, we present an example derived from industrial experience in enterprise resource planning. We further applied the approach to support the configuration of privacy settings in the Android ecosystem based on multiple variability models. We also evaluated the performance of different model enactment strategies used in our approach.ConclusionsTools and techniques allowing stakeholders to handle variability in a uniform manner can considerably foster the initiation and growth of software ecosystems from the perspective of software reuse and configuration.     

MoDisco: A model driven reverse engineering framework

ContextMost companies, independently of their size and activity type, are facing the problem of managing, maintaining and/or replacing (part of) their existing software systems. These legacy systems are often large applications playing a critical role in the company’s information system and with a non-negligible impact on its daily operations. Improving their comprehension (e.g., architecture, features, enforced rules, handled data) is a key point when dealing with their evolution/modernization.ObjectiveThe process of obtaining useful higher-level representations of (legacy) systems is called reverse engineering (RE), and remains a complex goal to achieve. So-called Model Driven Reverse Engineering (MDRE) has been proposed to enhance more traditional RE processes. However, generic and extensible MDRE solutions potentially addressing several kinds of scenarios relying on different legacy technologies are still missing or incomplete. This paper proposes to make a step in this direction.MethodMDRE is the application of Model Driven Engineering (MDE) principles and techniques to RE in order to generate relevant model-based views on legacy systems, thus facilitating their understanding and manipulation. In this context, MDRE is practically used in order to (1) discover initial models from the legacy artifacts composing a given system and (2) understand (process) these models to generate relevant views (i.e., derived models) on this system.ResultsCapitalizing on the different MDRE practices and our previous experience (e.g., in real modernization projects), this paper introduces and details the MoDisco open source MDRE framework. It also presents the underlying MDRE global methodology and architecture accompanying this proposed tooling.ConclusionMoDisco is intended to make easier the design and building of model-based solutions dedicated to legacy systems RE. As an empirical evidence of its relevance and usability, we report on its successful application in real industrial projects and on the concrete experience we gained from that.     

Architectural evolution of FamiWare using cardinality-based feature models

ContextAmbient Intelligence systems domain is an outstanding example of modern systems that are in permanent evolution, as new devices, technologies or facilities are continuously appearing. This means it would be desirable to have a mechanism that helps with the propagation of evolution changes in deployed systems.ObjectiveWe present a software product line engineering process to manage the evolution of FamiWare, a family of middleware for ambient intelligence environments. This process drives the evolution of FamiWare middleware configurations using cardinality-based feature models, which are especially well suited to express the structural variability of ambient intelligence systems.MethodFamiWare uses cardinality-based feature models and clonable features to model the structural variability present in ambient intelligence systems, composed of a large variety of heterogeneous devices. Since the management evolution of configurations with clonable features is manually untreatable due to the high number of features, our process automates it and propagates changes made at feature level to the architectural components of the FamiWare middleware. This is a model driven development process as the evolution management, the propagation of evolution changes and the code generation are performed using some kind of model mappings and transformations. Concretely we present a variability modelling language to map the selection of features to the corresponding FamiWare middleware architectural components.ResultsOur process is able to manage the evolution of cardinality-based feature models with thousands of features, something which is not possible to tackle manually. Thanks to the use of the variability language and the automatic code generation it is possible to propagate and maintain a correspondence between the FamiWare architectural model and the code. The process is then able to calculate the architectural differences between the evolved configuration and the previous one. Checking these differences, our process helps to calculate the effort needed to perform the evolution changes in the customized products. To perform those tasks we have defined two operators, one to calculate the differences between two feature model configurations and another to create a new configuration from a previous one.ConclusionOur process automatically propagates the evolution changes of the middleware family into the existing configurations where the middleware is already deployed and also helps us to calculate the effort in performing the changes in every configuration. Finally, we validated our approach, demonstrating the functioning of the defined operators and showing that by using our tool we can generate evolved configurations for FamiWare with thousands of cloned features, for several case studies.     

SimPL: A product-line modeling methodology for families of integrated control systems

ContextIntegrated control systems (ICSs) are heterogeneous systems where software and hardware components are integrated to control and monitor physical devices and processes. A family of ICSs share the same software code base, which is configured differently for each product to form a unique installation. Due to the complexity of ICSs and inadequate automation support, product configuration in this context is typically error-prone and costly.ObjectiveAs a first step to overcome these challenges, we propose a UML-based product-line modeling methodology that provides a foundation for semi-automated product configuration in the specific context of ICSs.MethodWe performed a comprehensive domain analysis to identify characteristics of ICS families, and their configuration challenges. Based on this, we formulated the characteristics of an adequate configuration solution, and derived from them a set of modeling requirements for a model-based solution to configuration. The SimPL methodology is proposed to fulfill these requirements.ResultsTo evaluate the ability of SimPL to fulfill the modeling requirements, we applied it to a large-scale industrial case study. Our experience with the case study shows that SimPL is adequate to provide a model of the product family that meets the modeling requirements. Further evaluation is still required to assess the applicability and scalability of SimPL in practice. Doing this requires conducting field studies with human subjects and is left for future work.ConclusionWe conclude that configuration in ICSs requires better automation support, and UML-based approaches to product family modeling can be tailored to provide the required foundation.     

Investigating the impact of a measurement program on software quality

ContextMeasurement programs have been around for several decades but have been often misused or misunderstood by managers and developers. This misunderstanding prevented their adoption despite their many advantages.ObjectiveIn this paper, we present the results of an empirical study on the impact of a measurement program, MQL (“Mise en Qualité du Logiciel”, French for “Quality Software Development”), in an industrial context.MethodWe analyzed data collected on 44 industrial systems of different sizes: 22 systems were developed using MQL while the other 22 used ad-hoc approaches to assess and control quality (control group, referred to as “ad-hoc systems”). We studied the impact of MQL on a set of nine variables: six quality factors (maintainability, evolvability, reusability, robustness, testability, and architecture quality), corrective-maintenance effort, code complexity, and the presence of comments.ResultsOur results show that MQL had a clear positive impact on all the studied indicators. This impact is statistically significant for all the indicators but corrective-maintenance effort.ConclusionWe bring concrete evidence that a measurement program can have a significant, positive impact on the quality of software systems if combined with appropriate decision making procedures and corrective actions.     

Requirements for product derivation support: Results from a systematic literature review and an expert survey

ContextAn increasing number of publications in product line engineering address product derivation, i.e., the process of building products from reusable assets. Despite its importance, there is still no consensus regarding the requirements for product derivation support.ObjectiveOur aim is to identify and validate requirements for tool-supported product derivation.MethodWe identify the requirements through a systematic literature review and validate them with an expert survey.ResultsWe discuss the resulting requirements and provide implementation examples from existing product derivation approaches.ConclusionsWe conclude that key requirements are emerging in the research literature and are also considered relevant by experts in the field.     

Operationalised product quality models and assessment: The Quamoco approach

ContextSoftware quality models provide either abstract quality characteristics or concrete quality measurements; there is no seamless integration of these two aspects. Quality assessment approaches are, hence, also very specific or remain abstract. Reasons for this include the complexity of quality and the various quality profiles in different domains which make it difficult to build operationalised quality models.ObjectiveIn the project Quamoco, we developed a comprehensive approach aimed at closing this gap.MethodThe project combined constructive research, which involved a broad range of quality experts from academia and industry in workshops, sprint work and reviews, with empirical studies. All deliverables within the project were peer-reviewed by two project members from a different area. Most deliverables were developed in two or three iterations and underwent an evaluation.ResultsWe contribute a comprehensive quality modelling and assessment approach: (1) A meta quality model defines the structure of operationalised quality models. It includes the concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects, and allows modularisation to create modules for specific domains. (2) A largely technology-independent base quality model reduces the effort and complexity of building quality models for specific domains. For Java and C# systems, we refined it with about 300 concrete product factors and 500 measures. (3) A concrete and comprehensive quality assessment approach makes use of the concepts in the meta-model. (4) An empirical evaluation of the above results using real-world software systems showed: (a) The assessment results using the base model largely match the expectations of experts for the corresponding systems. (b) The approach and models are well understood by practitioners and considered to be both consistent and well suited for getting an overall view on the quality of a software product. The validity of the base quality model could not be shown conclusively, however. (5) The extensive, open-source tool support is in a mature state. (6) The model for embedded software systems is a proof-of-concept for domain-specific quality models.ConclusionWe provide a broad basis for the development and application of quality models in industrial practice as well as a basis for further extension, validation and comparison with other approaches in research.