Components meet aspects: Assessing design stability of a software product line

Context

It is important for Product Line Architectures (PLA) to remain stable accommodating evolutionary changes of stakeholder’s requirements. Otherwise, architectural modifications may have to be propagated to products of a product line, thereby increasing maintenance costs. A key challenge is that several features are likely to exert a crosscutting impact on the PLA decomposition, thereby making it more difficult to preserve its stability in the presence of changes. Some researchers claim that the use of aspects can ameliorate instabilities caused by changes in crosscutting features. Hence, it is important to understand which aspect-oriented (AO) and non-aspect-oriented techniques better cope with PLA stability through evolution.

Objective

This paper evaluates the positive and negative change impact of component and aspect based design on PLAs. The objective of the evaluation is to assess how aspects and components promote PLA stability in the presence of various types of evolutionary change. To support a broader analysis, we also evaluate the PLA stability of a hybrid approach (i.e. combined use of aspects and components) against the isolated use of component-based, OO, and AO approaches.

Method

An quantitative and qualitative analysis of PLA stability which involved four different implementations of a PLA: (i) an OO implementation, (ii) an AO implementation, (iii) a component-based implementation, and (iv) a hybrid implementation where both components and aspects are employed. Each implementation has eight releases and they are functionally equivalent. We used conventional metrics suites for change impact and modularity to measure the architecture stability evaluation of the 4 implementations.

Results

The combination of aspects and components promotes superior PLA resilience than the other PLAs in most of the circumstances.

Conclusion

It is concluded that the combination of aspects and components supports the design of high cohesive and loosely coupled PLAs. It also contributes to improve modularity by untangling feature implementation.     

On the relationship of concern metrics and requirements maintainability

Context

Maintainability has become one of the most essential attributes of software quality, as software maintenance has shown to be one of the most costly and time-consuming tasks of software development. Many studies reveal that maintainability is not often a major consideration in requirements and design stages, and software maintenance costs may be reduced by a more controlled design early in the software life cycle. Several problem factors have been identified as harmful for software maintainability, such as lack of upfront consideration of proper modularity choices. In that sense, the presence of crosscutting concerns is one of such modularity anomalies that possibly exert negative effects on software maintainability. However, to the date there is little or no knowledge about how characteristics of crosscutting concerns, observable in early artefacts, are correlated with maintainability.

Objective

In this setting, this paper introduces an empirical analysis where the correlation between crosscutting properties and two ISO/IEC 9126 maintainability attributes, namely changeability and stability, is presented.

Method

This correlation is based on the utilization of a set of concern metrics that allows the quantification of crosscutting, scattering and tangling.

Results

Our study confirms that a change in a crosscutting concern is more difficult to be accomplished and that artefacts addressing crosscutting concerns are found to be less stable later as the system evolves. Moreover, our empirical analysis reveals that crosscutting properties introduce non-syntactic dependencies between software artefacts, thereby decreasing the quality of software in terms of changeability and stability as well. These subtle dependencies cannot be easily detected without the use of concern metrics.

Conclusion

The correlation provides evidence that the presence of certain crosscutting properties negatively affects to changeability and stability. The whole analysis is performed using as target cases three software product lines, where maintainability properties are of upmost importance not only for individual products but also for the core architecture of the product line.     

Design guidelines for software processes knowledge repository development

Context

Staff turnover in organizations is an important issue that should be taken into account mainly for two reasons:

1.

Employees carry an organization’s knowledge in their heads and take it with them wherever they go

2.

Knowledge accessibility is limited to the amount of knowledge employees want to share

Objective

The aim of this work is to provide a set of guidelines to develop knowledge-based Process Asset Libraries (PAL) to store software engineering best practices, implemented as a wiki.

Method

Fieldwork was carried out in a 2-year training course in agile development. This was validated in two phases (with and without PAL), which were subdivided into two stages: Training and Project.

Results

The study demonstrates that, on the one hand, the learning process can be facilitated using PAL to transfer software process knowledge, and on the other hand, products were developed by junior software engineers with a greater degree of independence.

Conclusion

PAL, as a knowledge repository, helps software engineers to learn about development processes and improves the use of agile processes.     

The longitudinal, chronological case study research strategy: A definition, and an example from IBM Hursley Park

Context

There is surprisingly little empirical software engineering research (ESER) that has analysed and reported the rich, fine-grained behaviour of phenomena over time using qualitative and quantitative data. The ESER community also increasingly recognises the need to develop theories of software engineering phenomena e.g. theories of the actual behaviour of software projects at the level of the project and over time.

Objective

To examine the use of the longitudinal, chronological case study (LCCS) as a research strategy for investigating the rich, fine-grained behaviour of phenomena over time using qualitative and quantitative data.

Method

Review the methodological literature on longitudinal case study. Define the LCCS and demonstrate the development and application of the LCCS research strategy to the investigation of Project C, a software development project at IBM Hursley Park. Use the study to consider prospects for LCCSs, and to make progress on a theory of software project behaviour.

Results

LCCSs appear to provide insights that are hard to achieve using existing research strategies, such as the survey study. The LCCS strategy has basic requirements that data is time-indexed, relatively fine-grained and collected contemporaneous to the events to which the data refer. Preliminary progress is made on a theory of software project behaviour.

Conclusion

LCCS appears well suited to analysing and reporting rich, fine-grained behaviour of phenomena over time.     

Applying agglomerative hierarchical clustering algorithms to component identification for legacy systems

Context

Component identification, the process of evolving legacy system into finely organized component-based software systems, is a critical part of software reengineering. Currently, many component identification approaches have been developed based on agglomerative hierarchical clustering algorithms. However, there is a lack of thorough investigation on which algorithm is appropriate for component identification.

Objective

This paper focuses on analyzing agglomerative hierarchical clustering algorithms in software reengineering, and then identifying their respective strengths and weaknesses in order to apply them effectively for future practical applications.

Method

A series of experiments were conducted for 18 clustering strategies combined according to various similarity measures, weighting schemes and linkage methods. Eleven subject systems with different application domains and source code sizes were used in the experiments. The component identification results are evaluated by the proposed size, coupling and cohesion criteria.

Results

The experimental results suggested that the employed similarity measures, weighting schemes and linkage methods can have various effects on component identification results with respect to the proposed size, coupling and cohesion criteria, so the hierarchical clustering algorithms produced quite different clustering results.

Conclusions

According to the experimental results, it can be concluded that it is difficult to produce perfectly satisfactory results for a given clustering algorithm. Nevertheless, these algorithms demonstrated varied capabilities to identify components with respect to the proposed size, coupling and cohesion criteria.     

A controlled experiment in assessing and estimating software maintenance tasks

Context

Software maintenance is an important software engineering activity that has been reported to account for the majority of the software total cost. Thus, understanding the factors that influence the cost of software maintenance tasks helps maintainers to make informed decisions about their work.

Objective

This paper describes a controlled experiment of student programmers performing maintenance tasks on a C++ program. The objective of the study is to assess the maintenance size, effort, and effort distributions of three different maintenance types and to describe estimation models to predict the programmer’s effort spent on maintenance tasks.

Method

Twenty-three graduate students and a senior majoring in computer science participated in the experiment. Each student was asked to perform maintenance tasks required for one of the three task groups. The impact of different LOC metrics on maintenance effort was also evaluated by fitting the data collected into four estimation models.

Results

The results indicate that corrective maintenance is much less productive than enhancive and reductive maintenance and program comprehension activities require as much as 50% of the total effort in corrective maintenance. Moreover, the best software effort model can estimate the time of 79% of the programmers with the error of or less than 30%.

Conclusion

Our study suggests that the LOC added, modified, and deleted metrics are good predictors for estimating the cost of software maintenance. Effort estimation models for maintenance work may use the LOC added, modified, deleted metrics as the independent parameters instead of the simple sum of the three. Another implication is that reducing business rules of the software requires a sizable proportion of the software maintenance effort. Finally, the differences in effort distribution among the maintenance types suggest that assigning maintenance tasks properly is important to effectively and efficiently utilize human resources.     

Towards understanding the underlying structure of motivational factors for software engineers to guide the definition of motivational programs

Aim

In this article, factors influencing the motivation of software engineers is studied with the goal of guiding the definition of motivational programs.

Method

Using a set of 20 motivational factors compiled in a systematic literature review and a general theory of motivation, a survey questionnaire was created to evaluate the influence of these factors on individual motivation. Then, the questionnaire was applied on a semi-random sample of 176 software engineers from 20 software companies located in Recife-PE, Brazil.

Results

The survey results show the actual level of motivation for each motivator in the target population. Using principal component analysis on the values of all motivators, a five factor structure was identified and used to propose a guideline for the creation of motivational programs for software engineers.

Conclusions

The five factor structure provides an intuitive categorization for the set of variables and can be used to explain other motivational models presented in the literature. This contributes to a better understanding of motivation in software engineering.     

Testing in Service Oriented Architectures with dynamic binding: A mapping study

Context

Service Oriented Architectures (SOA) have emerged as a new paradigm to develop interoperable and highly dynamic applications.

Objective

This paper aims to identify the state of the art in the research on testing in Service Oriented Architectures with dynamic binding.

Method

A mapping study has been performed employing both manual and automatic search in journals, conference/workshop proceedings and electronic databases.

Results

A total of 33 studies have been reviewed in order to extract relevant information regarding a previously defined set of research questions. The detection of faults and the decision making based on the information gathered from the tests have been identified as the main objectives of these studies. To achieve these goals, monitoring and test case generation are the most proposed techniques testing both functional and non-functional properties. Furthermore, different stakeholders have been identified as participants in the tests, which are performed in specific points in time during the life cycle of the services. Finally, it has been observed that a relevant group of studies have not validated their approach yet.

Conclusions

Although we have only found 33 studies that address the testing of SOA where the discovery and binding of the services are performed at runtime, this number can be considered significant due to the specific nature of the reviewed topic. The results of this study have contributed to provide a body of knowledge that allows identifying current gaps in improving the quality of the dynamic binding in SOA using testing approaches.     

A two-stage framework for UML specification matching

Context

Specification matching techniques are crucial for effective retrieval processes. Despite the prevalence for object-oriented methodologies, little attention has been given to Unified Modeling Language (UML) for matching.

Objective

This paper presents a two-stage framework for matching two UML specifications and quantifying the results based on the systematic integration of their structural and behavioral similarities in order to identify the candidate component set for reuse.

Method

The first stage in the framework is an evaluation of the similarities between UML class diagrams using the Structure-Mapping Engine (SME), a simulation of the analogical reasoning approach known as the structure-mapping theory. The second stage, performed on the components identified in the first stage, is based on a graph-similarity scoring algorithm in which UML class diagrams and sequence diagrams are transformed into an SME representation and a Message-Object-Order Graph (MOOG). The effectiveness of the proposed framework was evaluated using a case study.

Results

The experimental results showed a reduction in potential mismatches and an overall high precision and recall.

Conclusion

It is concluded that the two-stage framework is capable of performing more precise matching compared to those of other single-stage matching frameworks. Moreover, the two-stage framework could be utilized within a reuse process, bypassing the need for extra information for retrieval of the components described by UML.     

Is software “green”? Application development environments and energy efficiency in open source applications

Context

The energy efficiency of IT systems, also referred to as Green IT, is attracting more and more attention. While several researchers have focused on the energy efficiency of hardware and embedded systems, the role of application software in IT energy consumption still needs investigation.

Objective

This paper aims to define a methodology for measuring software energy efficiency and to understand the consequences of abstraction layers and application development environments for the energy efficiency of software applications.

Method

We first develop a measure of energy efficiency that is appropriate for software applications. We then examine how the use of application development environments relates to this measure of energy efficiency for a sample of 63 open source software applications.

Results

Our findings indicate that a greater use of application development environments - specifically, frameworks and external libraries - is more detrimental in terms of energy efficiency for larger applications than for smaller applications. We also find that different functional application types have distinctly different levels of energy efficiency, with text and image editing and gaming applications being the most energy inefficient due to their intense use of the processor.

Conclusion

We conclude that different designs can have a significant impact on the energy efficiency of software applications. We have related the use of software application development environments to software energy efficiency suggesting that there may be a trade-off between development efficiency and energy efficiency. We propose new research to further investigate this topic.     

Data warehouse testing: A prototype-based methodology

Context

Testing is an essential part of the development life-cycle of any software product. While most phases of data warehouse design have received considerable attention in the literature, not much has been written about data warehouse testing.

Objective

In this paper we propose a comprehensive approach to testing data warehouse systems. Its main features are earliness with respect to the life-cycle, modularity, tight coupling with design, scalability, and measurability through proper metrics.

Method

We introduce a number of specific testing activities, we classify them in terms of what is tested and how it is tested, and we show how they can be framed within a prototype-based methodology. We apply our approach to a real case study for a large retail company.

Results

The case study we faced, based on an iterative prototype-based medium-size project, confirmed the validity of our approach. In particular, the main benefits were obtained in terms of project transparency, coordination of the development team, and organization of design activities.

Conclusion

Though some general-purpose testing techniques can be applied to data warehouse projects, the effectiveness of testing can be largely improved by applying specifically-devised techniques and metrics.     

A systematic mapping study of software product lines testing

Context

In software development, Testing is an important mechanism both to identify defects and assure that completed products work as specified. This is a common practice in single-system development, and continues to hold in Software Product Lines (SPL). Even though extensive research has been done in the SPL Testing field, it is necessary to assess the current state of research and practice, in order to provide practitioners with evidence that enable fostering its further development.

Objective

This paper focuses on Testing in SPL and has the following goals: investigate state-of-the-art testing practices, synthesize available evidence, and identify gaps between required techniques and existing approaches, available in the literature.

Method

A systematic mapping study was conducted with a set of nine research questions, in which 120 studies, dated from 1993 to 2009, were evaluated.

Results

Although several aspects regarding testing have been covered by single-system development approaches, many cannot be directly applied in the SPL context due to specific issues. In addition, particular aspects regarding SPL are not covered by the existing SPL approaches, and when the aspects are covered, the literature just gives brief overviews. This scenario indicates that additional investigation, empirical and practical, should be performed.

Conclusion

The results can help to understand the needs in SPL Testing, by identifying points that still require additional investigation, since important aspects regarding particular points of software product lines have not been addressed yet.     

Software product line testing - A systematic mapping study

Context

Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped.

Objective

This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research.

Method

A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type.

Results

A majority of the papers are of proposal research types (64%). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority.

Conclusions

More validation and evaluation research is needed to provide a better foundation for SPL testing.     

Process models for service-based applications: A systematic literature review

Context

Service-Oriented Computing (SOC) is a promising computing paradigm which facilitates the development of adaptive and loosely coupled service-based applications (SBAs). Many of the technical challenges pertaining to the development of SBAs have been addressed, however, there are still outstanding questions relating to the processes required to develop them.

Objective

The objective of this study is to systematically identify process models for developing service-based applications (SBAs) and review the processes within them. This will provide a useful starting point for any further research in the area. A secondary objective of the study is to identify process models which facilitate the adaptation of SBAs.

Method

In order to achieve this objective a systematic literature review (SLR) of the existing software engineering literature is conducted.

Results

During this research 722 studies were identified using a predefined search strategy, this number was narrowed down to 57 studies based on a set of strict inclusion and exclusion criteria. The results are reported both quantitatively in the form of a mapping study, as well as qualitatively in the form of a narrative summary of the key processes identified.

Conclusion

There are many process models reported for the development of SBAs varying in detail and maturity, this review has identified and categorised the processes within those process models. The review has also identified and evaluated process models which facilitate the adaptation of SBAs.     

SOAdapt: A process reference model for developing adaptable service-based applications

Context

The loose coupling of services and Service-Based Applications (SBAs) have made them the ideal platform for context-based run-time adaptation. There has been a lot of research into implementation techniques for adapting SBAs, without much effort focused on the software process required to guide the adaptation.

Objective

This paper aims to bridge that gap by providing an empirically grounded software process model that can be used by software practitioners who want to build adaptable SBAs. The process model will focus only on the adaptation specific issues.

Method

The process model presented in this paper is based on data collected through interviews with 10 practitioners occupying various roles within eight different companies. The data was analyzed using qualitative data analysis techniques. We used the output to develop a set of activities, tasks, stakeholders and artifacts that were used to construct the process model.

Results

The outcome of the data analysis process was a process model identifying nine sets of adaptation process attributes. These can be used in conjunction with an organisation’s existing development life-cycle or another reference life-cycle.

Conclusion

The process model developed in this paper provides a solid reference for practitioners who are planning to develop adaptable SBAs. It has advantages over similar approaches in that it focuses on software process rather than the specific adaptation mechanism implementation techniques.     

Measuring and predicting software productivity: A systematic map and review

Context

Software productivity measurement is essential in order to control and improve the performance of software development. For example, by identifying role models (e.g. projects, individuals, tasks) when comparing productivity data. The prediction is of relevance to determine whether corrective actions are needed, and to discover which alternative improvement action would yield the best results.

Objective

In this study we identify studies for software productivity prediction and measurement. Based on the identified studies we first create a classification scheme and map the studies into the scheme (systematic map). Thereafter, a detailed analysis and synthesis of the studies is conducted.

Method

As a research method for systematically identifying and aggregating the evidence of productivity measurement and prediction approaches systematic mapping and systematic review have been used.

Results

In total 38 studies have been identified, resulting in a classification scheme for empirical research on software productivity. The mapping allowed to identify the rigor of the evidence with respect to the different productivity approaches. In the detailed analysis the results were tabulated and synthesized to provide recommendations to practitioners.

Conclusion

Risks with simple ratio-based measurement approaches were shown. In response to the problems data envelopment analysis seems to be a strong approach to capture multivariate productivity measures, and allows to identify reference projects to which inefficient projects should be compared. Regarding simulation no general prediction model can be identified. Simulation and statistical process control are promising methods for software productivity prediction. Overall, further evidence is needed to make stronger claims and recommendations. In particular, the discussion of validity threats should become standard, and models need to be compared with each other.     

Agile methods rapidly replacing traditional methods at Nokia: A survey of opinions on agile transformation

Context

Many organizations have started to deploy agile methods, but so far there exist only a few studies on organization-wide transformations. Are agile methods here to stay? Some claim that agile software development methods are in the mainstream adoption phase in the software industry, while others hope that those are a passing fad. The assumption here is that if agile would not provide real improvement, adopters would be eager at first but turn pessimistic after putting it into practice.

Objective

Despite the growing amount of anecdotal evidence on the success of agile methods across a wide range of different real-life development settings, scientific studies remain scarce. Even less is known about the perception of the impacts of agile transformation when it is deployed in a very large software development environment, and whether agile methods are here to stay. This study aims to fill that gap by providing evidence from a large-scale agile transformation within Nokia. While we have yet to confirm these findings with solid quantitative data, we believe that the perception of the impacts already pinpoints the direction of the impacts of large-scale agile transformation.

Method

The data were collected using a questionnaire. The population of the study contains more than 1000 respondents in seven different countries in Europe, North America, and Asia.

Results

The results reveal that most respondents agree on all accounts with the generally claimed benefits of agile methods. These benefits include higher satisfaction, a feeling of effectiveness, increased quality and transparency, increased autonomy and happiness, and earlier detection of defects. Finally, 60% of respondents would not like to return to the old way of working.

Conclusion

While the perception of the impact of agile methods is predominantly positive, several challenge areas were discovered. However, based on this study, agile methods are here to stay.     

Resolving unwanted couplings through interactive exploration of co-evolving software entities - An experience report

Context

Frequent changes to groups of software entities belonging to different parts of the system may indicate unwanted couplings between those parts. Visualizations of co-changing software entities have been proposed to help developers identify unwanted couplings. Identifying unwanted couplings, however, is only the first step towards an important goal of a software architect: to improve the decomposition of the software system. An in-depth analysis of co-changing entities is needed to understand the underlying reasons for co-changes, and also determine how to resolve the issues.

Objective

In this paper we discuss how interactive visualizations can support the process of analyzing the identified unwanted couplings.

Method

We applied a tool that interactively visualizes software evolution in 10 working sessions with architects and developers of a large embedded software system having a development history of more than a decade.

Results

The participants of the working sessions were overall very positive about their experiences with the interactive visualizations. In 70% of the cases investigated, a decision could be taken on how to resolve the unwanted couplings.

Conclusion

Our experience suggests that interactive visualization not only helps to identify unwanted couplings but it also helps experts to reason about and resolve them.     

Six years of systematic literature reviews in software engineering: An updated tertiary study

Context

Since the introduction of evidence-based software engineering in 2004, systematic literature review (SLR) has been increasingly used as a method for conducting secondary studies in software engineering. Two tertiary studies, published in 2009 and 2010, identified and analysed 54 SLRs published in journals and conferences in the period between 1st January 2004 and 30th June 2008.

Objective

In this article, our goal was to extend and update the two previous tertiary studies to cover the period between 1st July 2008 and 31st December 2009. We analysed the quality, coverage of software engineering topics, and potential impact of published SLRs for education and practice.

Method

We performed automatic and manual searches for SLRs published in journals and conference proceedings, analysed the relevant studies, and compared and integrated our findings with the two previous tertiary studies.

Results

We found 67 new SLRs addressing 24 software engineering topics. Among these studies, 15 were considered relevant to the undergraduate educational curriculum, and 40 appeared of possible interest to practitioners. We found that the number of SLRs in software engineering is increasing, the overall quality of the studies is improving, and the number of researchers and research organisations worldwide that are conducting SLRs is also increasing and spreading.

Conclusion

Our findings suggest that the software engineering research community is starting to adopt SLRs consistently as a research method. However, the majority of the SLRs did not evaluate the quality of primary studies and fail to provide guidelines for practitioners, thus decreasing their potential impact on software engineering practice.