Performance appraisal of software testers

ContextTo determine the effectiveness of software testers a suitable performance appraisal approach is necessary, both for research and practice purposes. However, review of relevant literature reveals little information of how software testers are appraised in practice.Objective(i) To enhance our knowledge of industry practice of performance appraisal of software testers and (ii) to collect feedback from project managers on a proposed performance appraisal form for software testers.MethodA web-based survey with questionnaire was used to collect responses. Participants were recruited using cluster and snowball sampling. 18 software development project managers participated.ResultsWe found two broad trends in performance appraisal of software testers – same employee appraisal process for all employees and a specialized performance appraisal method for software testers. Detailed opinions were collected and analyzed on how performance of software testers should be appraised. Our proposed appraisal approach was generally well-received.ConclusionFactors such as number of bugs found after delivery and efficiency of executing test cases were considered important in appraising software testers’ performance. Our proposed approach was refined based on the feedback received.     

Current state and potential of variability management practices in software-intensive SMEs: Results from a regional industrial survey

ContextMore and more, small and medium-sized enterprises (SMEs) are using software to augment the functionality of their products and offerings. Variability management of software is becoming an interesting topic for SMEs with expanding portfolios and increasingly complex product structures. While the use of software product lines to resolve high variability is well known in larger organizations, there is less known about the practices in SMEs.ObjectiveThis paper presents results from a survey of software developing SMEs. The purpose of the paper is to provide a snapshot of the current awareness and practices of variability modeling in organizations that are developing software with the constraints present in SMEs.MethodA survey with questions regarding the variability practices was distributed to software developing organizations in a region of Sweden that has many SMEs. The response rate was 13% and 25 responses are used in this analysis.ResultsWe find that, although there are SMEs that develop implicit software product lines and have relatively sophisticated variability structures for their solution space, the structures of the problem space and the product space have room for improvement.ConclusionsThe answers in the survey indicate that SMEs are in situations where they can benefit from more structured variability management, but the awareness need to be raised. Even though the problem space similarity is high, there is little reuse and traceability activities performed. The existence of SMEs with qualified variability management and product line practices indicates that small organizations are capable to apply such practices.     

Software process modeling languages: A systematic literature review

ContextOrganizations working in software development are aware that processes are very important assets as well as they are very conscious of the need to deploy well-defined processes with the goal of improving software product development and, particularly, quality. Software process modeling languages are an important support for describing and managing software processes in software-intensive organizations.ObjectiveThis paper seeks to identify what software process modeling languages have been defined in last decade, the relationships and dependencies among them and, starting from the current state, to define directions for future research.MethodA systematic literature review was developed. 1929 papers were retrieved by a manual search in 9 databases and 46 primary studies were finally included.ResultsSince 2000 more than 40 languages have been first reported, each of which with a concrete purpose. We show that different base technologies have been used to define software process modeling languages. We provide a scheme where each language is registered together with the year it was created, the base technology used to define it and whether it is considered a starting point for later languages. This scheme is used to illustrate the trend in software process modeling languages. Finally, we present directions for future research.ConclusionThis review presents the different software process modeling languages that have been developed in the last ten years, showing the relevant fact that model-based SPMLs (Software Process Modeling Languages) are being considered as a current trend. Each one of these languages has been designed with a particular motivation, to solve problems which had been detected. However, there are still several problems to face, which have become evident in this review. This let us provide researchers with some guidelines for future research on this topic.     

Designing an integrated AHP based decision support system for supplier selection in automotive industry

PurposeThe purpose of this paper is to propose a decision support model for supplier selection based on analytic hierarchy process (AHP) using a case of automotive industry in a developing country of Pakistan and further performs sensitivity analysis to check the robustness of the supplier selection decision.MethodologyThe model starts by identifying the main criteria (price, quality, delivery and service) using literature review and ranking the main criteria based on experts’ opinions using AHP. The second stage in the adopted methodology is the identification of sub criteria and ranking them on the basis of main criteria. Lastly perform sensitivity analysis to check the robustness of the decision using Expert Choice^ۛ software.FindingsThe suppliers are selected and ranked based on sub criteria. Sensitivity analysis suggests the effects of changes in the main criteria on the suppliers ranking. The use of AHP in the supplier selection gives the decision maker the confidence of the consistency and the robustness throughout the process.Practical implicationsThe AHP methodology adopted in this study provides managers in automotive industry in Pakistan with the insights of the various factors that need to be considered while selecting suppliers for their organizations. The selected approach also aids them in prioritizing the criterion. Managers can utilize the hierarchical structure of adopted supplier selection methodology suggested in this study to rank the suppliers on the basis of various factors/criteria.Originality/valueThis study makes three novel contributions in supplier selection area. First, AHP is applied to automotive industry and use of AHP in the supplier selection gives decision maker the confidence of the consistency. Second, sensitivity analysis enables in understanding the effects of changes in the main criteria on the suppliers ranking and help decision maker to check the robustness throughout the process. Last, we find it important to come with a simple methodology for managers of automotive industry so that they can select the best suppliers. Moreover, this approach will also help managers in dividing the complex decision making problem into simpler hierarchy.     

The financial aspect of managing technical debt: A systematic literature review

ContextTechnical debt is a software engineering metaphor, referring to the eventual financial consequences of trade-offs between shrinking product time to market and poorly specifying, or implementing a software product, throughout all development phases. Based on its inter-disciplinary nature, i.e. software engineering and economics, research on managing technical debt should be balanced between software engineering and economic theories.ObjectiveThe aim of this study is to analyze research efforts on technical debt, by focusing on their financial aspect. Specifically, the analysis is carried out with respect to: (a) how financial aspects are defined in the context of technical debt and (b) how they relate to the underlying software engineering concepts.MethodIn order to achieve the abovementioned goals, we employed a standard method for SLRs and applied it on studies retrieved from seven general-scope digital libraries. In total we selected 69 studies relevant to the financial aspect of technical debt.ResultsThe most common financial terms that are used in technical debt research are principal and interest, whereas the financial approaches that have been more frequently applied for managing technical debt are real options, portfolio management, cost/benefit analysis and value-based analysis. However, the application of such approaches lacks consistency, i.e., the same approach is differently applied in different studies, and in some cases lacks a clear mapping between financial and software engineering concepts.ConclusionThe results are expected to prove beneficial for the communication between technical managers and project managers, in the sense that they will provide a common vocabulary, and will help in setting up quality-related goals, during software development. To achieve this we introduce: (a) a glossary of terms and (b) a classification scheme for financial approaches used for managing technical debt. Based on these, we have been able to underline interesting implications for researchers and practitioners.     

Requirements communication and balancing in large-scale software-intensive product development

ContextSeveral industries developing products on a large-scale are facing major challenges as their products are becoming more and more software-intensive. Whereas software was once considered a detail to be bundled, it has since become an intricate and interdependent part of most products. The advancement of software increases the uncertainty and the interdependencies between development tasks and artifacts. A key success factor is good requirements engineering (RE), and in particular, the challenges of effectively and efficiently coordinating and communicating requirements.ObjectiveIn this work we present a lightweight RE framework and demonstrate and evaluate its industrial applicability in response to the needs of a Swedish automotive company for improving specific problems in inter-departmental requirements coordination and communication in large-scale development of software-intensive systems.MethodA case study approach and a dynamic validation were used to develop and evaluate the framework in close collaboration with our industrial partner, involving three real-life cases in an ongoing car project. Experience and feedback were collected through observations when applying the framework and from 10 senior industry professionals in a questionnaire and in-depth follow-up interviews.ResultsThe experience and feedback about using the framework revealed that it is relevant and applicable for the industry as well as a useful and efficient way to resolve real problems in coordinating and communicating requirements identified at the case company. However, other concerns, such as accessibility to necessary resources and competences in the early development phases, were identified when using the method, which allowed for earlier pre-emptive action to be taken.ConclusionOverall, the experience from using the framework and the positive feedback from industry professionals indicated a feasible framework that is applicable in the industry for improving problems related to coordination and communication of requirements. Based on the promising results, our industrial partner has decided upon further validations of the framework in a large-scale pilot program.     

Relative importance of evaluation criteria for enterprise systems: a conjoint study

Abstract.  While a large body of research exists on the development and implementation of software, organizations are increasingly acquiring enterprise software packages [e.g. enterprise resource planning (ERP) systems] instead of custom developing their own software applications. To be competitive in the marketplace, software package development firms must manage the three-pronged trade-off between cost, quality and functionality. Surprisingly, prior research has made little attempt to investigate the characteristics of packaged software that influence management information system (MIS) managers' likelihood of recommending purchase. As a result, both the criteria by which MIS managers evaluate prospective packaged systems and the attributes that lead to commercially competitive ERP software products are poorly understood. This paper examines this understudied issue through a conjoint study. We focus on ERP systems, which are among the largest and most complex packaged systems that are purchased by organizations. In a conjoint study, 1008 evaluation decisions based on hypothetical ERP software package profiles were completed by managers in 126 organizations. The study represents the first empirical investigation of the relative importance that managers ascribe to various factors that are believed to be important in evaluating packaged software. The results provide important insights for both organizations that acquire such systems and those that develop them. The results show that functionality, reliability, cost, ease of use and ease of customization are judged to be important criteria, while ease of implementation and vendor reputation were not found to be significant. Functionality and reliability were found to be the most heavily weighted factors. We conclude the paper with a detailed discussion of the results and their implications for software acquisition and development practice.     

A Basis for Analyzing Software Architecture Analysis Methods

A software architecture is a key asset for any organization that builds complex software-intensive systems. Because of an architecture's central role as a project blueprint, organizations should analyze the architecture before committing resources to it. An analysis helps to ensure that sound architectural decisions are made. Over the past decade a large number of architecture analysis methods have been created, and at least two surveys of these methods have been published. This paper examines the criteria for analyzing architecture analysis methods, and suggests a new set of criteria that focus on the essence of what it means to be an architecture analysis method. These criteria could be used to compare methods, to help understand the suitability of a method, or to improve a method. We then examine two methods—the Architecture Tradeoff Analysis Method and Architecture-level Modifiability Analysis—in light of these criteria, and provide some insight into how these methods can be improved. Rick Kazman is a Senior Member of the Technical Staff at the Software Engineering Institute of Carnegie Mellon University and Professor at the University of Hawaii. His primary research interests are software architecture, design and analysis tools, software visualization, and software engineering economics. He also has interests in human-computer interaction and information retrieval. Kazman has created several highly influential methods and tools for architecture analysis, including the SAAM and the ATAM. He is the author of over 80 papers, and co-author of several books, including “Software Architecture in Practice”, and “Evaluating Software Architectures: Methods and Case Studies”. Len Bass is a Senior Member of the Technical Staff at the Software Engineering Institute (SEI). He has written two award winning books in software architecture as well as several other books and numerous papers in a wide variety of areas of computer science and software engineering. He is currently working on techniques for the methodical design of software architectures and to understand how to support usability through software architecture. He has been involved in the development of numerous different production or research software systems ranging from operating systems to database management systems to automotive systems. Mark Klein is Senior Member of the Technical Staff of the Software Engineering Institute. He has over 20 years of experience in research on various facets of software engineering, dependable real-time systems and numerical methods. Klein's most recent work focuses on the analysis of software architectures, architecture tradeoff analysis, attribute-driven architectural design and scheduling theory. Klein's work in real-time systems involved the development of rate monotonic analysis (RMA), the extension of the theoretical basis for RMA, and its application to realistic systems. Klein's earliest work involved research in high-order finite element methods for solving fluid flow equations arising in oil reservoir simulation. He is the co-author two books: “A Practitioner's Handbook for Real-Time Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems” and “Evaluating Software Architecture: Methods and Case Studies”. Anthony J. Lattanze is an Associate Teaching Professor at the Institute for Software Research International (ISRI) at Carnegie Mellon University (CMU) and a senior member of the technical staff at the Software Engineering Institute (SEI). Anthony teaches courses in CMUs Masters of Software Engineering Program in Software Architecture, Real-Time/Embedded Systems, and Software Development Studio. His primary research interest is in the area software architectural design for embedded, software intensive systems. Anthony consults and teaches throughout industry in the areas of software architecture design and architecture evaluation. Prior to Carnegie Mellon, Mr. Lattanze was the Chief of Software Engineering for the Technology Development Group at the United States Flight Test Center at Edwards Air Force Base, CA. During his tenure at the Flight Test Center, he was involved with a number of software and systems engineering projects as a software and systems architect, project manager, and developer. During this time as he was involved with the development, test, and evaluation of avionics systems for the B-2 Stealth Bomber, F-117 Stealth Fighter, and F-22 Advanced Tactical Fighter among other systems. Linda Northrop is the director of the Product Line Systems Program at the Software Engineering Institute (SEI) where she leads the SEI work in software architecture, software product lines and predictable component engineering. Under her leadership the SEI has developed software architecture and product line methods that are used worldwide, a series of five highly-acclaimed books, and Software Architecture and Software Product Line Curricula. She is co-author of the book, “Software Product Lines: Practices and Patterns,” and a primary author of the SEI Framework for Software Product Line Practice.     

What are the roles of software product managers? An empirical investigation

Software product management covers both technical and business activities to management of products like roadmaps, strategic, tactical, and release planning. In practice, one product manager is seldom responsible for all these activities but several persons share the responsibilities. Therefore, it is important to understand the boundaries of product managers’ work in managing software products, as well as the impact a product manager has on the company business. The purpose of the study is to clarify what roles of software product managers exist and understand how these roles are interrelated with each other and the whole structure and business of an organization. The study is designed as an interpretative qualitative study using grounded theory as the research method. Based on the gathered data we developed a framework that reveals the role of a product manager in the organization and shows how this role can evolve by extending the level of responsibilities. Using the framework, we identified four stereotypical roles of product managers in the studied organizations: experts, strategists, leaders, and problem solvers. The presented framework shows that product managers’ roles are not limited to the conception of the “mini-CEO.” The results allow product managers and top management to collaborate effectively by assigning responsibilities and managing expectations by having a common tool for understanding the role of product managers in the organization.     

Team building criteria in software projects: A mix-method replicated study

ContextThe internal composition of a work team is an important antecedent of team performance and the criteria used to select team members play an important role in determining team composition. However, there are only a handful of empirical studies about the use of team building criteria in the software industry.ObjectiveThe goal of this article is to identify criteria used in industrial practice to select members of a software project team, and to look for relationships between the use of these criteria and project success. In addition, we expect to contribute with findings about the use of replication in empirical studies involving human factors in software engineering.MethodOur research was based on an iterative mix-method, replication strategy. In the first iteration, we used qualitative research to identify team-building criteria interviewing software project managers from industry. Then, we performed a cross-sectional survey to assess the correlations of the use of these criteria and project success. In the second iteration, we used the results of a systematic mapping study to complement the set of team building criteria. Finally, we performed a replication of the survey research with variations to verify and improve the results.ResultsOur results showed that the consistent use team building criteria correlated significantly with project success, and the criteria related to human factors, such as personality and behavior, presented the strongest correlations. The results of the replication did not reproduce the results of the original survey with respect to the correlations between criteria and success goals. Nevertheless, the variations in the design and the difference in the sample of projects allowed us to conclude that the two results were compatible, increasing our confidence on the existence of the correlations.ConclusionOur findings indicated that carefully selecting team member for software teams is likely to positively influence the projects in which these teams participate. Besides, it seems that the type of development method used can moderate (increase or decrease) this influence. In addition, our study showed that the choice of sampling technique is not straightforward given the many interacting factors affecting this type of investigation.     

Adoption of open source software in software-intensive organizations – A systematic literature review

ContextOpen source software (OSS) is changing the way organizations develop, acquire, use, and commercialize software.ObjectiveThis paper seeks to identify how organizations adopt OSS, classify the literature according to these ways of adopting OSS, and with a focus on software development evaluate the research on adoption of OSS in organizations.MethodBased on the systematic literature review method we reviewed publications from 24 journals and seven conference and workshop proceedings, published between 1998 and 2008. From a population of 24,289 papers, we identified 112 papers that provide empirical evidence on how organizations actually adopt OSS.ResultsWe show that adopting OSS involves more than simply using OSS products. We moreover provide a classification framework consisting of six distinctly different ways in which organizations adopt OSS. This framework is used to illustrate some of the opportunities and challenges organizations meet when approaching OSS, to show that OSS can be adopted successfully in different ways, and to organize and review existing research. We find that existing research on OSS adoption does not sufficiently describe the context of the organizations studied, and it fails to benefit fully from related research fields. While existing research covers a large number of topics, it contains very few closely related studies. To aid this situation, we offer directions for future research.ConclusionThe implications of our findings are twofold. On the one hand, practitioners should embrace the many opportunities OSS offers, but consciously evaluate the consequences of adopting it in their own context. They may use our framework and the success stories provided by the literature in their own evaluations. On the other hand, researchers should align their work, and perform more empirical research on topics that are important to organizations. Our framework may be used to position this research and to describe the context of the organization they are studying.     

Understanding the characteristics of quality for software engineering processes: A Grounded Theory investigation

ContextSoftware engineering organizations routinely define and implement processes to support, guide and control project execution. An assumption underlying this process-centric approach to business improvement is that the quality of the process will influence the quality, cost and time-to-release of the software produced. A critical question thus arises of what constitutes quality for software engineering processes.ObjectiveTo identify criteria used by experienced practitioners to judge the quality of software engineering processes and to understand how knowledge of these criteria and their relationships may be useful for those undertaking software process improvement activities.MethodInterviews were conducted with 17 experienced software engineering practitioners from a range of geographies, roles and industry sectors. Published reports from 30 software process improvement case-studies were selected from multiple peer-reviewed software journals. A qualitative Grounded Theory-based methodology was employed to systematically analyze the collected data to synthesize a model of quality for software engineering processes.ResultsThe synthesized model suggests that practitioners perceive the overall quality of a software engineering process based on four quality attributes: suitability, usability, manageability and evolvability. Furthermore, these judgments are influenced by key properties related to the semantic content, structure, representation and enactment of that process. The model indicates that these attributes correspond to particular organizational perspectives and that these differing views may explain role-based conflicts in the judgement of process quality.ConclusionConsensus exists amongst practitioners about which characteristics of software engineering process quality most influence project outcomes. The model presented provides a terminological framework that can facilitate precise discussion of software engineering process issues and a set of criteria that can support activities for software process definition, evaluation and improvement. The potential exists for further development of this model to facilitate optimization of process properties to match organizational needs.     

Comparing the relative importance of evaluation criteria in proprietary and open‐source enterprise application software selection – a conjoint study of ERP and Office systems

Until recently, organizations willing to acquire application systems have had no choice but to adopt proprietary software. With the advent of open‐source software (OSS), a new model for developing and distributing software has entered the stage. OSS has evolved from a generally horizontal infrastructure towards more highly visible applications in vertical domains, giving information systems (IS) managers more degrees of freedom in their selection of enterprise application software (EAS). Although a large body of research exists on the relative importance of evaluation criteria for proprietary EAS, the role of OSS in the EAS evaluation process has received little attention so far. To address this research gap, this study represents the first empirical investigation to compare the relative importance of evaluation criteria in proprietary and open‐source EAS selection. Through an online survey, we evaluated the responses of IS managers of 358 organizations to a conjoint study spawning 8592 trade‐off pair comparisons and 3580 purchase evaluations on proprietary and open‐source enterprise resource planning (ERP) and Office software packages. The results show that the relative importance of evaluation criteria significantly varies between proprietary and open‐source ERP systems. Implementation factors such as ease of implementation and support are much more crucial in the evaluation of open‐source than of proprietary ERP systems, which is generally due to IS managers' risk mitigation behaviour. Interestingly, there are no major differences in the ranking of evaluation criteria between proprietary and open‐source Office systems. We conclude our paper with a detailed discussion of our findings and their implications for researchers, companies, EAS vendors and open‐source communities.     

How are software defects found? The role of implicit defect detection,individual responsibility,documents, and knowledge

ContextPrior research has focused heavily on explicit defect detection, such as formal testing and reviews. However, in reality, humans find software defects in various activities. Implicit defect detection activities, such as preparing a product demonstration or updating a user manual, are not designed for defect detection, yet through such activities defects are discovered. In addition, the type of documentation, and knowledge used, in defect detection is diverse.ObjectiveTo understand how defect detection is affected by the perspectives of responsibility, activity, knowledge, and document use. To provide illustrative numbers concerning the multidimensionality of defect detection in an industrial context.MethodThe data were collected with a survey on four software development organizations in three different companies. We designed the survey based on our prior extensive work with these companies.ResultsWe found that among our subjects (n = 105), implicit defect detection made a higher contribution than explicit defect detection in terms of found defects, 62% vs. 38%. We show that defect detection was performed by subjects in various roles supporting the earlier reports of testing being a cross-cutting activity in software development organizations. We found a low use of test cases (18%), but a high use of other documents in software defect detection, and furthermore, we found that personal knowledge was applied as an oracle in defect detection much more often than documented oracles. Finally, we recognize that contextual factors largely affect the transferability of our results, and we provide elaborate discussion about the most important contextual factors. Furthermore, we must be cautious as the results were obtained with a survey, and come from a small number of organizations.ConclusionsIn this paper, we show the large impact of implicit defect detection activities in four case organizations. Implicit defect detection has a large contribution to defect detection in practice, and can be viewed as an extremely low-cost way of detecting defects. Thus, harnessing and supporting it better may increase quality without increasing costs. For example, if an employee can update the user manual, and simultaneously detect defects from the software, then the defect detection part of this activity can be seen as cost-free. Additionally, further research is needed on how diverse types of useful documentation and knowledge can be utilized in defect detection.     

A systematic review and an expert survey on capabilities supporting multi product lines

ContextComplex software-intensive systems comprise many subsystems that are often based on heterogeneous technological platforms and managed by different organizational units. Multi product lines (MPLs) are an emerging area of research addressing variability management for such large-scale or ultra-large-scale systems. Despite the increasing number of publications addressing MPLs the research area is still quite fragmented.ObjectiveThe aims of this paper are thus to identify, describe, and classify existing approaches supporting MPLs and to increase the understanding of the underlying research issues. Furthermore, the paper aims at defining success-critical capabilities of infrastructures supporting MPLs.MethodUsing a systematic literature review we identify and analyze existing approaches and research issues regarding MPLs. Approaches described in the literature support capabilities needed to define and operate MPLs. We derive capabilities supporting MPLs from the results of the systematic literature review. We validate and refine these capabilities based on a survey among experts from academia and industry.ResultsThe paper discusses key research issues in MPLs and presents basic and advanced capabilities supporting MPLs. We also show examples from research approaches that demonstrate how these capabilities can be realized.ConclusionsWe conclude that approaches supporting MPLs need to consider both technical aspects like structuring large models and defining dependencies between product lines as well as organizational aspects such as distributed modeling and product derivation by multiple stakeholders. The identified capabilities can help to build, enhance, and evaluate MPL approaches.     

The impact of global dispersion on coordination,team performance and software quality – A systematic literature review

ContextGlobal software development (GSD) contains different context setting dimensions, which are essential for effective teamwork and success of projects. Although considerable research effort has been made in this area, as yet, no agreement has been reached about the impact of these dispersion dimensions on team coordination and project outcomes.ObjectiveThis paper summarizes empirical evidence on the impact of global dispersion dimensions on coordination, team performance and project outcomes.MethodWe performed a systematic literature review of 46 publications from 25 journals and 19 conference and workshop proceedings, which were published between 2001 and 2013. Thematic analysis was used to identify global dimensions and their measures. Vote counting was used to decide on the impact trends of dispersion dimensions on team performance and software quality.ResultsGlobal dispersion dimensions are consistently conceptualized, but quantified in many different ways. Different dispersion dimensions are associated with a distinct set of coordination challenges. Overall, geographical dispersion tends to have a negative impact on team performance and software quality. Temporal dispersion tends to have a negative impact on software quality, but its impact on team performance is inconsistent and can be explained by type of performance.ConclusionFor researchers, we reveal several opportunities for future research, such as coordination challenges in inter-organizational software projects, impact of processes and practices mismatches on project outcomes, evolution of coordination needs and mechanism over time and impact of dispersion dimensions on open source project outcomes. For practitioners, they should consider the tradeoff between cost and benefits while dispersing tasks, alignment impact of dispersion dimensions with individual and organizational objectives, coordination mechanisms as situational approaches and collocation of development activities of high quality demand components in GSD projects.     

Understanding Data-Driven Decision Support Systems

Abstract

It is important for managers and Information Technology professionals to understand data-driven decision support systems and how such systems can provide business intelligence and performance monitoring. Data-driven DSS is one of five major types of computerized decision support systems and the features of such systems vary across specific implementations. Different development packages also impact the capabilities of data-driven DSS and hence criteria for evaluating data-driven DSS development software are important to understand. Overall, this article builds on an historic foundation of prior decision support systems theory.     

Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption

ContextA software product line is a family of related software products, typically created from a set of common assets. Users select features to derive a product that fulfills their needs. Users often expect a product to have specific non-functional properties, such as a small footprint or a bounded response time. Because a product line may have an exponential number of products with respect to its features, it is usually not feasible to generate and measure non-functional properties for each possible product.ObjectiveOur overall goal is to derive optimal products with respect to non-functional requirements by showing customers which features must be selected.MethodWe propose an approach to predict a product’s non-functional properties based on the product’s feature selection. We aggregate the influence of each selected feature on a non-functional property to predict a product’s properties. We generate and measure a small set of products and, by comparing measurements, we approximate each feature’s influence on the non-functional property in question. As a research method, we conducted controlled experiments and evaluated prediction accuracy for the non-functional properties footprint and main-memory consumption. But, in principle, our approach is applicable for all quantifiable non-functional properties.ResultsWith nine software product lines, we demonstrate that our approach predicts the footprint with an average accuracy of 94%, and an accuracy of over 99% on average if feature interactions are known. In a further series of experiments, we predicted main memory consumption of six customizable programs and achieved an accuracy of 89% on average.ConclusionOur experiments suggest that, with only few measurements, it is possible to accurately predict non-functional properties of products of a product line. Furthermore, we show how already little domain knowledge can improve predictions and discuss trade-offs between accuracy and required number of measurements. With this technique, we provide a basis for many reasoning and product-derivation approaches.     

Supporting distributed product configuration by integrating heterogeneous variability modeling approaches

ContextIn industrial settings products are developed by more than one organization. Software vendors and suppliers commonly typically maintain their own product lines, which contribute to a larger (multi) product line or software ecosystem. It is unrealistic to assume that the participating organizations will agree on using a specific variability modeling technique—they will rather use different approaches and tools to manage the variability of their systems.ObjectiveWe aim to support product configuration in software ecosystems based on several variability models with different semantics that have been created using different notations.MethodWe present an integrative approach that provides a unified perspective to users configuring products in multi product line environments, regardless of the different modeling methods and tools used internally. We also present a technical infrastructure and a prototype implementation based on web services.ResultsWe show the feasibility of the approach and its implementation by using it with the three most widespread types of variability modeling approaches in the product line community, i.e., feature-based, OVM-style, and decision-oriented modeling. To demonstrate the feasibility and flexibility of our approach, we present an example derived from industrial experience in enterprise resource planning. We further applied the approach to support the configuration of privacy settings in the Android ecosystem based on multiple variability models. We also evaluated the performance of different model enactment strategies used in our approach.ConclusionsTools and techniques allowing stakeholders to handle variability in a uniform manner can considerably foster the initiation and growth of software ecosystems from the perspective of software reuse and configuration.     

Unifying software engineering and systems engineering

The author describes CMMI (Capability Maturity Model Integration) and the emerging project methods which demonstrate the opportunities for process improvement gains open to organizations. The organization that changes from separated software and system engineering processes to a more unified approach will find itself far more suited to developing dynamically changing, software-intensive systems. Culture change is never easy, but the alternative is even less palatable