Intelligent lessons learned systems

Lessons learned processes have been deployed in commercial, government, and military organizations since the late 1980s to capture, store, disseminate, and share experiential working knowledge. However, recent studies have shown that software systems for supporting lesson dissemination do not effectively promote knowledge sharing. We found that the problems with these systems are related to their textual representation for lessons and that they are not incorporated into the processes they are intended to support. In this article, we survey lessons learned processes and systems, detail their capabilities and limitations, examine lessons learned system design issues, and identify how artificial intelligence technologies can contribute to knowledge management solutions for these systems.     

Managing object oriented framework reuse

Reusing frameworks instead of libraries can cause subtle architectural changes in an application, calling for innovative management solutions. We relate our experience in managing the Knowledge-Based Software Assistant project and offer tips for buying, building and using frameworks. One of the promises of object-oriented software development is that organizations can get a significant return on development investment because the code is easier to reuse. Software project managers are often eager to take the OO plunge for that reason, but are uncertain about the management issues they will face. There is also the problem of choosing the best form of reuse. Library-based reuse, the traditional reuse form, is more popular than framework-based reuse, but we have found that framework-based reuse offers many more benefits with the right management approach. We describe the lessons we learned when building the Knowledge-Based Software Assistant/Advanced Development Model     

A Basis for Analyzing Software Architecture Analysis Methods

A software architecture is a key asset for any organization that builds complex software-intensive systems. Because of an architecture's central role as a project blueprint, organizations should analyze the architecture before committing resources to it. An analysis helps to ensure that sound architectural decisions are made. Over the past decade a large number of architecture analysis methods have been created, and at least two surveys of these methods have been published. This paper examines the criteria for analyzing architecture analysis methods, and suggests a new set of criteria that focus on the essence of what it means to be an architecture analysis method. These criteria could be used to compare methods, to help understand the suitability of a method, or to improve a method. We then examine two methods—the Architecture Tradeoff Analysis Method and Architecture-level Modifiability Analysis—in light of these criteria, and provide some insight into how these methods can be improved. Rick Kazman is a Senior Member of the Technical Staff at the Software Engineering Institute of Carnegie Mellon University and Professor at the University of Hawaii. His primary research interests are software architecture, design and analysis tools, software visualization, and software engineering economics. He also has interests in human-computer interaction and information retrieval. Kazman has created several highly influential methods and tools for architecture analysis, including the SAAM and the ATAM. He is the author of over 80 papers, and co-author of several books, including “Software Architecture in Practice”, and “Evaluating Software Architectures: Methods and Case Studies”. Len Bass is a Senior Member of the Technical Staff at the Software Engineering Institute (SEI). He has written two award winning books in software architecture as well as several other books and numerous papers in a wide variety of areas of computer science and software engineering. He is currently working on techniques for the methodical design of software architectures and to understand how to support usability through software architecture. He has been involved in the development of numerous different production or research software systems ranging from operating systems to database management systems to automotive systems. Mark Klein is Senior Member of the Technical Staff of the Software Engineering Institute. He has over 20 years of experience in research on various facets of software engineering, dependable real-time systems and numerical methods. Klein's most recent work focuses on the analysis of software architectures, architecture tradeoff analysis, attribute-driven architectural design and scheduling theory. Klein's work in real-time systems involved the development of rate monotonic analysis (RMA), the extension of the theoretical basis for RMA, and its application to realistic systems. Klein's earliest work involved research in high-order finite element methods for solving fluid flow equations arising in oil reservoir simulation. He is the co-author two books: “A Practitioner's Handbook for Real-Time Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems” and “Evaluating Software Architecture: Methods and Case Studies”. Anthony J. Lattanze is an Associate Teaching Professor at the Institute for Software Research International (ISRI) at Carnegie Mellon University (CMU) and a senior member of the technical staff at the Software Engineering Institute (SEI). Anthony teaches courses in CMUs Masters of Software Engineering Program in Software Architecture, Real-Time/Embedded Systems, and Software Development Studio. His primary research interest is in the area software architectural design for embedded, software intensive systems. Anthony consults and teaches throughout industry in the areas of software architecture design and architecture evaluation. Prior to Carnegie Mellon, Mr. Lattanze was the Chief of Software Engineering for the Technology Development Group at the United States Flight Test Center at Edwards Air Force Base, CA. During his tenure at the Flight Test Center, he was involved with a number of software and systems engineering projects as a software and systems architect, project manager, and developer. During this time as he was involved with the development, test, and evaluation of avionics systems for the B-2 Stealth Bomber, F-117 Stealth Fighter, and F-22 Advanced Tactical Fighter among other systems. Linda Northrop is the director of the Product Line Systems Program at the Software Engineering Institute (SEI) where she leads the SEI work in software architecture, software product lines and predictable component engineering. Under her leadership the SEI has developed software architecture and product line methods that are used worldwide, a series of five highly-acclaimed books, and Software Architecture and Software Product Line Curricula. She is co-author of the book, “Software Product Lines: Practices and Patterns,” and a primary author of the SEI Framework for Software Product Line Practice.     

Trends in software reuse research: A tertiary study

ContextThe reuse of software has been a research topic for more than 50 years. Throughout that time, many approaches, tools and proposed techniques have reached maturity. However, it is not yet a widespread practice and some issues need to be further investigated. The latest study on software reuse trends dates back to 2005 and we think that it should be updated.ObjectiveTo identify the current trends in software reuse research.MethodA tertiary study based on systematic secondary studies published up to July 2018.ResultsWe identified 4,423 works related to software reuse, from which 3,102 were filtered by selection criteria and quality assessment to produce a final set of 56 relevant studies. We identified 30 current research topics and 127 proposals for future work, grouped into three broad categories: Software Product Lines, Other reuse approaches and General reuse topics.ConclusionsFrequently reported topics include: Requirements and Testing in the category of Lifecycle phases for Software Product Lines, and Systematic reuse for decision making in the category of General Reuse. The most mentioned future work proposals were Requirements, and Evolution and Variability management for Software Product Lines, and Systematic reuse for decision making. The identified trends, based on future work proposals, demonstrate that software reuse is still an interesting area for research. Researchers can use these trends as a guide to lead their future projects.     

Philips experiences in global distributed software development

Global software development is increasingly common. Main expected benefits are improvements in time-to-market efficiency and access to greater—and less costly—resources. A number of problems are still to be solved before the full potential of global development can be obtained. This paper describes the experience of over 10 years of global distributed development at Philips, derived from about 200 projects. We discuss the experience and lessons learnt from multi-site development. Main lessons learned are that explicit agreements and ways of working should be defined for the following areas needing the most attention; team coordination and communication, requirements and architectures, integration, and configuration management. In addition, we discuss the experience gained from subcontracting software development to suppliers. Main lesson learned from subcontracting software development is the need for explicit attention and ways of working with respect to selection of suppliers, specification of the work to be subcontracted and establishment and content of the contract.     

Capturing quality requirements of product family architecture

Software quality is one of the major issues with software intensive systems. Moreover, quality is a critical success factor in software product families exploiting shared architecture and common components in a set of products. Our contribution is the QRF (Quality Requirements of a software Family) method, which explicitly focuses on how quality requirements have to be defined, represented and transformed to architectural models. The method has been applied to two experiments; one in a laboratory environment and the other in industry. The use of the QRF method is exemplified by the Distribution Service Platform (DiSeP), the laboratory experiment. The lessons learned are also based on our experiences of applying the method in industrial settings.     

Performance appraisal of software testers

ContextTo determine the effectiveness of software testers a suitable performance appraisal approach is necessary, both for research and practice purposes. However, review of relevant literature reveals little information of how software testers are appraised in practice.Objective(i) To enhance our knowledge of industry practice of performance appraisal of software testers and (ii) to collect feedback from project managers on a proposed performance appraisal form for software testers.MethodA web-based survey with questionnaire was used to collect responses. Participants were recruited using cluster and snowball sampling. 18 software development project managers participated.ResultsWe found two broad trends in performance appraisal of software testers – same employee appraisal process for all employees and a specialized performance appraisal method for software testers. Detailed opinions were collected and analyzed on how performance of software testers should be appraised. Our proposed appraisal approach was generally well-received.ConclusionFactors such as number of bugs found after delivery and efficiency of executing test cases were considered important in appraising software testers’ performance. Our proposed approach was refined based on the feedback received.     

Practical experiences in modelling software engineering practices: The project patterns approach

Software process improvement in software development organisations is a complex task that can be solved using knowledge management strategies. In this area, the definition and use of process patterns are a proven approach to apply knowledge management strategies in software engineering organisations. One of the main problems for the effective application of process patterns in the software industry is the difficulty of formalising the knowledge about the development process using these approaches. This study presents a framework to manage software project patterns. This framework (which is composed of a metamodel and a platform for patterns modelling and reuse) is able to formalise the knowledge on software development projects including software engineers’ previous experience, development methodologies, references frameworks and lessons learnt. The authors carried out an empirical study at Carlos III University of Madrid, where junior software engineers used the project patterns defined in this research work. The evidences and findings obtained during the empirical study execution indicates that correctness of the pattern depends on relevance of the bibliographic references used to create it, implementation of a knowledge sharing strategy among the personnel involved and previous experience in the business areas related to the information systems being developed. The results obtained from the empirical study also envisage that the usefulness of an sdPP (Software Development Project Pattern) depends on the ease of identifying when and how to apply a specific sdPP in a software project.     

Software development in startup companies: A systematic mapping study

ContextSoftware startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present a unique combination of characteristics which pose several challenges to software development activities.ObjectiveThis study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers.MethodWe conducted a systematic mapping study, developing a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups.ResultsA total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed.ConclusionThis mapping study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.     

SEI's software product line tenets

Software product lines are emerging as a viable, important software development paradigm. Based on the Software Engineering Institute's research and experience, the concepts, activities, and practices described here can lead to successful product line development. How-to's, success stories, and lessons learned expand on the approach.     

Applying empirical software engineering to software architecture: challenges and lessons learned

In the last 15 years, software architecture has emerged as an important software engineering field for managing the development and maintenance of large, software-intensive systems. Software architecture community has developed numerous methods, techniques, and tools to support the architecture process (analysis, design, and review). Historically, most advances in software architecture have been driven by talented people and industrial experience, but there is now a growing need to systematically gather empirical evidence about the advantages or otherwise of tools and methods rather than just rely on promotional anecdotes or rhetoric. The aim of this paper is to promote and facilitate the application of the empirical paradigm to software architecture. To this end, we describe the challenges and lessons learned when assessing software architecture research that used controlled experiments, replications, expert opinion, systematic literature reviews, observational studies, and surveys. Our research will support the emergence of a body of knowledge consisting of the more widely-accepted and well-formed software architecture theories.     

Two controlled experiments on model-based architectural decision making

ContextIn recent years, architectural design decisions are becoming more and more common for documenting software architectures. Rather than describing the structure of software systems, architectural decisions capture the design rationale and – often reusable – architectural knowledge. Many approaches and tools have been proposed in the literature to support architectural decision making and documentation (for instance, based on models, ontologies, or templates). In this context, the capturing, organization, and effective reuse of architectural knowledge has gained a lot of attention.ObjectiveHowever, there is little empirical evidence about the supportive effect of reusable architectural knowledge on the effectiveness and efficiency of architectural decision making.MethodTo investigate these aspects, we conducted two separate controlled experiments with software architecture students in which we tested the supportive effect of reusable decision models in decision making and documentation.ResultsOur results show that the use of reusable decision models can significantly increase both the efficiency and the effectiveness of novice architects.ConclusionWe can report, that our findings are in line with similar studies and support the claims regarding reusable architectural design decisions in principle.     

Goodbye!

In his final Loyal Opposition column for IEEE Software, Robert Glass points out the continued need for testing academic theories in practice and for practitioners to discuss lessons learned. He also states that software estimation is a deeply flawed activity and that software practitioners should always remain open-minded.     

Knowledge management initiatives in software testing: A mapping study

ContextSoftware testing is a knowledge intensive process, and, thus, Knowledge Management (KM) principles and techniques should be applied to manage software testing knowledge.ObjectiveThis study conducts a survey on existing research on KM initiatives in software testing, in order to identify the state of the art in the area as well as the future research. Aspects such as purposes, types of knowledge, technologies and research type are investigated.MethodThe mapping study was performed by searching seven electronic databases. We considered studies published until December 2013. The initial resulting set was comprised of 562 studies. From this set, a total of 13 studies were selected. For these 13, we performed snowballing and direct search to publications of researchers and research groups that accomplished these studies.ResultsFrom the mapping study, we identified 15 studies addressing KM initiatives in software testing that have been reviewed in order to extract relevant information on a set of research questions.ConclusionsAlthough only a few studies were found that addressed KM initiatives in software testing, the mapping shows an increasing interest in the topic in the recent years. Reuse of test cases is the perspective that has received more attention. From the KM point of view, most of the studies discuss aspects related to providing automated support for managing testing knowledge by means of a KM system. Moreover, as a main conclusion, the results show that KM is pointed out as an important strategy for increasing test effectiveness, as well as for improving the selection and application of suited techniques, methods and test cases. On the other hand, inadequacy of existing KM systems appears as the most cited problem related to applying KM in software testing.     

PEM: The small company-dedicated software process quality evaluation method combining CMMISM and ISO/IEC 14598

Many small software organizations have recognized the need to improve their software product. Evaluating the software product alone seems insufficient since it is known that its quality is largely dependant on the process that is used to create it. Thus, small organizations are asking for evaluation of their software processes and products. The ISO/IEC 14598-5 standard is already used as a methodology basis for evaluating software products. This article explores how it can be combined with the CMMI to produce a methodology that can be tailored for process evaluation in order to improve their software processes. ^SM: CMMI is a service mark of Carnegie-Mellon University. Sylvie Trudel has over 20 years of experience in software. She worked for more than 10 years in development and implementation of management information systems and embedded real-time systems. Since 1996, she works as a process improvement specialist, implementing best practices into organizations processes from CMM and CMMI models. She performed several CMM and CMMI assessments and participated in many other CMM assessments such as CBA IPI, SCE, and other proprietary methods. She obtained a bachelors degree in computer science in 1986 from Laval University in Québec City and a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal. Sylvie is currently working as a software engineering advisor at the Centre de Recherche Informatique de Montréal (CRIM). Jean-Marc Lavoie has been working in software development for over 10 years. He performed and published a comparative study between the guide to the SWEBOK and the CMMI in 2003. Jean-Marc obtained a bachelor degree in Electrical Engineering. He is pursuing a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal while working as a software architect at Trisotech. Marie-Claude Pare has been working in software development for 7 years. Marie-Claude obtained a bachelor degree in Software Engineering from école Polytechnique in Montréal. She is pursuing a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal while working as a software engineer at Motorola GSG Canada. Dr Witold Suryn is a Professor at the école de technologie supérieure, Montreal, Canada (engineering school of the Université du Québec network of institutions) where he teaches graduate and undergraduate software engineering courses and conducts research in the domain of software quality engineering, software engineering body of knowledge and software engineering fundamental principles. Dr Suryn is also the principal researcher and the director of GELOG : IQUAL, the Software Quality Engineering Research Group at école de technologie supérieure. From October 2003 Dr. Suryn holds the position of the International Secretary of ISO/IEC SC7 – System and Software Engineering.     

Software quality trade-offs: A systematic map

BackgroundSoftware quality is complex with over investment, under investment and the interplay between aspects often being overlooked as many researchers aim to advance individual aspects of software quality.AimThis paper aims to provide a consolidated overview the literature that addresses trade-offs between aspects of software product quality.MethodA systematic literature map is employed to provide an overview of software quality trade-off literature in general. Specific analysis is also done of empirical literature addressing the topic.ResultsThe results show a wide range of solution proposals being considered. However, there is insufficient empirical evidence to adequately evaluate and compare these proposals. Further a very large vocabulary has been found to describe software quality.ConclusionGreater empirical research is required to sufficiently evaluate and compare the wide range of solution proposals. This will allow researchers to focus on the proposals showing greater signs of success and better support industrial practitioners.     

A formal analysis of the subsume relation between software testadequacy criteria

Software test adequacy criteria are rules to determine whether a software system has been adequately tested. A central question in the study of test adequacy criteria is how they relate to fault detecting ability. We identify two idealized software testing scenarios. In the first scenario, which we call prior testing scenario, software testers are provided with an adequacy criterion in addition to the software under test. The knowledge of the adequacy criterion is used to generate test cases. In the second scenario, which we call posterior testing scenario, software testers are not provided with the knowledge of adequacy criterion. The criterion is only used to decide when to stop the generation of test cases. In 1993, Frankl and Weyuker proved that the subsume relation between software test adequacy criteria does not guarantee better fault detecting ability in the prior testing scenario. We investigate the posterior testing scenario and prove that in this scenario the subsume relation does guarantee a better fault detecting ability. Two measures of fault detecting ability will be used, the probability of detecting faults and the expected number of exposed errors     

A systematic approach for performance assessment using process mining

Software performance engineering is a mature field that offers methods to assess system performance. Process mining is a promising research field applied to gain insight on system processes. The interplay of these two fields opens promising applications in the industry. In this work, we report our experience applying a methodology, based on process mining techniques, for the performance assessment of a commercial data-intensive software application. The methodology has successfully assessed the scalability of future versions of this system. Moreover, it has identified bottlenecks components and replication needs for fulfilling business rules. The system, an integrated port operations management system, has been developed by Prodevelop, a medium-sized software enterprise with high expertise in geospatial technologies. The performance assessment has been carried out by a team composed by practitioners and researchers. Finally, the paper offers a deep discussion on the lessons learned during the experience, that will be useful for practitioners to adopt the methodology and for researcher to find new routes.     

Software quality management: an Australian experience

Since the release ofAustralian Standard 3563 — Software Quality Management System in late 1988 there has been great interest in the application of formal quality management systems to the development of software. Consequently, the standard has been rewritten to conform to the International Standard (ISO) 9001 for the highest level of quality management and to incorporate some lessons learned from the first edition (which was in a slightly different format). This article discusses some experiences of both implementors of the standard and of assessors of the systems. There is also a summary of the clauses of the standard and discussion of some changes made between the two editions of the documents. Published with the 1991 edition is another document that is an implementation guide to help the smaller companies in the industry to implement their quality management systems. Some contents of this guide are also discussed.     

PRODUCTION SYSTEMS FOR PROCESS CONTROL: ADVANCES AMD EXPERIENCES

Abstract

Production systems are still regarded as one of the most important paradigms for solving complex tasks. However, in embedded control systems production systems have not yet been applied intensively. We identify open issues for production systems in process control by exemplary presenting two industrial fielded expert systems. We describe how these expert systems have been applied to process control, discuss lessons learned, and show how practical experience has influenced our own research activities. Finally we identify future research directions of general importance.