Evaluating performance in the development of software-intensive products

ContextOrganizational performance measurements in software product development have received a lot of attention in the literature. Still, there is a general discontent regarding the way performance is evaluated in practice, with few studies really focusing on why this is the case. In this paper research focusing on the context of developing software-intensive products in large established multi-national organizations is reported on.ObjectiveThe purpose of this research is to investigate performance measurement practices related to software product development activities. More specifically, focus is on exploring how managers engaged in software product development activities perceive and evaluate performance in large organizations from a managerial perspective.MethodThe research approach pursued in this research consist of exploratory multiple case studies. Data is collected mainly through 54 interviews in five case studies in large international organizations developing software-intensive products in Sweden. Focused group interviews with senior managers from eight companies have also been used in the data collection.ResultsThe results of this research indicate that managers within software product development in general are dissatisfied with their current way of evaluating performance. Performance measurements and the perception of performance are today focused on cost, time, and quality, i.e. what is easily measurable and not necessarily what is important. The dimensions of value creation and learning are missing. Moreover, measurements tend to be result oriented, rather than process oriented, making it difficult to integrate these measurements in the management practices.ConclusionManagers that are dissatisfied with their performance measurement system and want to improve the current situation should not start by focusing on the current measurements directly; instead they should focus on how the organization perceives performance and how important performance criteria are being developed. By developing relevant performance criteria the first step in developing an effective performance measurement system is made. Moreover, it is concluded that manager’s perception of performance is affected by the currently used measurements, hence limiting the scope of the performance criteria. Thus, a change in the way managers perceive performance is necessary before there can be any changes in the way performance is evaluated.     

Challenges and industry practices for managing software variability in small and medium sized enterprises

Software variability is an ability to change (configure, customize, extend) software artefacts (e.g. code, product, domain requirements, models, design, documentation, test cases) for a specific context. Optimized variability management can lead a software company to 1) shorter development lead time, 2) improved customer and improved user satisfaction, 3) reduced complexity of product management (more variability, same $) and 4) reduced costs (same variability, less $). However, it is not easy for software companies, especially small and medium size of enterprises to deal with variability. In this paper we present variability challenges and used practices collected from five SMEs. Our study indicates that increased product complexity can lead growing SMEs to the time-consuming decision-making. Many of the analyzed medium size of companies also expect improved tool support to help them to boost their productivity when managing increasingly complex products and increasing amount of variants In fact, in many of the analysed SMEs, a high level of automation in design, release management and testing are or become a key factor for market success By introducing the challenges and used practices related to variability the paper deepens understanding of this highly relevant but relatively under-researched phenomenon and contributes to the literature on software product line engineering.     

Best practice fusion of CMMI-DEV v1.2 (PP,PMC, SAM) and PMBOK 2008

ContextThe establishment of effective and efficient project management practices still remains a challenge to software organizations. In striving to address these needs, “best practice” models, such as, CMMI or PMBOK, are being developed to assist organizations interested in improving project management. And, although, those models share overlapping content, there are still differences and, therefore, each of the models offers different advantages.ObjectiveThis paper proposes a set of unified project management best practices by integrating and harmonizing on a high-level perspective PMBOK (4th ed.) processes and CMMI-DEV v1.2 specific practices of the basic project management process areas PP, PMC and SAM.MethodBased on the analysis of both models, a unified set of best practices has been defined by a group of researchers with theoretical and practical expertise on the CMMI framework and software process improvement as well as project management and the PMBOK. The proposed set has been revised by different researchers from different institutions in several review rounds until consensus was achieved.ResultsAs a result, a set of unified best practices is defined and explicitly mapped to the correspondent PMBOK processes and CMMI specific practices of the current versions of both models.ConclusionWe can conclude that an integration and harmonization of both models is possible and may help to implement and assess project management processes more effectively and efficiently, optimizing software process improvement investments.     

Software process improvement in small and medium software enterprises: a systematic review

Small and medium enterprises are a very important cog in the gears of the world economy. The software industry in most countries is composed of an industrial scheme that is made up mainly of small and medium software enterprises—SMEs. To strengthen these types of organizations, efficient Software Engineering practices are needed—practices which have been adapted to their size and type of business. Over the last two decades, the Software Engineering community has expressed special interest in software process improvement (SPI) in an effort to increase software product quality, as well as the productivity of software development. However, there is a widespread tendency to make a point of stressing that the success of SPI is only possible for large companies. In this article, a systematic review of published case studies on the SPI efforts carried out in SMEs is presented. Its objective is to analyse the existing approaches towards SPI which focus on SMEs and which report a case study carried out in industry. A further objective is that of discussing the significant issues related to this area of knowledge, and to provide an up-to-date state of the art, from which innovative research activities can be thought of and planned.

An industrial case study on variability handling in large enterprise software systems

ContextEnterprise software systems (e.g., enterprise resource planning software) are often deployed in different contexts (e.g., different organizations or different business units or branches of one organization). However, even though organizations, business units or branches have the same or similar business goals, they may differ in how they achieve these goals. Thus, many enterprise software systems are subject to variability and adapted depending on the context in which they are used.ObjectiveOur goal is to provide a snapshot of variability in large scale enterprise software systems. We aim at understanding the types of variability that occur in large industrial enterprise software systems. Furthermore, we aim at identifying how variability is handled in such systems.MethodWe performed an exploratory case study in two large software organizations, involving two large enterprise software systems. Data were collected through interviews and document analysis. Data were analyzed following a grounded theory approach.ResultsWe identified seven types of variability (e.g., functionality, infrastructure) and eight mechanisms to handle variability (e.g., add-ons, code switches).ConclusionsWe provide generic types for classifying variability in enterprise software systems, and reusable mechanisms for handling such variability. Some variability types and handling mechanisms for enterprise software systems found in the real world extend existing concepts and theories. Others confirm findings from previous research literature on variability in software in general and are therefore not specific to enterprise software systems. Our findings also offer a theoretical foundation for describing variability handling in practice. Future work needs to provide more evaluations of the theoretical foundations, and refine variability handling mechanisms into more detailed practices.     

Software SMEs’ unofficial readiness for CMMI<Superscript>®</Superscript>-based software process improvement

The goal of software process improvement (SPI) is to improve software processes and produce high-quality software, but the results of SPI efforts in small- and medium-sized enterprises (SMEs) that develop software have been unsatisfactory. The objective of this study is to support the prolific and successful CMMI-based implementation of SPI in SMEs by presenting the facts related to the unofficial adoption of CMMI level 2 process area-specific practices by software SMEs. Two questionnaire surveys were performed, and 42 questionnaires were selected for data analysis. The questionnaires were filled out by experts from 42 non-CMMI-certified software SMEs based in Malaysia and Pakistan. In the case of each process area of CMMI level 2, the respondents were asked to choose from three categories, namely ‘below 50 %,’ ‘50–75 %,’ and ‘above 75 %’. The percentages indicated the extent to which process area-specific practices are routinely followed in the respondents’ respective organizations. To deal with differing standards for defining SMEs, the notion of the common range standard has been introduced. The results of the study show that a large segment of software development SMEs informally follows the specific practices of CMMI level 2 process areas and thus has true potential for rapid and effective CMMI-based SPI. The results further indicate that, in the case of four process areas of CMMI level 2, there are statistically significant differences between the readiness of small and medium software enterprises to adopt the specific practices of those process areas, and between trends on their part to do so unofficially. The findings, manifesting various degrees of unofficial readiness for CMMI-based SPI among SMEs, can be used to define criteria for the selection of SMEs that would be included in SPI initiatives funded by relevant authorities. In the interests of developing fruitful CMMI-based SPI and to enhance the success rate of CMMI-based SPI initiatives, the study suggests that ‘ready’ or ‘potential’ SMEs should be given priority for SPI initiatives.     

An investigation of ‘build vs. buy’ decision for software acquisition by small to medium enterprises

ContextThe prevalence of computing and communication technologies, combined with the availability of sophisticated and highly specialised software packages from software vendors has made package acquisition a viable option for many organisations. While some research has addressed the factors that influence the selection of the software acquisition method in large organisations, little is known about the factors affecting SMEs.ObjectiveTo provide an understanding of factors that affect the decision process of software acquisition for SMEs. It is expected that results from this study: (i) will assist the SME decision process for software acquisition, and (ii) will assist policy makers in terms of developing appropriate guidelines for SME software acquisition.MethodA positivist research perspective has been adopted involving semi-structured interviews in eight SMEs in Thailand with the interviewees assigning to each of the potential factors.ResultsThe study found that the following factors affect both SMEs and large organisations: requirements fit, cost, scale and complexity, commoditization/flexibility, time, in-house experts, support structure, and operational factors. Factors mainly applying to large organisations were strategic role of the software, intellectual property concerns, and risk, Factors particularly relevant to SMEs (ubiquitous systems, availability of free download, and customizable to specific government/tax regulations).ConclusionThe results suggest that: (i) when deciding on their software acquisition method, SMEs are generally less likely to pursue a long-term vision compared with larger organisations, possibly because SMEs mainly serve their local markets; and (ii) contrary to the large organisations, the role that the IT plays in SMEs may not be as vital to the SMEs’ core business processes, to their supply chains, and/or to the management of their customer relationship. Furthermore, neither the level of technological intensity nor size of the SME appears to affect the ranks given by the interviewees for the various factors.     

Understanding agile software development practices using shared mental models theory

ContextAgile software development is an alternative software development methodology that originated from practice to encourage collaboration between developers and users, to leverage rapid development cycles, and to respond to changes in a dynamic environment. Although agile practices are widely used in organizations, academics call for more theoretical research to understand the value of agile software development methodologies.ObjectiveThis study uses shared mental models theory as a lens to examine practices from agile software methodologies to understand how agile practices enable software development teams to work together to complete tasks and work together effectively as a team.MethodA conceptual analysis of specific agile practices was conducted using the lens of shared mental models theory. Three agile practices from Xtreme Programming and Scrum are examined in detail, system metaphor, stand-up meeting, and on-site customer, using shared mental models theory.ResultsExamining agile practices using shared mental models theory elucidates how agile practices improve collaboration during the software development process. The results explain how agile practices contribute toward a shared understanding and enhanced collaboration within the software development team.ConclusionsThis conceptual analysis demonstrates the value of agile practices in developing shared mental models (i.e. shared understanding) among developers and customers in software development teams. Some agile practices are useful in developing a shared understanding about the tasks to be completed, while other agile practices create shared mental models about team processes and team interactions. To elicit the desired outcomes of agile software development methods, software development teams should consider whether or not agile practices are used in a manner that enhances the team’s shared understanding. Using three specific agile practices as examples, this research demonstrates how theory, such as shared mental models theory, can enhance our understanding regarding how agile practices are useful in enhancing collaboration in the workplace.     

Supporting distributed product configuration by integrating heterogeneous variability modeling approaches

ContextIn industrial settings products are developed by more than one organization. Software vendors and suppliers commonly typically maintain their own product lines, which contribute to a larger (multi) product line or software ecosystem. It is unrealistic to assume that the participating organizations will agree on using a specific variability modeling technique—they will rather use different approaches and tools to manage the variability of their systems.ObjectiveWe aim to support product configuration in software ecosystems based on several variability models with different semantics that have been created using different notations.MethodWe present an integrative approach that provides a unified perspective to users configuring products in multi product line environments, regardless of the different modeling methods and tools used internally. We also present a technical infrastructure and a prototype implementation based on web services.ResultsWe show the feasibility of the approach and its implementation by using it with the three most widespread types of variability modeling approaches in the product line community, i.e., feature-based, OVM-style, and decision-oriented modeling. To demonstrate the feasibility and flexibility of our approach, we present an example derived from industrial experience in enterprise resource planning. We further applied the approach to support the configuration of privacy settings in the Android ecosystem based on multiple variability models. We also evaluated the performance of different model enactment strategies used in our approach.ConclusionsTools and techniques allowing stakeholders to handle variability in a uniform manner can considerably foster the initiation and growth of software ecosystems from the perspective of software reuse and configuration.     

Issue-based variability management

ContextVariability management is a key activity in software product line engineering. This paper focuses on managing rationale information during the decision-making activities that arise during variability management. By decision-making we refer to systematic problem solving by considering and evaluating various alternatives. Rationale management is a branch of science that enables decision-making based on the argumentation of stakeholders while capturing the reasons and justifications behind these decisions.ObjectiveDecision-making should be supported to identify variability in domain engineering and to resolve variation points in application engineering. We capture the rationale behind variability management decisions. The captured rationale information is useful to evaluate future changes of variability models as well as to handle future instantiations of variation points. We claim that maintaining rationale will enhance the longevity of variability models. Furthermore, decisions should be performed using a formal communication between domain engineering and application engineering.MethodWe initiate the novel area of issue-based variability management (IVM) by extending variability management with rationale management. The key contributions of this paper are: (i) an issue-based variability management methodology (IVMM), which combines questions, options and criteria (QOC) and a specific variability approach; (ii) a meta-model for IVMM and a process for variability management and (iii) a tool for the methodology, which was developed by extending an open source rationale management tool.ResultsRationale approaches (e.g. questions, options and criteria) guide distributed stakeholders when selecting choices for instantiating variation points. Similarly, rationale approaches also aid the elicitation of variability and the evaluation of changes. The rationale captured within the decision-making process can be reused to perform future decisions on variability.ConclusionIVMM was evaluated comparatively based on an experimental survey, which provided evidence that IVMM is more effective than a variability modeling approach that does not use issues.     

Impact of physical ambiance on communication,collaboration and coordination in agile software development: An empirical evaluation

ContextCommunication, collaboration and coordination are key enablers of software development and even more so in agile methods. The physical environment of the workspace plays a significant role in effective communication, collaboration, and coordination among people while developing software.ObjectiveIn this paper, we have studied and further evaluated empirically the effect of different constituents of physical environment on communication, coordination, and collaboration, respectively. The study aims to provide a guideline for prospective agile software developers.MethodA survey was conducted among software developers at a software development organization. To collect data, a survey was carried out along with observations, and interviews.ResultsIt has been found that half cubicles are ‘very effective’ for the frequency of communication. Further, half cubicles were discovered ‘effective’ but not ‘very effective’ for the quality/effectiveness of communication. It is found that half-height cubicles and status boards are ‘very effective’ for the coordination among team members according to the survey. Communal/discussion space is found to be ‘effective’ but not ‘very effective’ for coordination among team members. Our analysis also reveals that half-height glass barriers are ‘very effective’ during the individuals problem-solving activities while working together as a team. Infact, such a physically open environment appears to improve communication, coordination, and collaboration.ConclusionAccording to this study, an open working environment with only half-height glass barriers and communal space plays a major role in communication among team members. The presence of status boards significantly help in reducing unnecessary communication by providing the required information to individuals and therefore, in turn reduce distractions a team member may confront in their absence. As communication plays a significant role in improving coordination and collaboration, it is not surprising to find the effect of open working environment and status boards in improving coordination and collaboration. An open working environment increases the awareness among software developers e.g. who is doing what, what is on the agenda, what is taking place, etc. That in turn, improves coordination among them. A communal/discussion space helps in collaboration immensely.     

The DOPLER meta-tool for decision-oriented variability modeling: a multiple case study

The variability of a product line is typically defined in models. However, many existing variability modeling approaches are rigid and don’t allow sufficient domain-specific adaptations. We have thus been developing a flexible and extensible approach for defining product line variability models. Its main purposes are to guide stakeholders through product derivation and to automatically generate product configurations. Our approach is supported by the DOPLER (Decision-Oriented Product Line Engineering for effective Reuse) meta-tool that allows modelers to specify the types of reusable assets, their attributes, and dependencies for their specific system and context. The aim of this paper is to investigate the suitability of our approach for different domains. More specifically, we explored two research questions regarding the implementation of variability and the utility of DOPLER for variability modeling in different domains. We conducted a multiple case study consisting of four cases in the domains of industrial automation systems and business software. In each of these case studies we analyzed variability implementation techniques. Experts from our industry partners then developed domain-specific meta-models, tool extensions, and variability models for their product lines using DOPLER. The four cases demonstrate the flexibility of the DOPLER approach and the extensibility and adaptability of the supporting meta tool.     

Acquiring and sharing tacit knowledge in software development teams: An empirical study

ContextSharing expert knowledge is a key process in developing software products. Since expert knowledge is mostly tacit, the acquisition and sharing of tacit knowledge along with the development of a transactive memory system (TMS) are significant factors in effective software teams.ObjectiveWe seek to enhance our understanding human factors in the software development process and provide support for the agile approach, particularly in its advocacy of social interaction, by answering two questions: How do software development teams acquire and share tacit knowledge? What roles do tacit knowledge and transactive memory play in successful team performance?MethodA theoretical model describing the process for acquiring and sharing tacit knowledge and development of a TMS through social interaction is presented and a second predictive model addresses the two research questions above. The elements of the predictive model and other demographic variables were incorporated into a larger online survey for software development teams, completed by 46 software SMEs, consisting of 181 individual team members.ResultsOur results show that team tacit knowledge is acquired and shared directly through good quality social interactions and through the development of a TMS with quality of social interaction playing a greater role than transactive memory. Both TMS and team tacit knowledge predict effectiveness but not efficiency in software teams.ConclusionIt is concluded that TMS and team tacit knowledge can differentiate between low- and high-performing teams in terms of effectiveness, where more effective teams have a competitive advantage in developing new products and bringing them to market. As face-to-face social interaction is key, collocated, functionally rich, domain expert teams are advocated rather than distributed teams, though arguably the team manager may be in a separate geographic location provided that there is frequent communication and effective use of issue tracking tools as in agile teams.     

Software Process Improvement barriers: A cross-cultural comparison

ContextSoftware Process Improvement initiatives have been around for many years with the growing globalisation of software development is making them increasingly important.ObjectiveThe objective of this exploratory research is to gain an in-depth understanding of barriers that can undermine SPI, in the context of Global Software Development, from the perspective of software development practitioners; this will enable SPI managers to better manage SPI initiatives. We intend to discover if the barriers to SPI initiatives in a developed country are different to those in a developing country.MethodIn an empirical study, Vietnamese software practitioners’ experiences of SPI barriers are compared with barriers identified by Australian practitioners. Face-to-face questionnaire-based survey sessions with 23 Vietnamese SPI practitioners were conducted. Our survey included barriers to SPI improvement initiatives identified in previous research. We asked the participants to rank each SPI barrier on a three-point scale (high, medium, low) to determine the importance of each barrier. We then compare our results, with results (identified in previous work), from 34 Australian software development practitioners.ResultsWe identify (1) lack of project management, (2) lack of resources, (3) lack of sponsorship, (4) inexperienced staff/lack of knowledge, and (5) lack of SPI awareness as ‘high’ value SPI barriers in Vietnam. The results also reveal similarities and differences between the experiences of Australian and Vietnamese practitioners regarding the importance of the SPI barriers identified. While the Australian practitioners were also concerned with (1) lack of SPI awareness, they were even more concerned with (2) organisational politics, and (3) lack of support.ConclusionsPractitioners identify SPI barriers based on previous SPI implementation experience. Their role(s) in their different organisations have helped them to understand the importance of that barrier. Vietnamese software practitioners cited more SPI barriers than their counterparts in Australia. The Vietnamese SPI barriers relate to project management, resources, and sponsorship while the Australian barriers are concerned with organisational politics and lack of support.     

PEM: The small company-dedicated software process quality evaluation method combining CMMISM and ISO/IEC 14598

Many small software organizations have recognized the need to improve their software product. Evaluating the software product alone seems insufficient since it is known that its quality is largely dependant on the process that is used to create it. Thus, small organizations are asking for evaluation of their software processes and products. The ISO/IEC 14598-5 standard is already used as a methodology basis for evaluating software products. This article explores how it can be combined with the CMMI to produce a methodology that can be tailored for process evaluation in order to improve their software processes. ^SM: CMMI is a service mark of Carnegie-Mellon University. Sylvie Trudel has over 20 years of experience in software. She worked for more than 10 years in development and implementation of management information systems and embedded real-time systems. Since 1996, she works as a process improvement specialist, implementing best practices into organizations processes from CMM and CMMI models. She performed several CMM and CMMI assessments and participated in many other CMM assessments such as CBA IPI, SCE, and other proprietary methods. She obtained a bachelors degree in computer science in 1986 from Laval University in Québec City and a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal. Sylvie is currently working as a software engineering advisor at the Centre de Recherche Informatique de Montréal (CRIM). Jean-Marc Lavoie has been working in software development for over 10 years. He performed and published a comparative study between the guide to the SWEBOK and the CMMI in 2003. Jean-Marc obtained a bachelor degree in Electrical Engineering. He is pursuing a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal while working as a software architect at Trisotech. Marie-Claude Pare has been working in software development for 7 years. Marie-Claude obtained a bachelor degree in Software Engineering from école Polytechnique in Montréal. She is pursuing a Masters degree in Software Engineering at école de Technologie Supérieure (éTS) in Montréal while working as a software engineer at Motorola GSG Canada. Dr Witold Suryn is a Professor at the école de technologie supérieure, Montreal, Canada (engineering school of the Université du Québec network of institutions) where he teaches graduate and undergraduate software engineering courses and conducts research in the domain of software quality engineering, software engineering body of knowledge and software engineering fundamental principles. Dr Suryn is also the principal researcher and the director of GELOG : IQUAL, the Software Quality Engineering Research Group at école de technologie supérieure. From October 2003 Dr. Suryn holds the position of the International Secretary of ISO/IEC SC7 – System and Software Engineering.     

Problems and their mitigation in system and software architecting

ContextToday, software and embedded systems act as enablers for developing new functionality in traditional industries such as the automotive, process automation, and manufacturing automation domains. This differs from 25–30 years ago when these systems where based on electronics and electro-mechanical solutions. The architecture of the embedded system and of the software is important to ensure the qualities of these applications. However, the effort of designing and evolving the architecture is in practice often neglected during system development, whilst development efforts are centered on implementing new functionality.ObjectiveWe present problems and success factors that are central to the architectural development of software intensive systems in the domain of automotive and automation products as judged by practitioners.MethodThe method consisted of three steps. First, we used semi-structured interviews to collect data in an exploratory manner. As a second step, a survey based on problems extracted from the interview data was used to investigate the occurrence of these problems at a wider range of organizations. In order to identify and suggest how to mitigate the problems that were considered important, we finally performed root cause analysis workshops, and from these a number of success factors were elicited.ResultsA total of 21 problems have been identified based on the interview data, and these are related to the technical, organizational, project, and agreement processes. Based on the survey results, the following four problems were selected for a root cause analysis: (1) there is a lack of process for architecture development, (2) there is a lack of method or model to evaluate the business value when choosing the architecture, (3) there is a lack of clear long-term architectural strategy, and (4) processes and methods are less valued than knowledge and competence of individuals.ConclusionIn conclusion, the following identified success factors are crucial components to be successful in developing software intensive systems: (1) define an architectural strategy, (2) implement a process for architectural work, (3) ensure authority for architects, (4) clarify the business impact of the architecture, and (5) optimize on the project portfolio level instead of optimizing each project.     

How are software defects found? The role of implicit defect detection,individual responsibility,documents, and knowledge

ContextPrior research has focused heavily on explicit defect detection, such as formal testing and reviews. However, in reality, humans find software defects in various activities. Implicit defect detection activities, such as preparing a product demonstration or updating a user manual, are not designed for defect detection, yet through such activities defects are discovered. In addition, the type of documentation, and knowledge used, in defect detection is diverse.ObjectiveTo understand how defect detection is affected by the perspectives of responsibility, activity, knowledge, and document use. To provide illustrative numbers concerning the multidimensionality of defect detection in an industrial context.MethodThe data were collected with a survey on four software development organizations in three different companies. We designed the survey based on our prior extensive work with these companies.ResultsWe found that among our subjects (n = 105), implicit defect detection made a higher contribution than explicit defect detection in terms of found defects, 62% vs. 38%. We show that defect detection was performed by subjects in various roles supporting the earlier reports of testing being a cross-cutting activity in software development organizations. We found a low use of test cases (18%), but a high use of other documents in software defect detection, and furthermore, we found that personal knowledge was applied as an oracle in defect detection much more often than documented oracles. Finally, we recognize that contextual factors largely affect the transferability of our results, and we provide elaborate discussion about the most important contextual factors. Furthermore, we must be cautious as the results were obtained with a survey, and come from a small number of organizations.ConclusionsIn this paper, we show the large impact of implicit defect detection activities in four case organizations. Implicit defect detection has a large contribution to defect detection in practice, and can be viewed as an extremely low-cost way of detecting defects. Thus, harnessing and supporting it better may increase quality without increasing costs. For example, if an employee can update the user manual, and simultaneously detect defects from the software, then the defect detection part of this activity can be seen as cost-free. Additionally, further research is needed on how diverse types of useful documentation and knowledge can be utilized in defect detection.