Data Mining of Software Development Databases

Software quality models can predict which modules will have high risk, enabling developers to target enhancement activities to the most problematic modules. However, many find collection of the underlying software product and process metrics a daunting task.Many software development organizations routinely use very large databases for project management, configuration management, and problem reporting which record data on events during development. These large databases can be an unintrusive source of data for software quality modeling. However, multiplied by many releases of a legacy system or a broad product line, the amount of data can overwhelm manual analysis. The field of data mining is developing ways to find valuable bits of information in very large databases. This aptly describes our software quality modeling situation.This paper presents a case study that applied data mining techniques to software quality modeling of a very large legacy telecommunications software system's configuration management and problem reporting databases. The case study illustrates how useful models can be built and applied without interfering with development.     

Accuracy of software quality models over multiple releases

Many evolving mission‐critical systems must have high software reliability. However, it is often difficult to identify fault‐prone modules early enough in a development cycle to guide software enhancement efforts effectively and efficiently. Software quality models can yield timely predictions of membership in the fault‐prone class on a module‐by‐module basis, enabling one to target enhancement techniques. However, it is an open empirical question, “Can a software quality model remain useful over several releases?” Most prior software quality studies have examined only one release of a system, evaluating the model with modules from the same release. We conducted a case study of a large legacy telecommunications system where measurements on one software release were used to build models, and three subsequent releases of the same system were used to evaluate model accuracy. This is a realistic assessment of model accuracy, closely simulating actual use of a software quality model. A module was considered fault‐prone if any of its faults were discovered by customers. These faults are extremely expensive due to consequent loss of service and emergency repair efforts. We found that the model maintained useful accuracy over several releases. These findings are initial empirical evidence that software quality models can remain useful as a system is maintained by a stable software development process.     

Accuracy of software quality models over multiple releases

Many evolving mission‐critical systems must have high software reliability. However, it is often difficult to identify fault‐prone modules early enough in a development cycle to guide software enhancement efforts effectively and efficiently. Software quality models can yield timely predictions of membership in the fault‐prone class on a module‐by‐module basis, enabling one to target enhancement techniques. However, it is an open empirical question, “Can a software quality model remain useful over several releases?” Most prior software quality studies have examined only one release of a system, evaluating the model with modules from the same release. We conducted a case study of a large legacy telecommunications system where measurements on one software release were used to build models, and three subsequent releases of the same system were used to evaluate model accuracy. This is a realistic assessment of model accuracy, closely simulating actual use of a software quality model. A module was considered fault‐prone if any of its faults were discovered by customers. These faults are extremely expensive due to consequent loss of service and emergency repair efforts. We found that the model maintained useful accuracy over several releases. These findings are initial empirical evidence that software quality models can remain useful as a system is maintained by a stable software development process. This revised version was published online in June 2006 with corrections to the Cover Date.     

Analyzing software measurement data with clustering techniques

For software quality estimation, software development practitioners typically construct quality-classification or fault prediction models using software metrics and fault data from a previous system release or a similar software project. Engineers then use these models to predict the fault proneness of software modules in development. Software quality estimation using supervised-learning approaches is difficult without software fault measurement data from similar projects or earlier system releases. Cluster analysis with expert input is a viable unsupervised-learning solution for predicting software modules' fault proneness and potential noisy modules. Data analysts and software engineering experts can collaborate more closely to construct and collect more informative software metrics.     

Analogy-Based Practical Classification Rules for Software Quality Estimation

Software metrics-based quality estimation models can be effective tools for identifying which modules are likely to be fault-prone or not fault-prone. The use of such models prior to system deployment can considerably reduce the likelihood of faults discovered during operations, hence improving system reliability. A software quality classification model is calibrated using metrics from a past release or similar project, and is then applied to modules currently under development. Subsequently, a timely prediction of which modules are likely to have faults can be obtained. However, software quality classification models used in practice may not provide a useful balance between the two misclassification rates, especially when there are very few faulty modules in the system being modeled.This paper presents, in the context of case-based reasoning, two practical classification rules that allow appropriate emphasis on each type of misclassification as per the project requirements. The suggested techniques are especially useful for high-assurance systems where faulty modules are rare. The proposed generalized classification methods emphasize on the costs of misclassifications, and the unbalanced distribution of the faulty program modules. We illustrate the proposed techniques with a case study that consists of software measurements and fault data collected over multiple releases of a large-scale legacy telecommunication system. In addition to investigating the two classification methods, a brief relative comparison of the techniques is also presented. It is indicated that the level of classification accuracy and model-robustness observed for the case study would be beneficial in achieving high software reliability of its subsequent system releases. Similar observations are made from our empirical studies with other case studies.     

Ordering Fault-Prone Software Modules

Software developers apply various techniques early in development to improve software reliability, such as extra reviews, additional testing, and strategic assignment of personnel. Due to limited resources and time, it is often not practical to enhance the reliability of all modules. Our goal is to target reliability enhancement activities to those modules that would otherwise have problems later. Prior research has shown that a software quality model based on software product and process metrics can predict which modules are likely to have faults.A module-order model is a quantitative software quality model that is used to predict the rank-order of modules according to a quality factor, such as the number of faults. The contribution of this paper is definition of module-order models and a method for their evaluation and use. Two empirical case studies of full-scale industrial software systems provide empirical evidence of the usefulness of module-order models for targeting reliability enhancement.     

基于结构的软件可靠性建模研究进展

软件体系结构作为提高软件系统质量、支持复杂软件开发和复用的重要手段,已经成为软件工程的一个重要研究领域,软件结构的研究对软件可靠性度量也带来的新的问题。分析和总结近年来基于软件体系结构的可靠性建模方法,并对其今后若干研究方向进行了探讨。     

Reuse-based software production technology

Software reuse is viewed as a key technology to improve software product quality and productivity. This paper discusses a series of technologies related with software reuse and software component technology: component model, which describes component's essential characteristics; component acquisition technology, of which domain engineering is the main approach; component management technology, of which component library is the kernel; application integration and composition technology, of which application engineering is the main approach; software evolution technology, of which software reengineering is the main approach, etc. This paper introduces the software development environment: JadeBird Software Production Line System, which effectively integrates the above-mentioned technologies.     

Balancing Misclassification Rates in Classification-Tree Models of Software Quality

Software product and process metrics can be useful predictorsof which modules are likely to have faults during operations.Developers and managers can use such predictions by softwarequality models to focus enhancement efforts before release.However, in practice, software quality modeling methods in theliterature may not produce a useful balance between the two kindsof misclassification rates, especially when there are few faultymodules.This paper presents a practical classificationrule in the context of classification tree models that allowsappropriate emphasis on each type of misclassification accordingto the needs of the project. This is especially important whenthe faulty modules are rare.An industrial case study using classification trees, illustrates the tradeoffs.The trees were built using the TREEDISC algorithm whichis a refinement of the CHAID algorithm. We examinedtwo releases of a very large telecommunications system, and builtmodels suited to two points in the development life cycle: theend of coding and the end of beta testing. Both trees had onlyfive significant predictors, out of 28 and 42 candidates, respectively.We interpreted the structure of the classification trees, andwe found the models had useful accuracy.     

基于软件产品线的需求分析研究

软件产品线方法是一种面向特定领域的、大规模、大粒度的软件复用技术.在软件产品线的开发过程中,产品线需求分析是软件产品线开发的关键活动之一,软件产品线需求分析奠定了产品线构架的基础.通过分析软件产品线开发过程和软件产品线需求分析的特点,阐述了软件产品线需求分析方法以及软件产品线需求分析的实践风险.以领域分析和建模为切入点,对软件产品线的领域分析、需求建模和用例建模等关键方法和技术进行了重点的研究.     

B/S结构ERP管理软件的测试方法研究

软件测试是软件开发过程中非常重要的一个环节,直接关系着软件的质量。文中阐述了软件各个开发阶段需要进行的测试,并以B/S结构ERP管理软件为代表,着重介绍了系统实现阶段软件测试的方法,同时简要描述了软件维护阶段测试的过程和方法,最后总结了软件测试可能的发展方向。     

A model for software reuse

Software reusability is widely believed to be a key to help overcome the ongoing software crisis by improving software productivity and quality. New computer technology and the demands of an advancing society require new and more complicated software. It is unrealistic to expect that every software system can be developed solely reusing existing software; however, software reusability has proven to be a way of enhancing software productivity and quality in different organizations. It is also believed that reuse of code segments alone does not significantly reduce software development effort.By creating reuse support information while software is being developed (when the software is best known to software developers), the reuse effort for both software development and maintenance can be potentially reduced.In this paper, five types of RSI are presented: Semantic Web, Horizontal Web, Vertical Web, Syntactic Web, and Alternative Web. We collectively call these five webs a Quintet Web. The Semantic Web enables software developers to locate a software segment. The Horizontal Web provides a means to reuse a block of soflware along with other types of software from other phases of a development process. The inter-phase integration of software can be performed through this Web. The Vertical Web identifies the vertical relationship of a software block to its operational environment. The Syntactic Web locates all statements in which a variable is used. The Alternative Web provides alternative software blocks that perform an identical operation.A prototype of the Quintet Web is presented.     

Cost-Benefit Analysis of Software Quality Models

Software reliability is increasingly important in today's marketplace. When traditional software development processes fail to deliver the level of reliability demanded by customers, radical changes in software development processes may be needed. Business process reengineering (BPR) is the popular term for comprehensive redesign of business processes. This paper focuses on the business processes that produce commercial software, and illustrates the central role that models have in implementation of BPR. Software metrics and software-quality modeling technology enable reengineering of software development processes, moving from a static process model to a dynamic one that adapts to the expected quality of each module. We present a method for cost-benefit analysis of BPR of software development processes as a function of model accuracy. The paper defines costs, benefits, profit, and return on investment from both short-term and long-term perspectives. The long-term perspective explicitly accounts for software maintenance efforts. A case study of a very large legacy telecommunications system illustrates the method. The dependent variable of the software-quality model was whether a module will have faults discovered by customers. The independent variables were software product and process metrics. In an example, the costs and benefits of using the model are compared to using random selection of modules for reliability enhancement. Such a cost-benefit analysis clarifies the implications of following model recommendations.     

An empirical study for software change prediction using imbalanced data

Software change prediction is crucial in order to efficiently plan resource allocation during testing and maintenance phases of a software. Moreover, correct identification of change-prone classes in the early phases of software development life cycle helps in developing cost-effective, good quality and maintainable software. An effective software change prediction model should equally recognize change-prone and not change-prone classes with high accuracy. However, this is not the case as software practitioners often have to deal with imbalanced data sets where instances of one type of class is much higher than the other type. In such a scenario, the minority classes are not predicted with much accuracy leading to strategic losses. This study evaluates a number of techniques for handling imbalanced data sets using various data sampling methods and MetaCost learners on six open-source data sets. The results of the study advocate the use of resample with replacement sampling method for effective imbalanced learning.     

Analogy-based domain analysis approach to software reuse

Domain analysis is an expansion of conventional requirements analysis. Domain analysis can support effective software reuse. However, domain analysis is time consuming and is limited to a particular application area. Analogical approaches to software reuse, on the other hand, often occur across domains. Analogical problem solving is a process of transferring knowledge from a well-understood base domain to a new target problem area. Analogy can facilitate software reuse for poorly understood problems or new application areas. Analogy shares similar concepts with reuse and some analogy theories have been applied to software reuse. However, current research on software analogy often overlooks the importance of analysis for the base domain and does not consider some critical aspects of analogy concepts. Reuse must be based on high quality artifacts, especially reuse across domains. This paper presents an approach to integrate domain analysis and analogy methods. In our view, domain analysis and software analogy have complementary roles. Domain analysis is regarded as a process to identify and supply necessary information for analogical transfer. Software analogy can provide the analyst with similar problems and solutions to reuse previous domain analysis knowledge or artifacts for a new domain. This paper presents case studies to demonstrate the increase of efficiency in applying the approach. Evaluation of the approach from various perspectives is also reported.     

Comprehensibility of UML-based software product line specifications

Software Product Line Engineering (SPLE) deals with developing artifacts that capture the common and variable aspects of software product families. Domain models are one kind of such artifacts. Being developed in early stages, domain models need to specify commonality and variability and guide the reuse of the artifacts in particular software products. Although different modeling methods have been proposed to manage and support these activities, the assessment of these methods is still in an inceptive stage. In this work, we examined the comprehensibility of domain models specified in ADOM, a UML-based SPLE method. In particular, we conducted a controlled experiment in which 116 undergraduate students were required to answer comprehension questions regarding a domain model that was equipped with explicit reuse guidance and/or variability specification. We found that explicit specification of reuse guidance within the domain model helped understand the model, whereas explicit specification of variability increased comprehensibility only to a limited extent. Explicit specification of both reuse guidance and variability often provided intermediate results, namely, results that were better than specification of variability without reuse guidance, but worse than specification of reuse guidance without variability. All these results were perceived in different UML diagram types, namely, use case, class, and sequence diagrams and for different commonality-, variability-, and reuse-related aspects.     

A study of software reuse in NASA legacy systems

Software reuse is regarded as a highly important factor in reducing development overheads for new software projects; however, much of the literature is concerned with cost and labor savings that reuse brings to industrial software development and little is known about the inherent risks associated with reuse, particularly in the case of mission and safety-critical software systems. We present the preliminary findings of a research project geared toward assessing the impact of risk in National Aeronautics and Space Administration (NASA) legacy software in flight control systems. We introduce the concept of context variables and the impact they have on reuse within these legacy systems as well as the genealogy classification models, which provide a simple, concise method of mapping reuse between families of software projects. This research was conducted at Global Science and Technology, Inc. under NASA grant number NCC0NNG06GI57G.     

软件复用经济学模型比较分析

说明了软件复用经济学模型的主要目的和任务,对目前国际上提出的12个模型进行了比较分析。将复用经济学模型划分为两类：成本收益模型和投资分析模型,并举例介绍了相关类型的模型。对分析的所有模型列表比较,分析了它们的适用情况和异同点,并对领域存在的问题进行了讨论。     

Assessing software product line potential: an exploratory industrial case study

Corporate organizations sometimes offer similar software products in certain domains due to former company mergers or due to the complexity of the organization. The functional overlap of such products is an opportunity for future systematic reuse to reduce software development and maintenance costs. Therefore, we have tailored existing domain analysis methods to our organization to identify commonalities and variabilities among such products and to assess the potential for software product line (SPL) approaches. As an exploratory case study, we report on our experiences and lessons learned from conducting the domain analysis in four application cases with large-scale software products. We learned that the outcome of a domain analysis was often a smaller integration scenario instead of an SPL and that business case calculations were less relevant for the stakeholders and managers from the business units during this phase. We also learned that architecture reconstruction using a simple block diagram notation aids domain analysis and that large parts of our approach were reusable across application cases.