Controlling and predicting the quality of space shuttle software using metrics

Software quality metrics have potential for helping to assure the quality of software on large projects such as the Space Shuttle flight software. It is feasible to validate metrics for controlling and predicting software quality during design by validating metrics against a quality factor. Quality factors, like reliability, are of more interest to customers than metrics, like complexity. However quality factors cannot be collected until late in a project. Therefore the need arises to validate metrics, which developers can collect early in a project, against a quality factor. We investigate the feasibility of validating metrics for controlling and predicting quality on the Space Shuttle. The key to the approach is the use of validated metrics for early identification and resolution of quality problems.     

Software metrics validation: Space Shuttle flight software example

Software quality metrics have potential for helping to ensure the quality of software on large projects such as theSpace Shuttle flight software. It is feasible to validate metrics for the purpose of controlling and predicting software quality during design by validating metrics against a quality factor. Quality factors, like reliability, are of more interest to customers than metrics, like complexity. However, quality factors cannot be collected until late in a project. Therefore, the need arises to validate metrics, which developers can collect early in a project, against a quality factor. We investigate the feasibility of validating metrics for controlling and predicting quality on theSpace Shuttle. The key to the approach is the use of validated metrics for early identification and resolution of quality problems.     

The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process

Many empirical studies have found that software metrics can predict class error proneness and the prediction can be used to accurately group error-prone classes. Recent empirical studies have used open source systems. These studies, however, focused on the relationship between software metrics and class error proneness during the development phase of software projects. Whether software metrics can still predict class error proneness in a system’s post-release evolution is still a question to be answered. This study examined three releases of the Eclipse project and found that although some metrics can still predict class error proneness in three error-severity categories, the accuracy of the prediction decreased from release to release. Furthermore, we found that the prediction cannot be used to build a metrics model to identify error-prone classes with acceptable accuracy. These findings suggest that as a system evolves, the use of some commonly used metrics to identify which classes are more prone to errors becomes increasingly difficult and we should seek alternative methods (to the metric-prediction models) to locate error-prone classes if we want high accuracy.     

Software measurement: Problems and practice

Software measurement is a key element in the evolving software engineering discipline. This paper seeks to identify some of the principal problems surrounding the measurement process and the metrics themselves. Metrics are shown in various modeling contexts. Measures of static software complexity are explored together with measures of dynamic program complexity. These data are then used to identify regions of software that are fault prone. This information is then exploited to develop a model of dynamic program complexity for the identification of failure prone software. Empirical experiences with the practice of measurement from an ongoing research project on the Space Shuttle Primary Avionics Software System are employed to augment the discussion.     

A Model-Based Approach to Object-Oriented Software Metrics

The need to improve software productivity and software quality has put forward the research on software metrics technology and the development of software metrics tool to support related activities.To support object-oriented software metrics practice efectively,a model-absed approach to object-oriented software metrics is proposed in this paper.This approach guides the metrics users to adopt the quality metrics model to measure the object-oriented software products .The development of the model can be achieved by using a top-down approach.This approach explicitly proposes the conception of absolute normalization computation and relative normalization computation for a metrics model.Moreover,a generic software metrics tool-Jade Bird Object-Oriented Metrics Tool(JBOOMT)is designed to implement this approach.The parser-based approach adopted by the tool makes the information of the source program accurate and complete for measurement.It supports various customizable hierarchical metrics models and provides a flexible user interface for users to manipulate the models.It also supports absolute and relative normalization mechanisms in different situations.     

Deriving complexity information from a formal communication protocol specification

Communication software systems have become very large and complex. Recognizing the complexity of such software systems is a key element in their development activities. Software metrics are useful quantitative indicators for assessing and predicting software quality attributes, like complexity. However, most of existing metrics are extracted from source programs at the implementation phase of the software life cycle. They cannot provide early feedback during the specification phase; and subsequently it is difficult and expensive to make changes to the system, if so indicated by the metrics. It is therefore important to be able to measure system complexity at the specification phase. However, most software specifications are written in natural languages from which metrics information is very hard to extract. In this paper, we describe how complexity information can be derived from a formal communication protocol specification written in Estelle so that it is possible to predict the complexity of its implementation and subsequently its development can be better managed. © 1998 John Wiley & Sons, Ltd.     

Validating metrics for ensuring Space Shuttle flight softwarequality

In this article, we cover the validation of software quality metrics for the Space Shuttle. Experiments with Space Shuttle flight software show that the Boolean OR discriminator function can successfully validate metrics for controlling and predicting quality. Further, we found that statement count and node count are the metrics most closely associated with the discrepancy reports count, and that with data smoothing their critical values can be used as predictors of software quality. We are continuing our research and comparing validated metrics with actual results in an attempt to determine whether the use of additional metrics provides sufficiently greater discriminative power to justify the increased inspection costs     

基于持续集成的软件度量

软件度量是针对软件开发项目、过程及产品进行数据定义、收集和分析的持续度量化过程;持续集成工具上的的构建工程每天自动完成从版本库更新代码、静态检查、编译、出包、自动化用例测试等任务,在进行集成构建的过程中可以为软件度量提供多种相关的度量数据;结合工作实践,叙述了基于持续集成的软件度量的原理;软件度量管理涉及到的角色;软件度量实现过程;叙述了基于持续集成的两种类型的软件度量指标的定义以及提取方法;最后详细叙述了在软件度量过程中遇到的几个典型案例;工作实践表明在软件的开发过程中做好软件度量工作有助于软件开发部门控制、预测、和改进软件产品的质量与软件开发过程;从而提高软件质量和软件开发效率,降低软件开发成本。     

Investigating ARIMA models of software system quality

In this paper, we investigate how to incorporate program complexity measures with a software quality model. We collect software complexity metrics and fault counts from each build during the testing phase of a large commercial software system. Though the data are limited in quantity, we are able to predict the number of faults in the next build. The technique we used is called times series analysis and forecasting. The methodology assumes that future predictions are based on the history of past observations. We will show that the combined complexity quality model is an improvement over the simpler quality only model. Finally, we explore how the testing process used in this development may be improved by using these predictions and suggest areas for future research.     

Evaluation of the DRM and the Time for Preventive Maintenance for Aging Software

An attempt to capture software aging and specify when to do preventive maintenance (PM) is presented in this paper. A composite measure termed the DRM, Deteriorating Response Measure, is defined. It is based on the analysis of the deteriorating speed of the software against time and load. This speed is characterized as follows: (1) It decays with increased load. (2) It does not increase again when the load decreases, this indicates loss of elasticity. The DRM is mathematically formulated based on a queueing system model.Specifying when to do preventive maintenance depends on the decision maker's perspective of the manifestation of aging. It is tried here to formalize this dependence. Three degrading performance metrics are defined for a DRM: (1) Decaying restored speed value. (2) Increasing speed offset ratio (recoverability index). (3) Increasing operation interval offset. These metrics can be used singly or aggregately in a DRM to specify when to do preventive maintenance. Results of model testing are also shown.This work can be part of an on-line procedure that calculates performance indexes for a server type software system like a web browser, an operating system, a database, and helps in the decision to do preventive maintenance for aging software.     

Multiple-parameter coupling metrics for layered component-based software

Coupling represents the degree of interdependence between two software components. Understanding software dependency is directly related to improving software understandability, maintainability, and reusability. In this paper, we analyze the difference between component coupling and component dependency, introduce a two-parameter component coupling metric and a three-parameter component dependency metric. An important parameter in both these metrics is coupling distance, which represents the relevance of two coupled components. These metrics are applicable to layered component-based software. These metrics can be used to represent the dependencies induced by all types of software coupling. We show how to determine coupling and dependency of all scales of software components using these metrics. These metrics are then applied to Apache HTTP, an open-source web server. The study shows that coupling distance is related to the number of modifications of a component, which is an important indicator of component fault rate, stability and subsequently, component complexity.

Rules and Tools for Software Evolution Planning and Management

When first formulated in the early seventies, the laws of software evolution were, for a number of reasons, not widely accepted as relevant to software engineering practice. Over the years, however, they have gradually become recognised as providing useful inputs to understanding of the software process. Now eight in number, they have been supplemented by the software uncertainty principle and the FEAST (Feedback, Evolution And Software Technology) hypothesis. Based on all these and on the further results of the FEAST research projects this paper develops and presents some fifty rules for application in software system process planning and management and indicates tools available or that could usefully be developed to support their application. The listing is structured according to the laws that encapsulate the observed phenomena and that lead to the recommendations. Each sublist is preceded by a textual discussion providing at least some of the reasoning that has led to the recommended procedures. The references direct the interested reader to the literature that records observed behaviours, interpretations, models and metrics obtained from industrially evolved systems, and from which the recommendations were derived.     

Comparative aspects of software complexity metrics and program modules — a multidimensional scaling approach

A multivariate statistical procedure called multidimensional scaling is used to study the relationship of various software complexity metrics and program modules. The program modules that make up a software system are analysed and their effects towards the overall characteristics of a software are viewed. This multidimensional scaling technique is applied to a sample data set. The scaling procedure clustered the similar and dissimilar software complexity metrics. Program modules with low complexity and few errors clustered together, while modules which were complex were isolated. This technique shows promise in the identification of complex modules that potentially contain disproportionate errors prior to the testing phase. The ability of the scaling techniques to cluster similar and dissimilar characteristics is explained and graphically presented.     

软件开发过程持续改进的方法研究

随着软件的规模和复杂度的不断增大，软件开发过程越来越难以度量和保持持续改进。通过对软件开发过程中软件度量计划、度量实施、度量分析的研究，提出了一种软件开发过程持续改进的螺旋式模型，用于促使软件开发过程的持续改进和软件组织能力的有效提升。     

Using multiple metrics for analysis of improvement

To make global decisions about a project or group of projects, it is necessary to analyse several metrics in concert, as well as changes in individual metrics. This paper discusses several approaches to collective metrics analysis. First, classification tree analysis is described as a technique for evaluating both process and metrics characteristics. Next, the notion of a multiple metrics graph is introduced. Developed initially as a way to evaluate software switch quality, a multiple metrics graph allows collections of metrics to be viewed in terms of overall product or process improvement.This work was done while the authors were affiliated with Contel Technology Center, Chantilly, Virginia, USA.     

A study of software reuse in NASA legacy systems

Software reuse is regarded as a highly important factor in reducing development overheads for new software projects; however, much of the literature is concerned with cost and labor savings that reuse brings to industrial software development and little is known about the inherent risks associated with reuse, particularly in the case of mission and safety-critical software systems. We present the preliminary findings of a research project geared toward assessing the impact of risk in National Aeronautics and Space Administration (NASA) legacy software in flight control systems. We introduce the concept of context variables and the impact they have on reuse within these legacy systems as well as the genealogy classification models, which provide a simple, concise method of mapping reuse between families of software projects. This research was conducted at Global Science and Technology, Inc. under NASA grant number NCC0NNG06GI57G.     

Assessing the maintainability of software product line feature models using structural metrics

A software product line is a unified representation of a set of conceptually similar software systems that share many common features and satisfy the requirements of a particular domain. Within the context of software product lines, feature models are tree-like structures that are widely used for modeling and representing the inherent commonality and variability of software product lines. Given the fact that many different software systems can be spawned from a single software product line, it can be anticipated that a low-quality design can ripple through to many spawned software systems. Therefore, the need for early indicators of external quality attributes is recognized in order to avoid the implications of defective and low-quality design during the late stages of production. In this paper, we propose a set of structural metrics for software product line feature models and theoretically validate them using valid measurement-theoretic principles. Further, we investigate through controlled experimentation whether these structural metrics can be good predictors (early indicators) of the three main subcharacteristics of maintainability: analyzability, changeability, and understandability. More specifically, a four-step analysis is conducted: (1) investigating whether feature model structural metrics are correlated with feature model maintainability through the employment of classical statistical correlation techniques; (2) understanding how well each of the structural metrics can serve as discriminatory references for maintainability; (3) identifying the sufficient set of structural metrics for evaluating each of the subcharacteristics of maintainability; and (4) evaluating how well different prediction models based on the proposed structural metrics can perform in indicating the maintainability of a feature model. Results obtained from the controlled experiment support the idea that useful prediction models can be built for the purpose of evaluating feature model maintainability using early structural metrics. Some of the structural metrics show significant correlation with the subjective perception of the subjects about the maintainability of the feature models.     

Software quality control and prediction model for maintenance

We develop a quality control and prediction model for improving the quality of software delivered by development to maintenance. This model identifies modules that require priority attention during development and maintenance by using Boolean discriminant functions. The model also predicts during development the quality that will be delivered to maintenance by using both point and confidence interval estimates of quality. We show that it is important to perform a marginal analysis when making a decision about how many metrics to include in a discriminant function. If many metrics are added at once, the contribution of individual metrics is obscured. Also, the marginal analysis provides an effective rule for deciding when to stop adding metrics. We also show that certain metrics are dominant in their effects on classifying quality and that additional metrics are not needed to increase the accuracy of classification. Related to this property of dominance is the property of concordance, which is the degree to which a set of metrics produces the same result in classifying software quality. A high value of concordance implies that additional metrics will not make a significant contribution to accurately classifying quality; hence, these metrics are redundant. Data from the Space Shuttle flight software are used to illustrate the model process.