Predicting software reliability

Critical business applications require reliable software, but developing reliable software is one of the most difficult problems facing the software industry. After the software is shipped, software vendors receive customer feedback about software reliability. However, by then it is too late; software vendors need to know whether their products are reliable before they are delivered to customers. Software reliability growth models help provide that information. Unfortunately, very little real data and models from commercial applications have been published, possibly because of the proprietary nature of the data. Over the past few years, the author and his colleagues at Tandem have experimented with software reliability growth models. At Tandem, a major software release consists of substantial modifications to many products and may contain several million lines of code. Major software releases follow a well defined development process and involve a coordinated quality assurance effort. We applied software reliability modeling to a subset of products for four major releases. The article reports on what was learned     

Compound-Poisson software reliability model

The probability density estimation of the number of software failures in the event of clustering or clumping of the software failures is considered. A discrete compound Poisson (CP) prediction model is proposed for the random variable X_rem, which is the remaining number of software failures. The compounding distributions, which are assumed to govern the failure sizes at Poisson arrivals, are respectively taken to be geometric when failures are forgetful and logarithmic-series when failures are contagious. The expected value (μ) of X_rem is calculated as a function of the time-dependent Poisson and compounding distribution based on the failures experienced. Also, the variance/mean parameter for the remaining number of failures, q_rem, is best estimated by q_past from the failures already experienced. Then, one obtains the PDF of the remaining number of failures estimated by CP(μ,q). CP is found to be superior to Poisson where clumping of failures exists. Its predictive validity is comparable to the Musa-Okumoto log-Poisson model in certain cases     

Recalibrating software reliability models

There is no universally applicable software reliability growth model which can be trusted to give accurate predictions of reliability in all circumstances. A technique of analyzing predictive accuracy called the u-plot allows a user to estimate the relationship between the predicted reliability and the true reliability. It is shown how this can be used to improve reliability predictions in a very general way by a process of recalibration. Simulation results show that the technique gives improved reliability predictions in a large proportion of cases. However, a user does not need to trust the efficacy of recalibration, since the new reliability estimates produced by the technique are truly predictive and their accuracy in a particular application can be judged using the earlier methods. The generality of this approach suggests its use whenever a software reliability model is used. Indeed, although this work arose from the need to address the poor performance of software reliability models, it is likely to have applicability in other areas such as reliability growth modeling for hardware     

Evaluating television-linked computer software

Broadcasting organisations have considerable expertise in the evaluation of television and radio programmes, and U.K. broadcasters are held in particularly high esteem in this respect. They are, however, on rather less secure ground when it comes to multimedia resources which attempt to integrate broadcast and computer-based materials.The project described here was sponsored by the Independent Broadcasting Authority (IBA) and was carried out in the West Midlands region of England during the summer of 1986. It was undertaken to develop guidelines for the evaluation of broadcast-linked software and to provide insights for the creators of new multimedia materials. The research focussed on two primary education packages, in the areas of mathematics and history respectively, produced by Independent Television Companies.Each package (television programmes and associated courseware) was evaluated in collaboration with a team of teachers from six different schools. The evaluation process lasted a full school term (3 months). Some details of the process are discussed. It was concluded that there was strong potential support for multimedia educational packages incorporating broadcast television, computer software and other media. This conclusion is related to the growing field of interactive videodisc.     

异构软件体系结构的可靠性模型

现有基于软件体系结构的可靠性模型只适用于符合马尔可夫过程的软件构件转移关系,而当构件间存在由确定性和随机性转移关系组成的异构结构时,仍按照马尔可夫过程对软件体系结构建模,会导致可靠性评估结果不准确。为了解决现有模型不适用于异构结构的问题,在分析构件转移关系基础上,提出了一个新的软件可靠性模型。基于软件UML顺序图,提出将异构软件结构转换为马尔可夫链的算法。通过对一个实例的可靠性评估,验证了新模型能够比现有模型获得更准确的评估结果。     

A software reliability engineering practice

A recently adopted best current practice (BCP) at AT&T involving performing software reliability engineering (SRE) tasks over the software life cycle is discussed. Software reliability is defined as the capability of a software product to operate failure-free for a given period under specified operating conditions. The support mechanisms for the SRE BCP are described     

Why software reliability predictions fail

Software reliability reflects a customer's view of the products we build and test, as it is usually measured in terms of failures experienced during regular system use. But our testing strategy is often based on early product measures, since we cannot measure failures until the software is placed in the field. The author shows us that such measurement is not effective at predicting the likely reliability of the delivered software.     

星载软件可靠性测试实践

为验证和评估软件可靠性水平,阐述了开展软件可靠性测试的方法和过程,针对星载嵌入式软件的特点,介绍了进行软件可靠性测试的关键步骤的实现途径和方法,包括:失效的定义、测试环境的搭建、使用剖面的构造、测试过程的执行以及失效数据的收集,最后给出了该软件的可靠性定量评估结果,表明了该方法和流程的实用性和可行性,为后续开展类似的软件可靠性测试提供实践经验.     

一种改进的基于架构的软件可靠性模型

基于架构的软件可靠性分析往往把构件的可靠性当作自身固有不变的属性,忽略了在不同的输入剖面下,因构件所处的交互环境不同造成的实际可靠性的变化。改进了一种基于架构的可靠性模型,引入转移目的构件剖面矩阵来建立系统操作剖面和构件可靠性的联系,并给出了改进后的可靠性合成算法。实例分析表明,该模型可以全面捕捉到在不同系统操作剖面下,因构件之间转移概率和构件可靠度这两个参数的变化对整体可靠性产生的影响,提高了系统设计阶段可靠性分析的精确性。     

Software reliability analysis and choice of software reliability criteria

Functional reliability of computer software is considered using fuzzy automaton representation of software systems.Translated from Kibernetika i Sistemnyi Analiz, No. 2, pp. 46–60, March–April, 1992.     

软件可靠性与硬件可靠性分析及对比

介绍了可靠性的定义和基本概念,分析了软件可靠性和硬件可靠性的区别与联系,讨论了软件可靠性和硬件可靠性这两个领域中的一些基本研究及建模方法。     

An automated software reliability prediction system for safety critical software

Software reliability is one of the most important software quality indicators. It is concerned with the probability that the software can execute without any unintended behavior in a given environment. In previous research we developed the Reliability Prediction System (RePS) methodology to predict the reliability of safety critical software such as those used in the nuclear industry. A RePS methodology relates the software engineering measures to software reliability using various models, and it was found that RePS’s using Extended Finite State Machine (EFSM) models and fault data collected through various software engineering measures possess the most satisfying prediction capability. In this research the EFSM-based RePS methodology is improved and implemented into a tool called Automated Reliability Prediction System (ARPS). The features of the ARPS tool are introduced with a simple case study. An experiment using human subjects was also conducted to evaluate the usability of the tool, and the results demonstrate that the ARPS tool can indeed help the analyst apply the EFSM-based RePS methodology with less number of errors and lower error criticality.     

用于软件可靠性分析的分解方法

随机Petri网(SPN)是一种有力的系统建模和分析工具.但SPN在应用中经常碰到状态空间爆炸问题.分解压缩技术是解决随机网状态空间指数性增长的有效方法之一.介绍了一种获得SPN可靠性模型瞬时状态的分解方法.该方法在保证评价和预测可靠性精确度的基础上,不仅能有效地降低可靠性描述与分析的复杂度,还扩大了分解压缩技术的适用范围.     

Improving reliability of large software systems

Improving field performance of telecommunication systems is the key objective of both telecom suppliers and operators, as an increasing amount of business critical systems worldwide are relying on dependable telecommunication. Early defect detection improves field performance in terms of reduced field failure rates and reduced intrinsic downtime. This paper describes an integrated approach to improve early defect detection and thus field reliability of telecommunication switching systems. The assumptions at the start of the projects discussed in this paper are: Wide application of code inspections and thorough module testing must lead to a lower fault detection density in subsequent phases. At the same time criteria for selecting the most critical components for code reviews, code inspections and module test are provided in order to optimize efficiency. The primary goal is to identify critical components and to make failure predictions as early as possible during the life cycle and hence reduce managerial risk combined with too early or too late release of such a system to the field. During test release time prediction and field performance prediction are both based on tailored and superposed ENHPP reliability models. Experiences from projects of Alcatel’s Switching and Routing Division are included to show practical impacts. This revised version was published online in June 2006 with corrections to the Cover Date.     

基于体系结构的软件可靠性评估综述

为深入了解基于体系结构的软件可靠性评估方法的发展情况,针对当前该方法的发展现状,对基于体系结构的软件可靠性评估的起源、发展、分类和典型模型进行阐述,重点介绍状态模型、路径模型、构建操作剖面模型及其它模型;根据各种模型的特点,分析现有模型中存在的问题,对软件可靠性模型今后的发展趋势进行展望。     

Neural computing for software reliability*

Abstract: We present the results of a feasibility study for the application of neural computing to the traditional problem of how to generate cost-effective, reliable implementations of complex problems—i.e. the central problem of software engineering. We treat neural computing as an innovative technology for conventional software engineering. We explore the reliability of neural networks (multilayer perceptrons trained with the backpropagation algorithm) as alternative versions in a multiversion software system. The basic idea is that versions trained differently will not exhibit common faults as independently developed, conventional versions (programmed in, for example, Modula-2) have been shown to do. The common design faults that run through independently developed versions appear to be the result of ‘difficult’ inputs which all programmers tend to misconstrue similarly. Network implementations, which are not directly designed in the conventional manner, should permit easy introduction of ‘diversity’ to combat this weakness. The initial results give credence to this possibility and have shown the way to generate substantial forced diversity within the neural computing paradigm.     

面向软件可信性的可信指针分析技术综述*

对可信指针分析技术的定义和描述、指针分析对软件可信性的保障、可信指针分析属性以及该领域主要研究成果等方面进行了综述。通过对现有可信指针分析技术的分析和比较,详细讨论了面向软件可信性的可信指针分析的关键技术;此外,重点介绍了流敏感指针分析及上下文敏感指针分析的方法和理论;最后对进一步研究工作的方向进行了展望。