Weibull分布引进故障的软件可靠性增长模型

软件调试是复杂过程,可能会受到很多种因素的影响,例如调试资源分配、调试工具的使用情况、调试技巧等.在软件调试过程中,当检测到的故障被去除时,新的故障可能会被引进.因此,研究故障引进的现象对建立高质量的软件可靠性增长模型具有重要意义.但是到目前为止,模拟故障引进过程仍是一个复杂和困难的问题.虽然有许多研究者开发了一些不完美调试的软件可靠性增长模型,但是一般都是假设故障内容（总数）函数为线性、指数分布或者是与故障去除的数量成正比.这个假设与实际的软件调试过程中故障引进情况并不完全一致.提出一种基于Weibull分布引进故障的软件可靠性增长模型,考虑故障内容（总数）函数服从Weibull分布,并用相关的实验验证了提出的模型的拟合和预测性能.在用两个故障数据集进行的模拟实验中,实验结果指出：提出的模型和其他模型相比,有更好的拟合和预测性能以及更好的鲁棒性.     

Mathematical modeling of software reliability testing with imperfect debugging

Software reliability testing is concerned with the quantitative relationship between software testing and software reliability. Our previous work develops a mathematically rigorous modeling framework for software reliability testing. However the modeling framework is confined to the case of perfect debugging, where detected defects are removed without introducing new defects. In this paper the modeling framework is extended to the case of imperfect debugging and two models are proposed. In the first model it is assumed that debugging is imperfect and may make the number of remaining defects reduce by one, remain intact, or increase by one. In the second model it is assumed that when the number of remaining defects reaches the upper bound, the probability that the number of remaining defects is increased by one by debugging is zero. The expected behaviors of the cumulative number of observed failures and the number of remaining defects in the first model show that the software testing process may induce a linear or nonlinear dynamic system, depending on the relationship between the probability of debugging introducing a new defect and that of debugging removing a detected defect. The second-order behaviors of the first model also show that in the case of imperfect debugging, although there may be unbiased estimator for the initial number of defects remaining in the software under test, the cumulative number of observed failures and the current number of remaining defects are not sufficient for precisely estimating the initial number of remaining defects. This is because the variance of the unbiased estimator approaches a non-zero constant as the software testing process proceeds. This may be treated as an intrinsic principle of uncertainty for software testing. The expected behaviors of the cumulative number of observed failures and the number of remaining defects in the second model show that the software testing process may induce a nonlinear dynamic system. However theoretical analysis and simulation results show that, if defects are more often removed from than introduced into the software under test, the expected behaviors of the two models tend to coincide with each other as the upper bound of the number of remaining defects approaches infinity.     

A study of the effect of imperfect debugging on software development cost

It is widely recognized that the debugging processes are usually imperfect. Software faults are not completely removed because of the difficulty in locating them or because new faults might be introduced. Hence, it is of great importance to investigate the effect of the imperfect debugging on software development cost, which, in turn, might affect the optimal software release time or operational budget. In this paper, a commonly used cost model is extended to the case of imperfect debugging. Based on this, the effect of imperfect debugging is studied. As the probability of perfect debugging, termed testing level here, is expensive to be increased, but manageable to a certain extent with additional resources, a model incorporating this situation is presented. Moreover, the problem of determining the optimal testing level is considered. This is useful when the decisions regarding the test team composition, testing strategy, etc., are to be made for more effective testing.     

Considering the fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time

Since the early 1970s tremendous growth has been seen in the research of software reliability growth modeling.In general, software reliability growth models (SRGMs) are applicable to the late stages of testing in software development and they can provide useful information about how to improve the reliability of software products.A number of SRGMs have been proposed in the literature to represent time-dependent fault identification/removal phenomenon;still new models are being proposed that could fit a greater number of reliability growth curves.Often,it is assumed that detected faults axe immediately corrected when mathematical models are developed.This assumption may not be realistic in practice because the time to remove a detected fault depends on the complexity of the fault,the skill and experience of the personnel,the size of the debugging team,the technique,and so on.Thus,the detected fault need not be immediately removed,and it may lag the fault detection process by a delay effect factor.In this paper,we first review how different software reliability growth models have been developed,where fault detection process is dependent not only on the number of residual fault content but also on the testing time,and see how these models can be reinterpreted as the delayed fault detection model by using a delay effect factor.Based on the power function of the testing time concept,we propose four new SRGMs that assume the presence of two types of faults in the software:leading and dependent faults.Leading faults are those that can be removed upon a failure being observed.However,dependent faults are masked by leading faults and can only be removed after the corresponding leading fault has been removed with a debugging time lag.These models have been tested on real software error data to show its goodness of fit,predictive validity and applicability.     

不完美排错软件可靠性增长模型效用量化研究

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合和预测性能,首先从故障总数增长趋势角度对不完美排错模型进行深入研究,提出两个一般性不完美排错框架模型,分别考虑了总故障数量函数与累计检测故障函数间的线性关系与微分关系,并求得累计检测的故障数量与软件中总故障数量函数表达式;其次,在六组真实的失效数据集上对比了提出的两种一般性不完美排错模型和六种不完美排错模型拟合预测性能表现。实例验证结果表明,提出的一般性不完美排错框架模型在大多数失效数据集上都具有优秀的拟合和预测性能,证明了新建模型的有效性和实用性;通过对提出的模型与其他不完美排错模型在数据集上的性能的深入分析,为实际应用中不完美排错模型的选择提出了建议。     

A quasi-renewal process for software reliability and testing costs

This paper models software reliability and testing costs using a new tool: a quasi-renewal process. It is assumed that the cost of fixing a fault during software testing phase, consists of both deterministic and incremental random parts, increases as the number of faults removed increases. Several software reliability and cost models by means of quasi-renewal processes are derived in which successive error-free times are independent and increasing by a fraction. The maximum likelihood estimates of parameters associated with these models are provided. Based on the valuable properties of quasi-renewal processes, the expected software testing and debugging cost, number of remaining faults in the software, and mean error-free time after testing are obtained. A class of related optimization problem is then contemplated and optimum testing policies incorporating both reliability and cost measures are investigated. Finally, numerical examples are presented through a set of real testing data to illustrate the models results     

考虑不完美排错情况的NHPP 类软件可靠性增长模型

针对现有NHPP 类软件可靠性增长模型对故障排错过程中不完美排错情况考虑不完全的现状,提出了一 种新的软件可靠性增长模型.该模型全面考虑了不完美排错的两种情况:既考虑了排错过程中引入新错误的可能性, 又考虑了不完全排错的情况,并且引入了一种故障排除率随时间变化的故障排除率函数,使模型更符合实际情况.利 用公开发表的两组不同的软件失效数据对该模型进行验证的结果表明,与现有的对不完美排错情况考虑不完全的 模型相比,该模型能够取得更好的拟合结果和预测效果.     

考虑不完美排错情况的NHPP 类软件可靠性增长模型

针对现有NHPP类软件可靠性增长模型对故障排错过程中不完美排错情况考虑不完全的现状,提出了一种新的软件可靠性增长模型.该模型全面考虑了不完美排错的两种情况:既考虑了排错过程中引入新错误的可能性,又考虑了不完全排错的情况,并且引入了一种故障排除率随时间变化的故障排除率函数,使模型更符合实际情况.利用公开发表的两组不同的软件失效数据对该模型进行验证的结果表明,与现有的对不完美排错情况考虑不完全的模型相比,该模型能够取得更好的拟合结果和预测效果.     

Discrete software reliability growth modeling for errors of different severity incorporating change-point concept

Several software reliability growth models (SRGM) have been developed to monitor the reliability growth during the testing phase of software development.In most of the existing research available in the literatures,it is considered that a similar testing effort is required on each debugging effort.However,in practice,different types of faults may require different amounts of testing efforts for their detection and removal.Consequently,faults are classified into three categories on the basis of severity:simple,hard and complex.This categorization may be extended to (?) type of faults on the basis of severity.Although some existing research in the literatures has incorporated this concept that fault removal rate (FRR) is different for different types of faults,they assume that the FRR remains constant during the overall testing period.On the contrary,it has been observed that as testing progresses,FRR changes due to changing testing strategy,skill,environment and personnel resources.In this paper,a general discrete SRGM is proposed for errors of different severity in software systems using the change-point concept.Then,the models are formulated for two particular environments.The models were validated on two real-life data sets.The results show better fit and wider applicability of the proposed models as to different types of failure datasets.     

Trends in reliability and test strategies

Software fails because it contains faults. If we could be certain that our attempts to fix a fault had been effective and that we had introduced no new faults, we would know that we had improved the software's reliability. Unfortunately, no single testing method can be trusted to give accurate results in every circumstance. There are at least two areas of uncertainty in testing. First, we cannot predict when a failure will occur as a result of choosing an input that cannot currently be processed correctly. Second, we do not know what effect fixing a fault will have on the software's reliability. Most of today's reliability models consider the software system to be a black box, using indirect measures of failure by making strong assumptions about the test cases. White-box testing measures, which are direct measures, are generally not used in software-reliability models. The assumption is that the more the program is covered, the more likely that the software is reliable. Hence we must develop new testing strategies. For example, we do not understand very well the notion of stress as it applies to software; what effect the dependence between versions of the same application has on reliability; the relationship between software reliability and white-box testing measures; and how to achieve the ultra-high reliability levels obtained through formal verification and fault-tolerance methods. These are just some of the areas for future research into improved testing strategies     

Reliability growth modeling and optimal release policy under fuzzy environment of an N-version programming system incorporating the effect of fault removal efficiency

Failure of a safety critical system can lead to big losses.Very high software reliability is required for automating the working of systems such as aircraft controller and nuclear reactor controller software systems.Fault-tolerant softwares are used to increase the overall reliability of software systems.Fault tolerance is achieved using the fault-tolerant schemes such as fault recovery (recovery block scheme),fault masking (N-version programming (NVP)) or a combination of both (Hybrid scheme).These softwares incorporate the ability of system survival even on a failure.Many researchers in the field of software engineering have done excellent work to study the reliability of fault-tolerant systems.Most of them consider the stable system reliability.Few attempts have been made in reliability modeling to study the reliability growth for an NVP system.Recently,a model was proposed to analyze the reliability growth of an NVP system incorporating the effect of fault removal efficiency.In this model,a proportion of the number of failures is assumed to be a measure of fault generation while an appropriate measure of fault generation should be the proportion of faults removed.In this paper,we first propose a testing efficiency model incorporating the effect of imperfect fault debugging and error generation.Using this model,a software reliability growth model (SRGM) is developed to model the reliability growth of an NVP system.The proposed model is useful for practical applications and can provide the measures of debugging effectiveness and additional workload or skilled professional required.It is very important for a developer to determine the optimal release time of the software to improve its performance in terms of competition and cost.In this paper,we also formulate the optimal software release time problem for a 3VP system under fuzzy environment and discuss a the fuzzy optimization technique for solving the problem with a numerical illustration.     

Optimal number of hosts in a distributed system based on cost criteria

Redundant or distributed systems are increasingly used in system design so that the required reliability and availability can be easily achieved. However, such an approach requires additional resources that can be very costly. Hence, how to design and test such a system in the most cost-effective way is of concern to the developers. A general cost model and a solution algorithm are presented for the determination of the optimal number of hosts and optimal system debugging time that minimize the total cost while achieving a certain performance objective. During testing, software faults are corrected and the reliability shows an increasing trend, and hence system reliability increases. A general system model is constructed based on a Markov process with software reliability and availability obtained from software reliability growth models. The optimization problem is formulated based on the cost criteria and the solution procedure is described. An application example is presented.     

考虑排错过程引进故障的开源软件可靠性模型研究

近年来,开源软件在软件行业很受欢迎。但是,开源软件的可靠性却受到人们的广泛质疑。如何评估开源软件的可靠性是一个重要的问题。与传统的闭源软件相比,在建立开源软件可靠性模型时,必须考虑故障引入和故障检测与排错之间的延迟时间这两个因素。本文考虑了排错过程和不完美调试现象,提出了相应的开源软件可靠性模型。并且我们用两个开源软件故障数据集实来验证提出模型的拟合性能与预测性能。实验结果表明,提出的模型在开源软件可靠性评估中具有良好的拟合和预测性能。提出的模型可以用于开源软件在实际的开发过程中的可靠性评估。     

Debugging‐workflow‐aware software reliability growth analysis

Software reliability growth models support the prediction/assessment of product quality, release time, and testing/debugging cost. Several software reliability growth model extensions take into account the bug correction process. However, their estimates may be significantly inaccurate when debugging fails to fully fit modelling assumptions. This paper proposes debugging‐workflow‐aware software reliability growth method (DWA‐SRGM), a method for reliability growth analysis leveraging the debugging data usually managed by companies in bug tracking systems. On the basis of a characterization of the debugging workflow within the software project under consideration (in terms of bug features and treatment phases), DWA‐SRGM pinpoints the factors impacting the estimates and to spot bottlenecks, thus supporting process improvement decisions. Two industrial case studies are presented, a customer relationship management system and an enterprise resource planning system, whose defects span a period of about 17 and 13 months, respectively. DWA‐SRGM revealed effective to obtain more realistic estimates and to capitalize on the awareness of critical factors for improving debugging.     

考虑测试工作量与覆盖率的软件可靠性模型

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合与预计精度,首先,提出一个同时考虑测试工作量与测试覆盖率的NHPP类软件可靠性建模框架.在此基础上,将变形S型测试工作量函数(IS-TEF)以及Logistic测试覆盖率函数(LO-TCF)带入该建模框架,建立了一个新的软件可靠性增长模型,即IS-LO-SRGM.同时,还对利用该框架进行建模过程中的两个重要问题进行了描述与分析,即如何确定具体的TEF和TCF以及模型参数估计.然后,在两组真实的失效数据集上,利用该建模框架建立了最为合适的增长模型,即IS-LO-SRGM,并将该模型与8种经典NHPP模型进行对比.实例验证结果表明,所提出的IS-LO-SRGM模型具有最为优秀的拟合与预计性能,从而证明新建模框架的有效性和实用性.最后,对不完美排错情况进行了初步的讨论与建模分析.     

Are we testing for true reliability?

The relationship between software testing and reliability is discussed. Two kinds of reliability models, reliability-growth models, which are applied during debugging, and reliability models, which are applied after debugging, are described. Several reasons for the failure of conventional reliability theory in software engineering are presented. It is argued that until random testing of a million points becomes practical, testing for quality is only a poor competitor for other heuristic defect-detection methods