A quasi-renewal process for software reliability and testing costs

This paper models software reliability and testing costs using a new tool: a quasi-renewal process. It is assumed that the cost of fixing a fault during software testing phase, consists of both deterministic and incremental random parts, increases as the number of faults removed increases. Several software reliability and cost models by means of quasi-renewal processes are derived in which successive error-free times are independent and increasing by a fraction. The maximum likelihood estimates of parameters associated with these models are provided. Based on the valuable properties of quasi-renewal processes, the expected software testing and debugging cost, number of remaining faults in the software, and mean error-free time after testing are obtained. A class of related optimization problem is then contemplated and optimum testing policies incorporating both reliability and cost measures are investigated. Finally, numerical examples are presented through a set of real testing data to illustrate the models results     

A new testing acceleration chip for low-cost memory tests

It is argued that the development of semiconductor memories has reached a turning point. In the multimegabit dynamic random access memories (DRAMs) of the future, major factors contributing to the chip cost are process complexity, die size, equipment cost, and test cost. If conventional test methods are used, test costs will grow at an especially rapid rate. A memory test concept called the testing acceleration chip, which could reduce future test costs a hundredfold and yet maintain AC testing reliability, is presented     

面向多用户软件可靠性评估和测试方案研究

软件可靠性估计是软件可靠性研究的重要问题之一。本文提出一种根据软件随机测试结果进行软件可靠性估计的方法，给出了软件可靠度的最优置信下限，并以此为基础对随机测试方案的优化设计进行了研究。     

An experimental study of adaptive testing for software reliability assessment

Adaptive testing is a new form of software testing that is based on the feedback and adaptive control principle and can be treated as the software testing counterpart of adaptive control. Our previous work has shown that adaptive testing can be formulated and guided in theory to minimize the variance of an unbiased software reliability estimator and to achieve optimal software reliability assessment. In this paper, we present an experimental study of adaptive testing for software reliability assessment, where the adaptive testing strategy, the random testing strategy and the operational profile based testing strategy were applied to the Space program in four experiments. The experimental results demonstrate that the adaptive testing strategy can really work in practice and may noticeably outperform the other two. Therefore, the adaptive testing strategy can serve as a preferable alternative to the random testing strategy and the operational profile based testing strategy if high confidence in the reliability estimates is required or the real-world operational profile of the software under test cannot be accurately identified.     

软件可靠性估计与计算复杂性的关系浅析

软件可靠性估计是软件可靠性研究的重要问题之一.提出一种根据软件随机测试的结果进行软件可靠性估计的方法.它使被测软件的复杂性成为估计公式中的一个因素,从而克服了现有同类方法把软件看作是黑箱的缺点.它以软件的邻域集合的伪维数作为软件复杂性度量,从而使根据软件的可能错误空间进行可靠性估计成为可能.还证明了算法的计算复杂性与软件所可能计算的函数集合的伪维数之间存在着一定的联系.     

软件可靠性测试的理论分析

目前软件可靠性测试没有真正开展起来,即使重要的军用软件也没有进行软件可靠性测试。软件可靠性模型的不一致性以及软件可靠性测试时间长、费用高、资源消耗大是造成这种局面的两个主要问题。而这两个问题很大程度是由传统基于随机系统假设的软件可靠性理论导致的。因此,在该理论下对可靠性测试方法及可靠性模型的修补与改善工作很难从根本上解决问题。只有从软件可靠性理论本身入手,提出新的研究思路,才有可能摆脱目前软件可靠性测试的困境。首先介绍了软件可靠性测试的现状,然后重点对现有可靠性理论进行了分析和总结,在此基础上,提出了软件可靠性测试研究的一些新思路。     

A software cost model with imperfect debugging,random life cycle and penalty cost

The paper develops a cost model with an imperfect debugging and random life cycle as well as a penalty cost that is used to determine the optimal release policies for a software system. The software reliability model, based on the nonhomogeneous Poisson process, allows for three different error types: critical, major and minor errors. The model also allows for the introduction of any of these errors during the removal of an error. Using the software reliability model presented, the cost model with multiple error types and imperfect debugging is developed. This cost also considers the penalty cost due to delay for a scheduled delivery time and the length of the software life cycle is random with a known distribution. The optimal software release policies that minimize the expected software system costs (subject to the various constraints) or maximize the software reliability subject to a cost constraint, are then determined. Numerical examples are provided to illustrate the results.     

Are we testing for true reliability?

The relationship between software testing and reliability is discussed. Two kinds of reliability models, reliability-growth models, which are applied during debugging, and reliability models, which are applied after debugging, are described. Several reasons for the failure of conventional reliability theory in software engineering are presented. It is argued that until random testing of a million points becomes practical, testing for quality is only a poor competitor for other heuristic defect-detection methods     

多模块软件系统的最优发布原则

本文从软件模块的角度对软件系统对的可靠性和测试成本做出具体分析,讨论了多模块软件系统的成本和可靠性分配方式,从而使整个软件系统用最低的测试成本达到所要求的可靠性目标,然后用一个简单的例子说明了如何进行软件模块的可靠性和成本分配,从而实现软件的最优发布。     

基于多版本较准的软件费用模型研究

软件可靠性模型为软件的费用模型提供了很好的依据.在软件可靠性增长模型的测试阶段和操作运行阶段,环境的不同导致了两个阶段故障检测率的不同.在多版本较准的软件可靠性模型基础上,构造了一个综合了软件设计费用、软件测试费用、软件维护费用、软件失效造成的风险损失的软件费用模型.最后从软件费用出发,讨论了软件的最佳发布时间.     

软件可靠性测试中不确定性问题的研究

介绍了软件可靠性测试的现状,然后重点对现有可靠性理论进行了分析和总结,在此基础上,提出了基于不确定性理论的软件可靠性测试的一些新思路。探讨了为解决测试中的不确定性所采用的复用技术。结果表明,该方法可以在一定程度上缩短测试周期,降低测试成本,提高测试的可靠性以及测试效率,具有可行性。     

基于可靠性增长模型的软件可靠性增长测试充分性准则

结合软件可靠性增长模型,扩充了基于可信度度量的软件可靠性增长测试充分性问题度量准则,提出了关于可靠性增长测试充分性问题新的度量准则：可靠性测试只有同时满足可靠性增量指标要求、可信度要求和每单位测试资源发现的故障数要求方可终止。然后,以一个真实数据集为例,应用所提出度量准则求出了测试应该停止的时间。实验证明,这样的度量准则,使得终止条件更加严格,可靠性测试更加充分,为可靠性测试充分性问题的解决提供了新的思路。     

A fast approximate method for parameter sensitivity estimation in Monte Carlo structural reliability

In estimating the effect of a change in a random variable parameter on the (time-invariant) probability of structural failure estimated through Monte Carlo methods the usual approach is to carry out a duplicate simulation run for each parameter being varied. The associated computational cost may become prohibitive when many random variables are involved. Herein a procedure is proposed in which the numerical results from a Monte Carlo reliability estimation procedure are converted to a form that will allow the basic ideas of the first order reliability method to be employed. Using these allows sensitivity estimates of low computational cost to be made. Illustrative examples with sensitivities computed both by conventional Monte Carlo and the proposed procedure show good agreement over a range of probability distributions for the input random variables and for various complexities of the limit state function.     

On comparisons of random, partition, and proportional partitiontesting

Early studies of random versus partition testing used the probability of detecting at least one failure as a measure of test effectiveness and indicated that partition testing is not significantly more effective than random testing. More recent studies have focused on proportional partition testing because a proportional allocation of the test cases (according to the probabilities of the subdomains) can guarantee that partition testing will perform at least as well as random testing. We show that this goal for partition testing is not a worthwhile one. Guaranteeing that partition testing has at least as high a probability of detecting a failure comes at the expense of decreasing its relative advantage over random testing. We then discuss other problems with previous studies and show that failure to include important factors (cost, relative effectiveness) can lead to misleading results     

基于Markov决策过程用交叉熵方法优化软件测试

研究了待测软件某些参数已知的条件下,以最小化平均测试费用为目标的软件测试优化问题.将软件测试过程处理成马尔可夫(Markov)决策过程,给出了软件测试的马尔可夫决策模型,运用交叉熵方法,通过一种学习策略获得软件测试的最优测试剖面,用于优化软件测试.模拟结果表明,学习策略给出的测试剖面要优于随机测试策略,检测和排除相同数目的软件缺陷,学习策略比随机测试能够显著地减少测试用例数,降低测试成本,提高缺陷检测效率.     

An Evaluation of Random Testing

Random testing of programs has usually (but not always) been viewed as a worst case of program testing. Testing strategies that take into account the program structure are generally preferred. Path testing is an often proposed ideal for structural testing. Path testing is treated here as an instance of partition testing, where by partition testing is meant any testing scheme which forces execution of at least one test case from each subset of a partition of the input domain. Simulation results are presented which suggest that random testing may often be more cost effective than partition testing schemes. Also, results of actual random testing experiments are presented which confirm the viability of random testing as a useful validation tool.     

Risk-Based Stopping Criteria for Test Sequencing

Testing complex manufacturing systems, like ASML lithographic machines, can take up to 45% of the total development time. The decision of when to stop testing is often difficult to make because less testing may leave critical faults in the system, while more testing increases time-to-market. In this paper, we solve the problem of deciding when to stop testing by introducing a test-sequencing method that incorporates several stopping criteria. These stopping criteria consist of objectives and constraints on the test cost and the remaining risk cost. For a given problem, a suitable stopping criterion can be chosen. For example, with the risk-based stopping criterion, testing stops when the test time or cost exceeds the risk cost. Furthermore, we show that it also is possible to model reliability problems with this test-sequencing method. The method is demonstrated on ASML systems with two case studies. The first case study was conducted in the test phase during the development of the software that is used to control an ASML lithographic machine. The second case study was conducted on the reliability testing of a lithographic machine.      

软件故障暴露率的计算方法

软件的故障暴露率是软件测试和软件可靠性测试的一个重要参数，对高可靠性软件特别是军用软件的可靠性评估有关键的作用．根据PIE分析技术，给出软件单个故障或多个故障随机测试的效率分析，并以此为基础，给出软件故障暴露率的计算方法，该方法是实用和有效的．     

Optimal and adaptive testing for software reliability assessment

Optimal software testing is concerned with how to test software such that the underlying testing goal is achieved in an optimal manner. Our previous work shows that the optimal testing problem for software reliability growth can be treated as closed-loop or feedback control problem, where the software under test serves as a controlled object and the software testing strategy serves as the corresponding controller. More specifically, the software under test is modeled as controlled Markov chains (CMCs) and the control theory of Markov chains is used to synthesize the required optimal testing strategy. In this paper, we show that software reliability assessment can be treated as a feedback control problem and the CMC approach is also applicable to dealing with the optimal testing problem for software reliability assessment. In this problem, the code of the software under test is frozen and the software testing process is optimized in the sense that the variance of the software reliability estimator is minimized. An adaptive software testing strategy is proposed that uses the testing data collected on-line to estimate the required parameters and selects next test cases. Simulation results show that the proposed adaptive software testing strategy can really work in the sense that the resulting variance of the software reliability estimate is much smaller than that resulting from the random testing strategies. The work presented in this paper is a contribution to the new area of software cybernetics that explores the interplay between software and control.     

MC9S12在数字单元测试仪中的应用

设计了一种用于弹载计算机电路测试的数字单元测试仪,以MC9S12XS128单片机为核心,外接测试机构和LED输出端,通过嵌入式编程实现对测试机构电阻测试及电路转换等功能,结合MC9S12XS128单片机良好的稳定性、低成本低功耗等优点使测试仪具有较高性价比。测试结果表明该测试仪可靠性高、使用方便满足测试要求。