军用软件可靠性测试管理中模型的应用

软件可靠性测试是软件开发过程中的一个重要环节,将软件可靠性增长模型应用到软件可靠性的测试过程中,可以为测试的进行提供有价值的管理决策依据。军用软件可靠性的测试及管理是控制军用软件质量的重要方法。该文给出了一种军用软件测试管理模型,在此基础上进一步讨论了Musa模型在测试管理中的应用,最后对实例进行了分析。     

Selecting software reliability growth models and improving their predictive accuracy using historical projects data

During software development two important decisions organizations have to make are: how to allocate testing resources optimally and when the software is ready for release. SRGMs (software reliability growth models) provide empirical basis for evaluating and predicting reliability of software systems. When using SRGMs for the purpose of optimizing testing resource allocation, the model's ability to accurately predict the expected defect inflow profile is useful. For assessing release readiness, the asymptote accuracy is the most important attribute. Although more than hundred models for software reliability have been proposed and evaluated over time, there exists no clear guide on which models should be used for a given software development process or for a given industrial domain.Using defect inflow profiles from large software projects from Ericsson, Volvo Car Corporation and Saab, we evaluate commonly used SRGMs for their ability to provide empirical basis for making these decisions. We also demonstrate that using defect intensity growth rate from earlier projects increases the accuracy of the predictions. Our results show that Logistic and Gompertz models are the most accurate models; we further observe that classifying a given project based on its expected shape of defect inflow help to select the most appropriate model.     

An Empirical Method for Selecting Software Reliability Growth Models

Estimating remaining defects (or failures) in software can help test managers make release decisions during testing. Several methods exist to estimate defect content, among them a variety of software reliability growth models (SRGMs). SRGMs have underlying assumptions that are often violated in practice, but empirical evidence has shown that many are quite robust despite these assumption violations. The problem is that, because of assumption violations, it is often difficult to know which models to apply in practice. We present an empirical method for selecting SRGMs to make release decisions. The method provides guidelines on how to select among the SRGMs to decide on the best model to use as failures are reported during the test phase. The method applies various SRGMs iteratively during system test. They are fitted to weekly cumulative failure data and used to estimate the expected remaining number of failures in software after release. If the SRGMs pass proposed criteria, they may then be used to make release decisions. The method is applied in a case study using defect reports from system testing of three releases of a large medical record system to determine how well it predicts the expected total number of failures.     

考虑测试工作量与覆盖率的软件可靠性模型

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合与预计精度,首先,提出一个同时考虑测试工作量与测试覆盖率的NHPP类软件可靠性建模框架.在此基础上,将变形S型测试工作量函数(IS-TEF)以及Logistic测试覆盖率函数(LO-TCF)带入该建模框架,建立了一个新的软件可靠性增长模型,即IS-LO-SRGM.同时,还对利用该框架进行建模过程中的两个重要问题进行了描述与分析,即如何确定具体的TEF和TCF以及模型参数估计.然后,在两组真实的失效数据集上,利用该建模框架建立了最为合适的增长模型,即IS-LO-SRGM,并将该模型与8种经典NHPP模型进行对比.实例验证结果表明,所提出的IS-LO-SRGM模型具有最为优秀的拟合与预计性能,从而证明新建模框架的有效性和实用性.最后,对不完美排错情况进行了初步的讨论与建模分析.     

软件可靠性及可靠性多模型综合研究

软件可靠性验证阶段的可靠性增长模型的建立与选择是软件可靠性工程人员长期关注的焦点。首先对软件可靠性基本概念及影响因素和工程确认的几种软件可靠性增长模型进行阐述,探讨近年来工程人员对软件可靠性增长模型的改进方法研究进展,提出了基于模型聚类的混合模型方法,并对该方法进行了实验性仿真分析。     

Considering the fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time

Since the early 1970s tremendous growth has been seen in the research of software reliability growth modeling.In general, software reliability growth models (SRGMs) are applicable to the late stages of testing in software development and they can provide useful information about how to improve the reliability of software products.A number of SRGMs have been proposed in the literature to represent time-dependent fault identification/removal phenomenon;still new models are being proposed that could fit a greater number of reliability growth curves.Often,it is assumed that detected faults axe immediately corrected when mathematical models are developed.This assumption may not be realistic in practice because the time to remove a detected fault depends on the complexity of the fault,the skill and experience of the personnel,the size of the debugging team,the technique,and so on.Thus,the detected fault need not be immediately removed,and it may lag the fault detection process by a delay effect factor.In this paper,we first review how different software reliability growth models have been developed,where fault detection process is dependent not only on the number of residual fault content but also on the testing time,and see how these models can be reinterpreted as the delayed fault detection model by using a delay effect factor.Based on the power function of the testing time concept,we propose four new SRGMs that assume the presence of two types of faults in the software:leading and dependent faults.Leading faults are those that can be removed upon a failure being observed.However,dependent faults are masked by leading faults and can only be removed after the corresponding leading fault has been removed with a debugging time lag.These models have been tested on real software error data to show its goodness of fit,predictive validity and applicability.     

Robust feedforward and recurrent neural network based dynamic weighted combination models for software reliability prediction

Traditional parametric software reliability growth models (SRGMs) are based on some assumptions or distributions and none such single model can produce accurate prediction results in all circumstances. Non-parametric models like the artificial neural network (ANN) based models can predict software reliability based on only fault history data without any assumptions. In this paper, initially we propose a robust feedforward neural network (FFNN) based dynamic weighted combination model (PFFNNDWCM) for software reliability prediction. Four well-known traditional SRGMs are combined based on the dynamically evaluated weights determined by the learning algorithm of the proposed FFNN. Based on this proposed FFNN architecture, we also propose a robust recurrent neural network (RNN) based dynamic weighted combination model (PRNNDWCM) to predict the software reliability more justifiably. A real-coded genetic algorithm (GA) is proposed to train the ANNs. Predictability of the proposed models are compared with the existing ANN based software reliability models through three real software failure data sets. We also compare the performances of the proposed models with the models that can be developed by combining three or two of the four SRGMs. Comparative studies demonstrate that the PFFNNDWCM and PRNNDWCM present fairly accurate fitting and predictive capability than the other existing ANN based models. Numerical and graphical explanations show that PRNNDWCM is promising for software reliability prediction since its fitting and prediction error is much less relative to the PFFNNDWCM.     

Better reliability assessment and prediction through data clustering

This paper presents a new approach to software reliability modeling by grouping data into clusters of homogeneous failure intensities. This series of data clusters associated with different time segments can be directly used as a piecewise linear model for reliability assessment and problem identification, which can produce meaningful results early in the testing process. The dual model fits traditional software reliability growth models (SRGMs) to these grouped data to provide long-term reliability assessments and predictions. These models were evaluated in the testing of two large software systems from IBM. Compared with existing SRGMs fitted to raw data, our models are generally more stable over time and produce more consistent and accurate reliability assessments and predictions.     

Optimal testing resource allocation during module testing considering cost, testing effort and reliability

Software reliability is one of the most important quality attributes of commercial software. During software testing, software reliability growth models (SRGMs) are commonly used to describe the phenomenon of failure occurrence and/or fault removal which consequently enhancements software reliability. Large software systems are developed by integrating a number of relatively small and independent modules, which are tested independently during module testing phase. The amount of testing resource available is limited which is desired to be consumed judiciously so as to optimize the testing process. In this paper we formulate a resource allocation problem of minimizing the cost of software testing under available amount of testing resource, given a reliability constraint. We use a flexible SRGM considering testing effort which, depending upon the values of parameters, can describe either exponential or S-shaped failure pattern of software modules. A systematic and sequential Algorithm is proposed to solve the optimization problem formulated. Numerical examples are given to illustrate the formulation and solution procedures. Sensitivity analysis is performed to examine the behavior of some parameters of SRGM with most significant influence.     

Generalized Discrete Software Reliability Modeling With Effect of Program Size

Generalized methods for software reliability growth modeling have been proposed so far. But, most of them are on continuous-time software reliability growth modeling. Many discrete software reliability growth models (SRGM) have been proposed to describe a software reliability growth process depending on discrete testing time such as the number of days (or weeks); the number of executed test cases. In this paper, we discuss generalized discrete software reliability growth modeling in which the software failure-occurrence times follow a discrete probability distribution. Our generalized discrete SRGMs enable us to assess software reliability in consideration of the effect of the program size, which is one of the influential factors related to the software reliability growth process. Specifically, we develop discrete SRGMs in which the software failure-occurrence times follow geometric and discrete Rayleigh distributions, respectively. Moreover, we derive software reliability assessment measures based on a unified framework for discrete software reliability growth modeling. Additionally, we also discuss optimal software release problems based on our generalized discrete software reliability growth modeling. Finally, we show numerical examples of software reliability assessment by using actual fault-counting data     

考虑测试环境和实际运行环境的软件可靠性增长模型

软件可靠性增长模型中测试阶段和操作运行阶段环境的不同导致了两个阶段故障检测率的不同.非齐次泊松过程类软件可靠性增长模型是评价软件产品可靠性指标的有效工具.在一些非齐次泊松过程类模型中,有些学者提出了常量的环境因子,用来描述测试环境和运行环境的差别.实际上,环境因子应该是随时间变化的变量.考虑了运行阶段和测试阶段环境的不同,根据实测数据得到了变化的环境因子,并且根据测试阶段的故障检测率和变化的环境因子,转化得到了操作运行阶段的故障检测率.考虑到故障的排除效率和故障引入率,从而建立了一个既考虑运行环境和测试环境差别,又考虑故障排除效率和故障引入率的非齐次泊松过程类软件可靠性增长模型（PTEO-SRGM）.在两组失效数据上的实验分析表明,对这组失效数据,PTEO-SRGM模型比G-O模型等模型的拟合效果和预测能力更好.     

考虑故障相关的软件可靠性增长模型研究

软件可靠性增长模型是用来评估和预测软件可靠性的重要工具.目前,绝大多数的软件可靠性增长模型并没有考虑故障之间的相关性,也没有考虑测试环境和运行环境的区别.文中提出了一种随机过程类非齐次泊松过程(NHPP)中的考虑故障相关性、测试环境和运行环境差别的模型.在两组失效数据上的实验分析表明:对这两组失效数据,文中提出的模型比其他一些非齐次泊松过程类模型的拟合效果和预测效果更好.     

一个基于响铃形故障检测率函数的软件可靠性增长模型

非齐次泊松过程类软件可靠性增长模型是评价软件产品可靠性指标的有效工具．影响软件可靠性增长模型评估和预测准确性的最重要的两个因素是软件中隐藏的初始故障数和故障检测率．一些非齐次泊松过程类模型假设故障检测率是不随测试时间变化的常量,有些模型假设故障检测率是增函数或减函数．这些假设或忽略了测试者的学习过程,或忽略了越迟被检测到的故障的概率就可能越低的特点．该文将测试者的学习过程和软件固有故障检测率的变化特征相结合,提出了一个铃形的故障检测率函数,建立了一个非齐次泊松过程类软件可靠性增长模型——Bbell—SRGM．在一组失效数据上的实验分析表明：对这组失效数据,Bbell—SRGM模型比G-O模型等的拟合效果更好．     

考虑用户行为和排错延迟的软件运行可靠性增长模型

传统的软件可靠性模型大多都假设软件测试环境和运行环境相同,也就是使用软件测试阶段的失效数据来预测软件运行可靠性。众所周知,软件固有故障的排除能提高系统可靠性,然而另一种现象就是随着用户对软件熟悉程度的提高,软件的失效率也会降低。文中研究了软件固有故障检测过程、固有故障纠正过程和外在失效过程的特征,建立了考虑用户行为和排错延迟下的软件运行可靠性增长模型。通过一组来自于开源软件用户缺陷跟踪系统中的真实数据进行数值分析,实验结果表明提出的模型具有较好的效果。     

A replicated empirical study of a selection method for software reliability growth models

Replications are commonly considered to be important contributions to investigate the generality of empirical studies. By replicating an original study it may be shown that the results are either valid or invalid in another context, outside the specific environment in which the original study was launched. The results of the replicated study show how much confidence we could possibly have in the original study. We present a replication of a method for selecting software reliability growth models to decide whether to stop testing and release software. We applied the selection method in an empirical study, conducted in a different development environment than the original study. The results of the replication study show that with the changed values of stability and curve fit, the selection method works well on the empirical system test data available, i.e., the method was applicable in an environment that was different from the original one. The application of the SRGMs to failures during functional testing resulted in predictions with low relative error, thus providing a useful approach in giving good estimates of the total number of failures to expect during functional testing.

Cost–reliability optimum release policies for a software system under penalty cost

Optimum software release policies are considered, minimizing the expected software cost simultaneously with the reliability requirement. Cost here also includes the penalty cost which is incurred by the manufacturer for not delivering the software at scheduled delivery time. The underlying software reliability growth models (SRGMs) are based on the non-homogeneous Poisson process (NHPP). Numerical results are also presented.     

Nonparametric dynamically weighted combination model to determine when to stop testing

Software manufacturers need to minimize the number of their software failures in their production environments. So, software reliability becomes a critical factor for these manufacturers to focus on. Software Reliability Growth Models (SRGMs) are used as indicators of the number of failures that may be faced after the shipping of the software and thus are indicators of the readiness of the software for shipping. SRGMs to handle varying operational profiles have been proposed by researchers earlier. However, as it is difficult to predict the nature of the project in advance, the reliability engineer has to try out each model one at a time before zeroing in on the model to be used in the project. We have derived a combination model, called dynamically weighted infinite NHPP combination, using the existing models for determining the release time. The nonparametric dynamically weighted combination model that we propose was validated and was found to be effective.

     

Software Reliability Assessment Using Hybrid Neuro-Fuzzy Model

Software reliability is the primary concern of software development organizations, and the exponentially increasing demand for reliable software requires modeling techniques to be developed in the present era. Small unnoticeable drifts in the software can culminate into a disaster. Early removal of these errors helps the organization improve and enhance the software’s reliability and save money, time, and effort. Many soft computing techniques are available to get solutions for critical problems but selecting the appropriate technique is a big challenge. This paper proposed an efficient algorithm that can be used for the prediction of software reliability. The proposed algorithm is implemented using a hybrid approach named Neuro-Fuzzy Inference System and has also been applied to test data. In this work, a comparison among different techniques of soft computing has been performed. After testing and training the real time data with the reliability prediction in terms of mean relative error and mean absolute relative error as 0.0060 and 0.0121, respectively, the claim has been verified. The results claim that the proposed algorithm predicts attractive outcomes in terms of mean absolute relative error plus mean relative error compared to the other existing models that justify the reliability prediction of the proposed model. Thus, this novel technique intends to make this model as simple as possible to improve the software reliability.