Considering the fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time

Since the early 1970s tremendous growth has been seen in the research of software reliability growth modeling.In general, software reliability growth models (SRGMs) are applicable to the late stages of testing in software development and they can provide useful information about how to improve the reliability of software products.A number of SRGMs have been proposed in the literature to represent time-dependent fault identification/removal phenomenon;still new models are being proposed that could fit a greater number of reliability growth curves.Often,it is assumed that detected faults axe immediately corrected when mathematical models are developed.This assumption may not be realistic in practice because the time to remove a detected fault depends on the complexity of the fault,the skill and experience of the personnel,the size of the debugging team,the technique,and so on.Thus,the detected fault need not be immediately removed,and it may lag the fault detection process by a delay effect factor.In this paper,we first review how different software reliability growth models have been developed,where fault detection process is dependent not only on the number of residual fault content but also on the testing time,and see how these models can be reinterpreted as the delayed fault detection model by using a delay effect factor.Based on the power function of the testing time concept,we propose four new SRGMs that assume the presence of two types of faults in the software:leading and dependent faults.Leading faults are those that can be removed upon a failure being observed.However,dependent faults are masked by leading faults and can only be removed after the corresponding leading fault has been removed with a debugging time lag.These models have been tested on real software error data to show its goodness of fit,predictive validity and applicability.     

Software-Reliability Modeling: The Case for Deterministic Behavior

Software-reliability models (SRMs) are used for the assessment and improvement of reliability in software systems. These models are normally based on stochastic processes, with the nonhomogeneous Poisson process being one of the most prominent model forms. An underlying assumption of these models is that software failures occur randomly in time. This assumption has never been quantitatively tested. Our contribution in this paper is to conduct an experimental investigation that contrasts random processes with nonlinear deterministic processes as a model for software failures. We study two sets of real-world software-reliability data using the techniques of chaotic time-series analysis. We have found that both appear to arise from a deterministic process, rather than a stochastic process, and that both show some evidence of chaotic dynamics. In addition, we have conducted a series of k-steps-ahead forecasting experiments in the datasets, pitting a number of well-known stochastic SRMs against radial basis function networks (RBFNs), which are deterministic in nature. The out-of-sample prediction results from the RBFNs showed an improvement of roughly 25% over the best of the stochastic models, for both of our datasets. Finally, we propose a causal model to explain these results, which hypothesizes that faults in a program are distributed over a fractal subset of the program's input space     

A general modeling and analysis framework for software fault detection and correction process

Software reliability growth modeling plays an important role in software reliability evaluation. To incorporate more information and provide more accurate analysis, modeling software fault detection and correction processes has attracted widespread research attention recently. In modeling software correction processes, the assumption of fault correction time is relaxed from constant delay to random delay. However, stochastic distribution of fault correction time brings more difficulties in modeling and corresponding parameter estimation. In this paper, a framework of software reliability models containing both information from software fault detection process and correction process is studied. Different from previous extensions on software reliability growth modeling, the proposed approach is based on Markov model other than a nonhomogeneous Poisson process model. Also, parameter estimation is carried out with weighted least‐square estimation method, which emphasizes the influence of later data on the prediction. Two data sets from practical software development projects are applied with the proposed framework, which shows satisfactory performance with the results. Copyright © 2016 John Wiley & Sons, Ltd.     

一个考虑多种排错延迟的NHPP类软件可靠性增长模型

软件可靠性增长模型通常假设软件的测试环境与软件实际运行的现场环境相同,期望利用测试阶段获得的失效数据评估软件在现场运行时的失效行为。多数非齐次泊松过程类软件可靠性增长模型假设软件故障被发现后立即被排除,这点假设无论是在测试环境还是在现场环境下都很难实现。根据故障对测试过程的影响,故障的排错时间可被分为多种。提出了一个考虑多种排错延迟的软件可靠性增长模型,讨论了基于这个模型的故障排除效率函数,指出从用户角度出发讨论软件可靠性时必须考虑重复性故障。     

Robust feedforward and recurrent neural network based dynamic weighted combination models for software reliability prediction

Traditional parametric software reliability growth models (SRGMs) are based on some assumptions or distributions and none such single model can produce accurate prediction results in all circumstances. Non-parametric models like the artificial neural network (ANN) based models can predict software reliability based on only fault history data without any assumptions. In this paper, initially we propose a robust feedforward neural network (FFNN) based dynamic weighted combination model (PFFNNDWCM) for software reliability prediction. Four well-known traditional SRGMs are combined based on the dynamically evaluated weights determined by the learning algorithm of the proposed FFNN. Based on this proposed FFNN architecture, we also propose a robust recurrent neural network (RNN) based dynamic weighted combination model (PRNNDWCM) to predict the software reliability more justifiably. A real-coded genetic algorithm (GA) is proposed to train the ANNs. Predictability of the proposed models are compared with the existing ANN based software reliability models through three real software failure data sets. We also compare the performances of the proposed models with the models that can be developed by combining three or two of the four SRGMs. Comparative studies demonstrate that the PFFNNDWCM and PRNNDWCM present fairly accurate fitting and predictive capability than the other existing ANN based models. Numerical and graphical explanations show that PRNNDWCM is promising for software reliability prediction since its fitting and prediction error is much less relative to the PFFNNDWCM.     

Discrete software reliability assessment with discretized NHPP models

A software reliability growth model is one of the fundamental technique to assess software reliability quantitatively. The software reliability growth model is required to have a good performance in terms of goodness-of-fit, predictability, and so forth. In this paper, we propose discretized software reliability growth models. As to the software reliability growth modeling, discretized nonhomogeneous Poisson process models are investigated particularly for accurate software reliability assessment. We show that the discrete nonhomogeneous Poisson process models have better performance than discretized deterministic software reliability growth models which have been proposed so far.     

不完美排错软件可靠性增长模型效用量化研究

为了进一步提升现有非齐次泊松过程类软件可靠性增长模型的拟合和预测性能,首先从故障总数增长趋势角度对不完美排错模型进行深入研究,提出两个一般性不完美排错框架模型,分别考虑了总故障数量函数与累计检测故障函数间的线性关系与微分关系,并求得累计检测的故障数量与软件中总故障数量函数表达式;其次,在六组真实的失效数据集上对比了提出的两种一般性不完美排错模型和六种不完美排错模型拟合预测性能表现。实例验证结果表明,提出的一般性不完美排错框架模型在大多数失效数据集上都具有优秀的拟合和预测性能,证明了新建模型的有效性和实用性;通过对提出的模型与其他不完美排错模型在数据集上的性能的深入分析,为实际应用中不完美排错模型的选择提出了建议。     

Better reliability assessment and prediction through data clustering

This paper presents a new approach to software reliability modeling by grouping data into clusters of homogeneous failure intensities. This series of data clusters associated with different time segments can be directly used as a piecewise linear model for reliability assessment and problem identification, which can produce meaningful results early in the testing process. The dual model fits traditional software reliability growth models (SRGMs) to these grouped data to provide long-term reliability assessments and predictions. These models were evaluated in the testing of two large software systems from IBM. Compared with existing SRGMs fitted to raw data, our models are generally more stable over time and produce more consistent and accurate reliability assessments and predictions.     

一个NHPP类软件可靠性增长模型框架

NHPP类软件可靠性增长模型已经成为软件可靠性工程实践中非常成功的工具,从某些模型的一些共同特征出发,研究了NHPP类软件可靠性增长模型的有限通用框架,提出了一 个既考虑软件测试的不完美性、故障检测率随时间的变化,又考虑了故障改正效率随时间变化的NHPP类软件可靠性增长模型框架。一些已经存在的NHPP类软件可靠性增长模型型是这个框架的特例。     

A time/structure based software reliability model

The past 20 years have seen the formulation of numerous analytical software reliability models for estimating the reliability growth of a software product. The predictions obtained by applying these models tend to be optimistic due to the inaccuracies in the operational profile, and saturation effect of testing. Incorporating knowledge gained about some structural attribute of the code, such as test coverage, into the time-domain models can help alleviate this optimistic trend. In this paper we present an enhanced non-homogeneous Poisson process (ENHPP) model which incorporates explicitly the time-varying test-coverage function in its analytical formulation, and provides for defective fault detection and test coverage during the testing and operational phases. It also allows for a time varying fault detection rate. The ENHPP model offers a unifying framework for all the previously reported finite failure NHPP models via test coverage. We also propose the log-logistic coverage function which can capture an increasing/decreasing failure detection rate per fault, which cannot be accounted for by the previously reported finite failure NHPP models. We present a methodology based on the ENHPP model for reliability prediction earlier in the testing phase. Expressions for predictions in the operational phase of the software, software availability, and optimal software release times subject to various constraints such as cost, reliability, and availability are developed based on the ENHPP model. We also validate the ENHPP model based on four different coverage functions using five failure data sets. This revised version was published online in June 2006 with corrections to the Cover Date.     

软件可靠性混沌模型

在分析软件失效机理后认为：有些软件失效行为具有混沌性,所以可以用混沌方法来处理其软件可靠性推断问题。但在应用混沌方法前先要进行系统辨识,确定为混沌系统后,才能应用嵌入空间技术从软件失效时间序列重建系统相空间和吸引子,进而用吸引子的揭示的混沌属性来估计软件可靠性。文中在三个标准数据集的基础上对此进行了实证分析,结果表明其中两个数据来源于混沌机制,他们的吸引子具有低维的小数极限维数,而且预测与实际可靠性吻合较好。值得指出的是文中所提混沌方法突破了软件可靠性一贯使用随机分析的局限。     

安全苛求软件的安全性混沌分析

对软件安全性的研究大多基于概率的或随机过程的软件可靠性理论,但是首先安全性并不等同于可靠性,再则可靠性概念在21世纪随科技的发展也在演化。在研究安全苛求软件及其失效的特征的基础上,使用混沌的方法研究其安全性具有合理性。采用嵌入空间的技术可从时间序列中重构出具有系统特征的相平面和吸引子,由此可以预测危险。铁路联锁软件是典型的安全苛求软件,安全性的混沌分析将有助于实施高效的铁路联锁软件的现场测试。     

Analysis of Software Fault Removal Policies Using a Non-Homogeneous Continuous Time Markov Chain

Software reliability is an important metric that quantifies the quality of a software product and is inversely related to the residual number of faults in the system. Fault removal is a critical process in achieving desired level of quality before software deployment in the field. Conventional software reliability models assume that the time to remove a fault is negligible and that the fault removal process is perfect. In this paper we examine various kinds of fault removal policies, and analyze their effect on the residual number of faults at the end of the testing process, using a non-homogeneous continuous time Markov chain. The fault removal rate is initially assumed to be constant, and it is subsequently extended to cover time and state dependencies. We then extend the non-homogeneous continuous time Markov chain (NHCTMC) framework to include imperfections in the fault removal process. A method to compute the failure intensity of the software in the presence of explicit fault removal is also proposed. The fault removal scenarios can be easily incorporated using the state-space view of the non-homogeneous Poisson process.     

考虑排错过程引进故障的开源软件可靠性模型研究

近年来,开源软件在软件行业很受欢迎。但是,开源软件的可靠性却受到人们的广泛质疑。如何评估开源软件的可靠性是一个重要的问题。与传统的闭源软件相比,在建立开源软件可靠性模型时,必须考虑故障引入和故障检测与排错之间的延迟时间这两个因素。本文考虑了排错过程和不完美调试现象,提出了相应的开源软件可靠性模型。并且我们用两个开源软件故障数据集实来验证提出模型的拟合性能与预测性能。实验结果表明,提出的模型在开源软件可靠性评估中具有良好的拟合和预测性能。提出的模型可以用于开源软件在实际的开发过程中的可靠性评估。     

Generalized Discrete Software Reliability Modeling With Effect of Program Size

Generalized methods for software reliability growth modeling have been proposed so far. But, most of them are on continuous-time software reliability growth modeling. Many discrete software reliability growth models (SRGM) have been proposed to describe a software reliability growth process depending on discrete testing time such as the number of days (or weeks); the number of executed test cases. In this paper, we discuss generalized discrete software reliability growth modeling in which the software failure-occurrence times follow a discrete probability distribution. Our generalized discrete SRGMs enable us to assess software reliability in consideration of the effect of the program size, which is one of the influential factors related to the software reliability growth process. Specifically, we develop discrete SRGMs in which the software failure-occurrence times follow geometric and discrete Rayleigh distributions, respectively. Moreover, we derive software reliability assessment measures based on a unified framework for discrete software reliability growth modeling. Additionally, we also discuss optimal software release problems based on our generalized discrete software reliability growth modeling. Finally, we show numerical examples of software reliability assessment by using actual fault-counting data     

Applying the Hybrid Model of EMD,PSR, and ELM to Exchange Rates Forecasting

Financial time series forecasting has been a challenge for time series analysts and researchers because it is noisy, nonstationary and chaotic. To overcome this limitation, this study uses empirical mode decomposition (EMD) and phase space reconstruction (PSR) to assist in the task of financial time series forecasting. In addition, we propose an approach that combines these two data preprocessing methods with extreme learning machine (ELM). The approach contains four steps as follows. (1) EMD is used to decompose the dynamics of the exchange rate time series into several components of intrinsic mode function (IMF) and one residual component. (2) The IMF and residual time series phase space is reconstructed to reveal its unseen dynamics according to the optimum time delay \(\tau \) and embedding dimension m. (3) The reconstructed time series datasets are divided into two datasets: training and testing, in which the training datasets are used to build ELM models. (4) A regression forecast model is set up for each IMF as well as the residual component by using ELM. The final prediction results are obtained by compositing the prediction values. To verify the effectiveness of the proposed approach, four exchange rates are chosen as the forecasting targets. Compared with some existing state-of-the-art models, the proposed approach yields superior results. Academically, we demonstrated the validity and superiority of the proposed approach that integrates EMD, PSR, and ELM. Corporations or individuals can apply the results of this study to acquire accurate exchange rate information and reduce exchange rate expenses.     

软件可靠性组合预测模型研究

根据灰色模型、谐波分析和时间序列分析理论,对软件测试阶段的失效数据构成的时间序列进行分析,得到软件可靠性组合预测模型。结合实际数据,给出了具体的实现方法。数据试验结果表明,与单一时间序列预测模型相比,该模型具有较高的预测精度和很好的模型适应性。     

基于排错等待延迟的广义动态集成神经网络模型

软件可靠性增长模型在可靠性评估与保障中具有重要作用,针对软件测试过程中的故障检测和排错等待延迟问题,提出了一种考虑故障排错等待延迟的广义动态集成神经网络模型(RWD-SRGM)。该模型考虑软件工程的多样性,利用神经网络方法构建广义动态集成模型,并考虑排错等待延迟现象完成故障检测和预测。通过2组真实失效数据集(DS1和DS2)的实验,将所提模型与现有的软件可靠性增长模型进行了比较,结果显示考虑故障排错等待延迟的神经网络模型拟合效果最优,表现出了更好的软件可靠性评估性能和模型通用性。     

Does software reliability growth behavior follow a non-homogeneous Poisson process

It is widely believed in software reliability community that software reliability growth behavior follows a non-homogeneous Poisson process (NHPP) based on analyzing the behavior of the mean of the cumulative number of observed software failures. In this paper we present two controlled software experiments to examine this belief. The behavior of the mean of the cumulative number of observed software failures and that of the corresponding variance are examined simultaneously. Both empirical observations and statistical hypothesis testing suggest that software reliability behavior does not follow a non-homogeneous Poisson process in general, and does not fit the Goel–Okumoto NHPP model in particular. Although this new finding should be further tested on other software experiments, it is reasonable to cast doubt on the validity of the NHPP framework for software reliability modeling. The importance of the work presented in this paper is not only for the new finding which is distinctly different from existing popular belief of software reliability modeling, but also for the adopted research approach which is to examine the behavior of the mean and that of the corresponding variance simultaneously on basis of controlled software experiments.