Comparing cost prediction models by resampling techniques

The accurate software cost prediction is a research topic that has attracted much of the interest of the software engineering community during the latest decades. A large part of the research efforts involves the development of statistical models based on historical data. Since there are a lot of models that can be fitted to certain data, a crucial issue is the selection of the most efficient prediction model. Most often this selection is based on comparisons of various accuracy measures that are functions of the model’s relative errors. However, the usual practice is to consider as the most accurate prediction model the one providing the best accuracy measure without testing if this superiority is in fact statistically significant. This policy can lead to unstable and erroneous conclusions since a small change in the data is able to turn over the best model selection. On the other hand, the accuracy measures used in practice are statistics with unknown probability distributions, making the testing of any hypothesis, by the traditional parametric methods, problematic. In this paper, the use of statistical simulation tools is proposed in order to test the significance of the difference between the accuracy of two prediction methods: regression and estimation by analogy. The statistical simulation procedures involve permutation tests and bootstrap techniques for the construction of confidence intervals for the difference of measures. Four known datasets are used for experimentation in order to validate the results and make comparisons between the simulation methods and the traditional parametric and non-parametric procedures.     

Using an Approximate Bayesian Bootstrap to multiply impute nonignorable missing data

An Approximate Bayesian Bootstrap (ABB) offers advantages in incorporating appropriate uncertainty when imputing missing data, but most implementations of the ABB have lacked the ability to handle nonignorable missing data where the probability of missingness depends on unobserved values. This paper outlines a strategy for using an ABB to multiply impute nonignorable missing data. The method allows the user to draw inferences and perform sensitivity analyses when the missing data mechanism cannot automatically be assumed to be ignorable. Results from imputing missing values in a longitudinal depression treatment trial as well as a simulation study are presented to demonstrate the method’s performance. We show that a procedure that uses a different type of ABB for each imputed data set accounts for appropriate uncertainty and provides nominal coverage.     

Using an Approximate Bayesian Bootstrap to multiply impute nonignorable missing data

An Approximate Bayesian Bootstrap (ABB) offers advantages in incorporating appropriate uncertainty when imputing missing data, but most implementations of the ABB have lacked the ability to handle nonignorable missing data where the probability of missingness depends on unobserved values. This paper outlines a strategy for using an ABB to multiply impute nonignorable missing data. The method allows the user to draw inferences and perform sensitivity analyses when the missing data mechanism cannot automatically be assumed to be ignorable. Results from imputing missing values in a longitudinal depression treatment trial as well as a simulation study are presented to demonstrate the method’s performance. We show that a procedure that uses a different type of ABB for each imputed data set accounts for appropriate uncertainty and provides nominal coverage.     

Optimising an observational water monitoring network for Archipelago Sea, South West Finland

Water quality monitoring in topographically fragmented archipelago coasts calls for a dense observational network. However, visiting multiple sites and analyzing the samples requires a significant amount of work, leading to considerable economic cost. It is of interest to determine an efficient set of sites, which still offers adequate information on the water quality with a sufficient spatial accuracy. A method for optimizing an existing observational network is proposed. The method is concretized by applying it for an observational network in the Archipelago Sea, South West Finland. The network is pruned with the requirement that the observations of the removed sites can be estimated using those of the remaining sites. Suboptimal heuristics are used in pruning to keep the computational time acceptable. Some observations are not available and need to be estimated (imputed) before the pruning. For the network in the Archipelago Sea, the results of the pruning are somewhat sensitive to differences in imputed datasets and heuristics used for site selection.     

基于偶然正确性概率的回归测试选择方法

数据驱动的智能系统的核心是处理数据的算法,对算法正确性的要求高,导致其测试开销大,需要有效地缩减测试的规模,其中回归测试选择是控制测试规模的有效手段.数据驱动的智能系统由于其动态信息流强度弱的原因,发生偶然正确性现象的概率较高,并且该现象会导致常用的回归测试选择技术所选择出的测试集包含大量检测不到故障的测试用例.因此,...     

Exact optimal inference in regression models under heteroskedasticity and non-normality of unknown form

Simple point-optimal sign-based tests are developed for inference on linear and nonlinear regression models with non-Gaussian heteroskedastic errors. The tests are exact, distribution-free, robust to heteroskedasticity of unknown form, and may be inverted to build confidence regions for the parameters of the regression function. Since point-optimal sign tests depend on the alternative hypothesis considered, an adaptive approach based on a split-sample technique is proposed in order to choose an alternative that brings power close to the power envelope. The performance of the proposed quasi-point-optimal sign tests with respect to size and power is assessed in a Monte Carlo study. The power of quasi-point-optimal sign tests is typically close to the power envelope, when approximately 10% of the sample is used to estimate the alternative and the remaining sample to compute the test statistic. Further, the proposed procedures perform much better than common least-squares-based tests which are supposed to be robust against heteroskedasticity.     

一种无参数的微阵列缺失值填补方法

微阵列数据中的缺失值会对随后的数据分析造成影响。因此,正确地估计这些缺失值是很必要的。将一个k值选取算法结合到有序的局部最小二乘填补算法中,提出了一种无参数的缺失值填补方法(SLLSkimpute)。该方法的三个特点是:第一,无需事先确定参数;第二,针对不同的目标基因使用不同数目的邻居基因;第三,有序地估计缺失值,并有选择地将已得到的估计值应用到后续的估计过程中。实验结果证实了该算法的有效性,其估计性能优于其它一些常用的填补方法。     

Statistical analysis of water-quality data containing multiple detection limits: S-language software for regression on order statistics

Trace contaminants in water, including metals and organics, often are measured at sufficiently low concentrations to be reported only as values below the instrument detection limit. Interpretation of these “less thans” is complicated when multiple detection limits occur. Statistical methods for multiply censored, or multiple-detection limit, datasets have been developed for medical and industrial statistics, and can be employed to estimate summary statistics or model the distributions of trace-level environmental data.We describe S-language-based software tools that perform robust linear regression on order statistics (ROS). The ROS method has been evaluated as one of the most reliable procedures for developing summary statistics of multiply censored data. It is applicable to any dataset that has 0 to 80% of its values censored. These tools are a part of a software library, or add-on package, for the R environment for statistical computing. This library can be used to generate ROS models and associated summary statistics, plot modeled distributions, and predict exceedance probabilities of water-quality standards.     

AESOP: an architectural relational database

This paper describes the principles and implementation of AESOP, an architectural relational database which aims to integrate building design procedures and to provide good graphic manipulation facilities. It is also intended to be highly flexible in that it allows inexact as well as exact data, can support a variety of user views of data and modes of use, and can accommodate itself to new techniques as these become available. AESOP uses ‘fuzzy’ logic. It holds information in the form of components which are linked lists of records, or relations, which are linked lists of tuples containing pointers to records, or, more generally, as directed graphs with records at the boundaries. New relations can be defined interactively from old using set theoretic operations thus enabling the logical operations of the first order predicate calculus to be performed on relations. The database also holds internal procedures called maps. Maps may be used to transform data as required by the user or to test the properties of a design proposal. New maps can be defined interactively from existing maps by applying composition, recursion or the logical operations of the first order calculus. An example of the use of the principles of fuzzy logic for guiding the selection of building fabri? is given. Some tests of AESOP in practice on live problems are noted. It is concluded that AESOP could provide a useful framework for a CAAD system and that further work is required in two areas, the user interface, and operations involving geometrical relations.     

Path testing of computer programs with loops using a tool for simple loop patterns

The purpose of this paper is to present a method for testing computer programs with iteration loops. Given such programs, we have shown that for classes of program paths, identified as sequences of simple loop paths, there is a characterizing function called a simple loop pattern. The key idea of simple loop patterns is that these special functions form a base set which can represent any path computation in the given program. A software tool called SILOP has been developed to automatically generate these simple loop patterns, and each corresponding sequence of simple loop paths can be considered as a test case. The tester uses each test case, and with knowledge of the application program, can generate corresponding test data. This paper also presents a method for selecting the specific paths and test data to determine the simple loop pattern reliably. The tester can use this selection method to predict the number of tests required. In order to apply this selection method, the given program must be a linear computer program. The SILOP tool and this test selection method have been applied to commercial software; in this paper, this computational experience is reported and several examples are given to demonstrate the approach.     

心理运动能力测评系统的开发应用

飞行事故的发生,有一部分原因是因为飞行员自身不适合飞行活动。为了减少这种原因导致的飞行事故发生的几率,同时为了避免民航公司因为飞行学员高强度训练而被淘汰造成的培训费用浪费,需要对飞行员候选者开展相关的心理选拔,考察他们是否具备合格飞行员所需的各种心理品质。目前,我国民航飞行员心理选拔大多沿用传统的纸笔测验,对应聘者进行人格和基本能力测验,对飞行特殊能力的考察和检测不够重视。文中研究的主要目的是为了实现民航飞行员心理选拔的仪器检测化,以达到选拔过程的客观化和标准化。本单位自行开发了一套计算机化的心理运动能力检测系统,并将该软件用于民航飞行员招聘中,并对结果进行统计分析,通过对现役飞行员和应聘者之间内容效度的比较,发现其成绩之间有较大差异,表明该软件对民航飞行员心理运动能力选拔具有实际应用价值。该测试软件操作简单方便,界面友好,结果准确可靠。     

REGRES: A FORTRAN-77 program to calculate nonparametric and “structural” parametric solutions to bivariate regression equations

NPSTAT compares the location (median), dispersion (variance), and overall shape of 2 or more groups of data (samples), using the Kruskal-Wallis and Van der Waerden tests (> 2 groups), Mann-Whitney and Kolmogorov-Smirnov tests (2 groups), and squared ranks test (2 or more groups). Exact or approximate significance levels of test statistics are calculated in all situations. Multivariate data are treated one variable at a time. NPSTAT reproduces results from textbooks despite previous inconsistencies with calculation and significance assessment methods. Commercial routines (e.g. in MINITAB, NAG) cover only 3 of the 5 tests programmed, but yield consistent results for these. NPSTAT is useful particularly for comparing nonnormally distributed data, multivariate data with missing values, and data measured only on an ordinal scale. It also can be used to assess outlying values.     

Regression testing minimization,selection and prioritization: a survey

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.     

顺序敏感的多源感知数据填补技术

近年来,随着感知网络的广泛应用,感知数据呈爆炸式增长.但是由于受到硬件设备的固有限制、部署环境的随机性以及数据处理过程中的人为失误等多方面因素的影响,感知数据中通常包含大量的缺失值.而大多数现有的上层应用分析工具无法处理包含缺失值的数据集,因此对缺失数据进行填补是不可或缺的.目前也有很多缺失数据填补算法,但在缺失数据较为密集的情况下,已有算法的填补准确性很难保证,同时未考虑填补顺序对填补精度的影响.基于此,提出了一种面向多源感知数据且顺序敏感的缺失值填补框架OMSMVI（order-sensitive missing value imputation framework for multi-source sensory data）.该框架充分利用感知数据特有的多维度相关性：时间相关性、空间相关性、属性相关性,对不同数据源间的相似度进行衡量;进而,基于多维度相似性构建以缺失数据源为中心的相似图,并将已填补的缺失值作为观测值用于后续填补过程中.同时考虑缺失数据源的整体分布,提出对缺失值进行顺序敏感的填补,即：首先对缺失值的填补顺序进行决策,再对缺失值进行填补.对缺失值进行顺序填补能够有效缓解在缺失数据较为密集的情况下,由于缺失数据源的完整近邻与其相似度较低引起的填补精度下降问题;最后,对KNN填补算法进行改进,提出一种新的基于近邻节点的缺失值填补算法NI（neighborhood-based imputation）,该算法利用感知数据的多维度相似性对缺失数据源的所有近邻节点进行查找,解决了KNN填补算法K值难以确定的问题,也进一步提高了填补准确性.利用两个真实数据集,并与基本填补算法进行对比,验证了算法的准确性及有效性.     

基于最大主子图分解的贝叶斯网络等价类学习算法

针对基于约束方法学习贝叶斯网络(BN)结构的不足,以及随着条件集的增大,利用统计方法进行条件独立(CI)测试不稳定等问题,提出一种基于最大主子图分解(MPD)的BN等价类学习算法.该算法首先通过MPD分解技术对BN的道德图进行分解;然后利用0阶和1阶CI测试识别部分子图中的V结构,对于初步未定的V结构利用局部评分搜索确定,从而避免了冗余检验,有效地减小了条件集的维数,并且提高了算法的效率.理论证明和实验结果均表明了所提出算法的有效性和合理性.     

The Construction of Smooth Models using Irregular Embeddings Determined by a Gamma Test Analysis

One of the key problems in forming a smooth model from input-output data is the determination of which input variables are relevant in predicting a given output. In this paper, we show how the Gamma test can be used to select that combination of input variables which can best be employed to form a smooth model of an output. For time series prediction this amounts to the selection of an appropriate irregular embedding. We give some simple zero noise examples of time series analysis, and illustrate how using these techniques a binary message encoded into a chaotic carrier can be retrieved without knowledge of the dynamics used to generate the carrier. Provided the underlying dynamics are such as to produce a smooth embedding model with bounded partial derivatives, the sampling distribution is dense in input space, and any associated distribution of measurement error has the first few moments bounded, so that the typical prerequisite conditions of the Gamma test are satisfied, we conclude that the Gamma test is an effective tool in the determination of irregular time series embeddings. These techniques can also be useful in practical applications which involve filtering seismic data to detect anomalous events.     

Neural-network construction and selection in nonlinear modeling

We study how statistical tools which are commonly used independently can advantageously be exploited together in order to improve neural network estimation and selection in nonlinear static modeling. The tools we consider are the analysis of the numerical conditioning of the neural network candidates, statistical hypothesis tests, and cross validation. We present and analyze each of these tools in order to justify at what stage of a construction and selection procedure they can be most useful. On the basis of this analysis, we then propose a novel and systematic construction and selection procedure for neural modeling. We finally illustrate its efficiency through large-scale simulations experiments and real-world modeling problems.     

一种结构测试数据自动生成的框架

针对结构测试中控制流和数据流覆盖测试数据的生成都可以归结为面向路径的测试数据生成的问题，提出了一个通用的基于控制流和数据流的结构测试数据自动生成的框架。该框架根据控制流和数据流测试中所采用的覆盖标准优化选取测试路径，并以改进后的迭代松弛法为核心，对所选取的路径生成测试数据。以基于路径覆盖、分支覆盖和数据流覆盖测试数据自动生成这3种算法为核心，开发了一个测试数据自动生成的框架原型。实验结果表明该框架是可行的。     

面向GPU的批LOD地形实时绘制

为提高大规模地形实时渲染时的绘制效率,提出一种使用地形分块作为处理单元的批LOD算法。在预处理阶段,将多分辨率的地形数据划分成适于GPU批处理的分块,使用四叉树进行分块的有效组织。在此基础上,提出一种基于分块绘制的LOD误差标准,简化层次选取的计算量,通过增加"裙"和进行几何变形实现了层次间的有效过渡;实时绘制过程中,使用视锥裁剪减少进入图形硬件的数据量,利用地形四叉树列表和预测机制实现地形数据的有效加载管理。实验结果表明,本文算法能够充分发挥图形硬件的性能,具有较高的地形实时渲染效率。     

系统级BIT设计中的测试选择方法

根据系统级BIT设计的要求,明确了系统级BIT设计中测试选择所需解决的问题,提出了测试选择的基本思想;在此基础上,利用系统级故障隔离的间接熵法实现了系统级BIT设计中的测试选择,并引入测试时间和故障发生频率参数对间接熵法进行了修正,获得了可以较快地判断系统是否可用并隔离大概率故障的测试集。理论分析及实验数据表明,该文提出的系统级BIT设计中测试选择方法是有效的。