A framework for specification-based testing

Test templates and a test template framework are introduced as useful concepts in specification-based testing. The framework can be defined using any model-based specification notation and used to derive tests from model-based specifications-in this paper, it is demonstrated using the Z notation. The framework formally defines test data sets and their relation to the operations in a specification and to other test data sets, providing structure to the testing process. Flexibility is preserved, so that many testing strategies can be used. Important application areas of the framework are discussed, including refinement of test data, regression testing, and test oracles     

A framework for Good Enough testing

After a year in Sunnyvale, California, doing test management, the author is moving to Virginia, where he's been hired by Reliable Software Technologies, a company known for tools and consulting dedicated to achieving very high quality software. He'll spend most of his time in a consulting role, which is a very different reality from software management. But it will be an interesting one, nonetheless, especially since he's never worked in an environment that puts reliability ahead of marketability, or even abreast of it. He's known for his Good Enough quality model, so it may seem strange that RST recruited him. But as it turns out, RST believes that a very high reliability standard is-for them-just part of good enough quality. The same thinking about quality applies no matter where one places oneself on the quality scale: at every point in the life cycle, one must compare the present quality of the product against the cost and value of further improvement     

A framework for testing DBMS features

Testing a specific feature of a DBMS requires controlling the inputs and outputs of the operators in the query execution plan. However, that is practically difficult to achieve because the inputs/outputs of a query depend on the content of the test database. In this paper, we propose a framework to test DBMS features. The framework includes a database generator called QAGen so that the generated test databases are able to meet the test requirements defined on the test queries. The framework also includes a set of tools to automate test case constructions and test executions. A wide range of DBMS feature testing tasks can be facilitated by the proposed framework.     

A testing framework for mobile computing software

We present a framework for testing applications for mobile computing devices. When a device is moved into and attached to a new network, the proper functioning of applications running on the device often depends on the resources and services provided locally in the current network. This framework provides an application-level emulator for mobile computing devices to solve this problem. Since the emulator is constructed as a mobile agent, it can carry applications across networks on behalf of its target device and allow the applications to connect to local servers in its current network in the same way as if they had been moved with and executed on the device itself. This paper also demonstrates the utility of this framework by describing the development of typical network-dependent applications in mobile and ubiquitous computing settings.     

A property-based testing framework for encryption programs

In recent years, a variety of encryption algorithms were proposed to enhance the security of software and systems. Validating whether encryption algorithms are correctly implemented is a challenging issue. Software testing delivers an effective and practical solution, but it also faces the oracle problem (that is, under many practical situations, it is impossible or too computationally expensive to know whether the output for any given input is correct). In this paper, we propose a property-based approach to testing encryption programs in the absence of oracles. Our approach makes use of the so-called metamorphic properties of encryption algorithms to generate test cases and verify test results. Two case studies were conducted to illustrate the proposed approach and validate its effectiveness. Experimental results show that even without oracles, the proposed approach can detect nearly 50% inserted faults with at most three metamorphic relations (MRs) and fifty test cases.     

基于XML描述的构件测试框架

针对构件技术的特点,提出并采用XML描述的方式实现构件自动测试.介绍了构件、XML、XML Schema的特性,并在分析了当前构件测试方法和所面临问题的基础上,对构件的测试技术和测试框架进行了深入研究,提出一个基于XML描述的构件测试框架.详细介绍了采用该方法设计的构件测试框架的体系结构、各单元之间的数据流和控制流关系以及各单元的功能和功能实现所采用的部分技术.工程实践应用表明了此框架的可行性及其特性.     

A framework for table driven testing of Java classes

With the advent of object‐oriented languages and the portability of Java, the development and use of class libraries has become widespread. Effective class reuse depends on class reliability which in turn depends on thorough testing. This paper describes a class testing approach based on modeling each test case with a tuple and then generating large numbers of tuples to thoroughly cover an input space with many interesting combinations of values. The testing approach is supported by the Roast framework for the testing of Java classes. Roast provides automated tuple generation based on boundary values, unit operations that support driver standardization, and test case templates used for code generation. Roast produces thorough, compact test drivers with low development and maintenance cost. The framework and tool support are illustrated on a number of non‐trivial classes, including a graphical user interface policy manager. Quantitative results are presented to substantiate the practicality and effectiveness of the approach. Copyright © 2002 John Wiley & Sons, Ltd.     

A framework for testing first-order logic axioms in program verification

Program verification systems based on automated theorem provers rely on user-provided axioms in order to verify domain-specific properties of code. However, formulating axioms correctly (that is, formalizing properties of an intended mathematical interpretation) is non-trivial in practice, and avoiding or even detecting unsoundness can sometimes be difficult to achieve. Moreover, speculating soundness of axioms based on the output of the provers themselves is not easy since they do not typically give counterexamples. We adopt the idea of model-based testing to aid axiom authors in discovering errors in axiomatizations. To test the validity of axioms, users define a computational model of the axiomatized logic by giving interpretations to the function symbols and constants in a simple declarative programming language. We have developed an axiom testing framework that helps automate model definition and test generation using off-the-shelf tools for meta-programming, property-based random testing, and constraint solving. We have experimented with our tool to test the axioms used in Auto-Cert, a program verification system that has been applied to verify aerospace flight code using a first-order axiomatization of navigational concepts, and were able to find counterexamples for a number of axioms.     

A generic interoperability testing framework and a systematic development process for automated interoperability testing

Interoperability is a prerequisite to allow users to access systems implemented by different vendors seamlessly. A good baseline to achieve interoperability is the implementation of a common set of standards. However, this is often not sufficient as different implementations of a standard are not necessarily interoperable. Therefore, different implementations of systems need to be assessed for interoperability by applying interoperability testing. In this article, we present a generic framework that enables automated interoperability testing with message checks, which assess the compliance of messages exchanged between systems. The goal of this framework is the provision of a basic functionality of interoperability test entities, the definition of a generic interoperability test environment, and guidelines for the specification of automated interoperability tests. The framework also considers aspects related to interoperability testing including verdicts, automation, and limitations of the system under test. Through the application of the framework, interoperability of systems can be assessed, systems can be validated, and standards can be improved. In addition, we present a systematic development process for automated interoperability tests to formalize the development and specification of an interoperability test system. We also consider aspects and critical issues, which are important for the development of a complete interoperability test system. The framework and the process are language and system technology independent. We present their application in a case study that includes interoperability tests for the Internet Protocol Multimedia Subsystem (IMS) using the Testing and Test Control Notation Version 3 (TTCN-3).     

CP2: Cryptographic privacy protection framework for online social networks

In Online Social Networks (OSNs), users interact with each other by sharing their personal information. One of the concerns in OSNs is how user privacy is protected since the OSN providers have full control over users’ data. The OSN providers typically store users’ information permanently; the privacy controls embedded in OSNs offer few options to users for customizing and managing the dissipation of their data over the network. In this paper, we propose an efficient privacy protection framework for OSNs that can be used to protect the privacy of users’ data and their online social relationships from third parties. The recommended framework shifts the control over data sharing back to the users by providing them with flexible and dynamic access policies. We employ a public-key broadcast encryption scheme as the cryptographic tool for managing information sharing with a subset of a user’s friends. The privacy and complexity evaluations show the superiority of our approach over previous.     

A martingale framework for detecting changes in data streams by testing exchangeability

In a data streaming setting, data points are observed sequentially. The data generating model may change as the data are streaming. In this paper, we propose detecting this change in data streams by testing the exchangeability property of the observed data. Our martingale approach is an efficient, nonparametric, one-pass algorithm that is effective on the classification, cluster, and regression data generating models. Experimental results show the feasibility and effectiveness of the martingale methodology in detecting changes in the data generating model for time-varying data streams. Moreover, we also show that: 1) An adaptive support vector machine (SVM) utilizing the martingale methodology compares favorably against an adaptive SVM utilizing a sliding window, and 2) a multiple martingale video-shot change detector compares favorably against standard shot-change detection algorithms.     

A multi-criteria decision framework to support measurement-system design for bridge load testing

Due to conservative design models and safe construction practices, infrastructure usually has unknown amounts of reserve capacity that exceed code requirements. Quantification of this reserve capacity has the potential to lead to better asset-management decisions by avoiding unnecessary replacement and by lowering maintenance expenses. However, such quantification is challenging due to systematic uncertainties that are present in typical structural models. Field measurements, collected during load tests, combined with good structural-identification methodologies may improve the accuracy of model predictions. In most structural-identification tasks, engineers usually select and place sensors based on experience and high signal-to-noise estimations. Since the success of structural identification depends on the measurement system, research into measurement system design has been carried out over several decades. Despite the multi-criteria nature of the problem, most researchers have focused only on the information gained by the measurement system. This study presents a framework to evaluate and rank possible measurement-system designs based on a tiered multi-criteria strategy. Performance criteria for the design of measurement systems include monitoring costs, information gain, ability to detect outliers and impact of loss of information in case of sensor failure. Through including conflicting criteria, such as cost of monitoring and information gain, the optimal measuring system becomes a Pareto-like choice that ultimately depends on asset-manager preference hierarchies. Several potential preference scenarios are generated and results are compared using a full-scale test study, the Exeter Bascule Bridge. The framework successfully supports an informed design of measurement systems by providing an extensive set of alternatives, including the best solution defined probabilistically and for specific conditions when other near-optimal solutions might be preferred.     

FAST: a framework for automating statistics-based testing

To achieve software quality, testing is an essential component in all software development. It involves the execution of a deterministic software system with test data and a comparison of the results with the expected output, which must satisfy the users' requirements. This accounts for over 25% of the cost of a software development. Therefore, automation has considerable potential. The quality programming introduced by Cho can automatically generate data for testing, based on a so-called SIAD tree which is used to represent the hierarchical and network relation between input elements and also incorporates rules into the tree for using the inputs. However, it lacks a clear framework which would show how automated testing can be achieved. To address this problem, we present a Framework for Automating Statistics-based Testing (FAST), which is an extension of the testing concept in quality programming to achieve automated testing. In FAST, we propose a SOAD tree, which is similar to the structure of the SIAD tree, to describe the syntactic structure of the product unit and its defectiveness. Based on this tool, the inspection of test results can be automatically achieved by lexical and syntax analysis. The implementation of automated software testing or Command File Interpreter (CFI) software which incorporates the framework is also described.     

GPS导航软件的一种功能自动化测试框架

GPS导航软件已经大量涌入市场,对于GPS导航软件本身的质量越来越受到重视。针对GPS导航软件的应用特点,提出了一种GPS导航软件功能自动测试框架,并介绍了相关实现技术,它能自动模拟软件的使用环境和GPS测试数据来验证软件的正确性。在实践中证明此框架行之有效,能有效构建测试环境,及时发现软件的缺陷,并可提高其回归测试的效率。     

LIFT: A new framework of learning from testing data for face recognition

In this paper, a novel learning methodology for face recognition, LearnIng From Testing data (LIFT) framework, is proposed. Considering many face recognition problems featured by the inadequate training examples and availability of the vast testing examples, we aim to explore the useful information from the testing data to facilitate learning. The one-against-all technique is integrated into the learning system to recover the labels of the testing data, and then expand the training population by such recovered data. In this paper, neural networks and support vector machines are used as the base learning models. Furthermore, we integrate two other transductive methods, consistency method and LRGA method into the LIFT framework. Experimental results and various hypothesis testing over five popular face benchmarks illustrate the effectiveness of the proposed framework.     

A search-based framework for automatic testing of MATLAB/Simulink models

Search-based test-data generation has proved successful for code-level testing but almost no search-based work has been carried out at higher levels of abstraction. In this paper the application of such approaches at the higher levels of abstraction offered by MATLAB/Simulink models is investigated and a wide-ranging framework for test-data generation and management is presented. Model-level analogues of code-level structural coverage criteria are presented and search-based approaches to achieving them are described. The paper also describes the first search-based approach to the generation of mutant-killing test data, addressing a fundamental limitation of mutation testing. Some problems remain whatever the level of abstraction considered. In particular, complexity introduced by the presence of persistent state when generating test sequences is as much a challenge at the Simulink model level as it has been found to be at the code level. The framework addresses this problem. Finally, a flexible approach to test sub-set extraction is presented, allowing testing resources to be deployed effectively and efficiently.     

一种关键字驱动的自动化测试框架

为了提高软件自动化测试脚本的可复用性,提出了一种基于关键字驱动的自动化测试框架（LKDT）。首先对目前现有自动化测试框架进行了归纳分析;然后提出了LKDT的系统架构,给出了其中所包含的关键技术;最后以实际的Linux桌面应用程序为例对LKDT框架与现有框架进行实际对比验证,结果表明该框架具有降低测试脚本规模、提高整体测试效率等优越性。     

Wodel-Test: a model-based framework for language-independent mutation testing

Software and Systems Modeling - Mutation testing (MT) targets the assessment of test cases by measuring their efficiency to detect faults. This technique involves modifying the program under test...     

A new approach in development of distributed framework for automated software testing using agents

Software testing is the technical kernel of software quality engineering, and to develop critical and complex software systems not only requires a complete, consistent and unambiguous design, and implementation methods, but also a suitable testing environment that meets certain requirements, particularly, to face the complexity issues. Traditional methods, such as analyzing each requirement and developing test cases to verify correct implementation, are not effective in understanding the software’s overall complex behavior. In that respect, existing approaches to software testing are viewed as time-consuming and insufficient for the dynamism of the modern business environment. This dynamics requires new tools and techniques, which can be employed in tandem with innovative approaches to using and combining existing software engineering methods. This work advocates the use of a recently proposed software engineering paradigm, which is particularly suited to the construction of complex and distributed software-testing systems, which is known as Agent-Oriented Software Engineering. This methodology is a new one, which gives the basic approach to agent-based frameworks for testing.