服务接口测试自动化工具的研究

在面向服务的架构中,软件功能通常被封装为服务,并以开放可编程接口的方式提供应用.因此,服务接口的正确性和可靠性对于服务理解、重用和集成至关重要.随着软件即服务的思想被广为认可,互联网上越来越多的软件应用以服务接口方式提供开放式访问.面对众多的服务接口,如何进行高效且有效的测试是一个亟待解决的问题.为此,提出了一种基于模型驱动的自动化测试方法,首先采用接口语义契约模型对被测服务接口建模,然后基于接口模型和测试覆盖率需求为单个服务和组合服务生成测试数据和测试用例.在此基础上,设计并实现了AutoTest自动化测试工具,该工具采用多种算法用于优化测试用例生成,且支持图形化测试计划编排和跨语言测试代码生成.实验结果表明,AutoTest能够有效支持大批量测试用例的设计和生成,根据OED(正交实验设计)算法生成的测试用例与IPO组合算法得到的用例相比,具有更为理想的测试覆盖率.     

基于异步时钟的SoC功耗约束测试调度优化

测试调度是一种能有效减少片上系统(system-on-chip, SoC)测试耗时(test application time, TAT)以降低测试成本的经典技术.然而,随着功耗问题的日益加剧,功耗约束成为测试调度中必须考虑的重要问题.可以调节各测试周期长度的异步时钟测试在对单个电路进行测试用时优化时效果显著,但直接将其应用于SoC测试调度并非总能获得最优的调度结果,使用传统测试调度模型往往会产生明显非最优的结果.在结合图论中团(clique)的概念,并分析异步时钟机制的特点后,提出一种将异步时钟特性应用于功耗约束SoC测试调度的方案.使用测试兼容图(test compatibility graph, TCG)和混合整型线性规划(mixed integer linear programming, MILP)建立相对应数学模型,理论分析和在ITC02基准SoC集上的模拟实验结果表明,该方案能有效地减少测试耗时.     

面向对象的软件测试多Agent系统研究

文中提出一种软件测试多Agent系统(STMAS)，该系统由用户界面Agent、测试用例选择和测试Agent、回归测试Agent三部分组成，这些Agent运用它们的智能规则为面向对象的软件测试提供积极的帮助。系统由于能进行自动测试，可以将测试者的干预减到最少；能进行没有冗余的、一致性的测试用例的智能选择，可以在提高故障检测有效性的同时缩短测试时间。     

Software Engineering with Transactional Memory Versus Locks in Practice

Transactional Memory (TM) promises to simplify parallel programming by replacing locks with atomic transactions. Despite much recent progress in TM research, there is very little experience using TM to develop realistic parallel programs from scratch. In this article, we present the results of a detailed case study comparing teams of programmers developing a parallel program from scratch using transactional memory and locks. We analyze and quantify in a realistic environment the development time, programming progress, code metrics, programming patterns, and ease of code understanding for six teams who each wrote a parallel desktop search engine over a fifteen week period. Three randomly chosen teams used Intel’s Software Transactional Memory compiler and Pthreads, while the other teams used just Pthreads. Our analysis is exploratory: Given the same requirements, how far did each team get? The TM teams were among the first to have a prototype parallel search engine. Compared to the locks teams, the TM teams spent less than half the time debugging segmentation faults, but had more problems tuning performance and implementing queries. Code inspections with industry experts revealed that TM code was easier to understand than locks code, because the locks teams used many locks (up to thousands) to improve performance. Learning from each team’s individual success and failure story, this article provides valuable lessons for improving TM.     

Model-based testing of a real-time adaptive motion planning system

Abstract

To enable effective and safe operations of autonomous robots in environments with unknowns and unpredictability, a key practical problem is how to test the functionality and assess the performance of real-time motion planning systems. This is a challenge because the underlying algorithms are real-time, sensing-based, and often non-deterministic. These systems’ performance depends on task environments, which can vary in countless ways. Existing testing techniques are designed heavily based on testers’ experience and hardly provide a good coverage of possible test scenarios. This paper introduces a systematic model-based testing (MBT) approach to evaluate the functionality and performance of a real-time adaptive motion planning (RAMP) system. The MBT approach uses the formal communicating extended finite state machine model to model RAMP’s concurrent components and leverage graph traversal algorithms to systematically generate behavioral test cases. First, component integration is considered by modeling the RAMP components and their interactions. Next, system-level testing is considered by modeling mobile obstacles of unpredictable motion behavior. The behavior models are leveraged to generate Abstract Behavioral Test Cases, which are transformed by test data into executable test cases. The test results demonstrate the effectiveness of applying the systematic MBT approach to the evaluation of real-time robotic systems.     

并行测试技术在自动测试系统中的应用

并行测试拥有减少测试时间、降低测试成本的强大优势,正成为研究热点之一。首先详细分析了并行测试的基本概念,介绍了目前实现并行测试可以采用的两大类4种结构,对这4种结构各自的优缺点进行了比较。接着以多线程并行测试程序为例描述了并行测试程序中同步、异步和单线程的三种模型,最后重点对多线程并行测试实现中几个值得注意的重要问题进行了讨论。     

Global software testing under deadline pressure: Vendor-side experiences

ContextIn the era of globally-distributed software engineering, the practice of global software testing (GST) has witnessed increasing adoption. Although there have been ethnographic studies of the development aspects of global software engineering, there have been fewer studies of GST, which, to succeed, can require dealing with unique challenges.ObjectiveTo address this limitation of existing studies, we conducted, and in this paper, report the findings of, a study of a vendor organization involved in one kind of GST practice: outsourced, offshored software testing.MethodWe conducted an ethnographically-informed study of three vendor-side testing teams over a period of 2 months. We used methods, such as interviews and participant observations, to collect the data and the thematic-analysis approach to analyze the data.FindingsOur findings describe how the participant test engineers perceive software testing and deadline pressures, the challenges that they encounter, and the strategies that they use for coping with the challenges. The findings reveal several interesting insights. First, motivation and appreciation play an important role for our participants in ensuring that high-quality testing is performed. Second, intermediate onshore teams increase the degree of pressure experienced by the participant test engineers. Third, vendor team participants perceive productivity differently from their client teams, which results in unproductive-productivity experiences. Lastly, participants encounter quality-dilemma situations for various reasons.ConclusionThe study findings suggest the need for (1) appreciating test engineers’ efforts, (2) investigating the team structure’s influence on pressure and the GST practice, (3) understanding culture’s influence on other aspects of GST, and (4) identifying and addressing quality-dilemma situations.     

From testing to anti-product development

The requirements for testing in large software projects drastically changed in recent years. The reasons for this mainly come from evolving development processes, demanding early stabilisations of complete software loads, early handover to subsequent steps, high software release quality, and in particular flexible adaptations to changing roadmaps. This study is an experience report from industrial testing in the field of telecommunications software and based on a speech given by the author at the Testing and Test Control Notation version 3 (TTCN-3) User Conference in Berlin 2006. It advocates to consider modern testing rather as the development of an anti-product that is to be developed in parallel to the product. This anti-product is itself a complex software system. Following this approach has a significant impact on the testing in these large software projects mentioned and leads to a number of consequences for the development of test systems and the structure of test teams. In this study, the role and virtues of TTCN-3 as programming language for anti-products as well as for defining the test system structure are investigated. Dealing with large embedded software systems that have several concurrent parts and additionally obey soft real-time requirements, similar constraints also hold for the test system. The virtues of parallel testing/software development life cycles as it becomes mandatory in following modern development processes are presented, and it is shown how TTCN-3 turned out in our experience to be a language particularly well suited for this purpose because of its advanced structured programming concepts derived from strong typing, template concepts and parametrizations of templates.     

Putting your best tests forward

Test case prioritization orders tests so that they help you meet your testing goals earlier during regression testing. Prioritization techniques can, for example, order tests to achieve coverage at the fastest rate possible, exercise features in order of expected frequency of use, or reveal faults as early as possible. We focus on the last goal, which we describe as "increasing a test suite's rate of fault detection" or the speed with which the test suite reveals faults. A faster fault detection rate during regression testing provides earlier feedback on a system under test, supporting earlier strategic decisions about release schedules and letting engineers begin debugging sooner. Also, if testing time is limited or unexpectedly reduced, prioritization increases the chance that testing resources will have been spent as cost effectively as possible in the available time.     

面向ROS的差分模糊测试方法

机器人操作系统(robot operating system,简称ROS)是一种广泛应用于机器人开发的开源系统,它可以为开发者提供硬件抽象、设备驱动、库函数、可视化、消息传递和软件包管理等诸多功能,应用前景广阔.ROS集成了可以实现不同功能的功能包,例如定位绘图、行动规划、感知、模拟等等,但其中可能存在一些漏洞,破坏整...     

Mutation testing in UTP

This paper presents a theory of testing that integrates into Hoare and He’s Unifying Theory of Programming (UTP). We give test cases a denotational semantics by viewing them as specification predicates. This reformulation of test cases allows for relating test cases via refinement to specifications and programs. Having such a refinement order that integrates test cases, we develop a testing theory for fault-based testing. Fault-based testing uses test data designed to demonstrate the absence of a set of pre-specified faults. A well-known fault-based technique is mutation testing. In mutation testing, first, faults are injected into a program by altering (mutating) its source code. Then, test cases that can detect these errors are designed. The assumption is that other faults will be caught, too. In this paper, we apply the mutation technique to both, specifications and programs. Using our theory of testing, two new test case generation laws for detecting injected (anticipated) faults are presented: one is based on the semantic level of UTP design predicates, the other on the algebraic properties of a small programming language.     

Pattern‐based GUI testing: Bridging the gap between design and quality assurance

Software systems with a graphical user interface (GUI) front end are typically designed using user interface (UI) Patterns, which describe generic solutions (with multiple possible implementations) for recurrent GUI design problems. However, existing testing techniques do not take advantage of this fact to test GUIs more efficiently. In this paper, we present a new pattern‐based GUI testing (PBGT) approach that formalizes the notion of UI Test Patterns, which are generic test strategies to test UI patterns over their different implementations. The PBGT approach is evaluated via 2 case studies. The first study involves 2 fielded Web application subjects; findings show that PBGT is both practical and useful, as testing teams were able to find real bugs in a reasonable time interval. The second study allows deeper analysis by studying software subjects seeded with artificial faults; the findings show that PBGT is more effective than a manual model‐based test case generation approach.     

用遗传算法优化测试通路结构设计

嵌入核测试通路问题是片上系统设计中的重要问题，由于嵌入核与芯片的输入/输出管脚没有直接通路，因此需要设计专门的测试通路结构对它们进行测试，以减少测试时间，降低测试成本，提出一种基于遗传算法的优化算法来设计测试通路结构，并选取了两个假定的、比较复杂的片上系统作为例子，实验结果表明，文中算法搜索到全局最优解(或近似全局最优解)的能力优于现有的整数线性规划方法。     

某式某型高超声速风洞连续变攻角测力试验测试系统研制及关键技术研究

在某式某型高超声速风洞开展了连续变攻角测力试验技术研究,针对性地建立了某式某型高超声速风洞专用的连续变攻角试验软硬件测试系统,完成了模型攻角的实时测量、滤波截止频率和机构运行角速度择选、多类信号高速连续采集、信号相关性研究和数据同步修正、专用数据采集和处理软件研制等多项研究内容.该项技术通过了多轮动态试验验证,从试验结...     

Prioritizing MCDC test cases by spectral analysis of Boolean functions

Test case prioritization aims at scheduling test cases in an order that improves some performance goal. One performance goal is a measure of how quickly faults are detected. Such prioritization can be performed by exploiting the fault exposing potential (FEP) parameters associated to the test cases. The FEP is usually approximated by mutation analysis under certain fault assumptions. Although this technique is effective, it could be relatively expensive compared to the other prioritization techniques. This study proposes a cost‐effective FEP approximation for prioritizing modified condition decision coverage (MCDC) test cases. A strict negative correlation between the FEP of an MCDC test case and the influence value of the associated input condition allows to order the test cases easily without the need of an extensive mutation analysis. The method is entirely based on mathematics and it provides useful insight into how spectral analysis of Boolean functions can benefit software testing.     

Pair programming teams and high-quality knowledge sharing: A comparative study of coopetitive reward structures

There has been a growing research interest in understanding knowledge sharing in agile development. Yet, empirical research that sheds light on its underlying practices, such as pair programming, is evolving. This study uses insights from coopetition and software literature to focus inquiry on the relation between coopetitive rewards and high-quality knowledge sharing in pair programming teams. Theoretical hypotheses are developed and validated, suggesting that: ‘coopetitive rewards influence high-quality knowledge sharing both directly and over time through their impact on the level of knowledge sharing satisfaction’, and, ‘the impact of coopetitive rewards on high-quality knowledge sharing is dependent upon task complexity and the history of working under similar reward structure’. This study generates new understanding related to the use of rewards in pair programming teams, and offers a rigorous and replicable seven-step experimental process for simulating coopetitive structures and investigating their role in pair programming and in similar collaborative contexts.     

Similarity-based prioritization of test case automation

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.     

ClassBench: a Framework for Automated Class Testing

In contrast to the explosion of activity in object-oriented design and programming, little attention has been given to object testing. We present a novel approach to automated testing designed especially for collection classes. In the ClassBench methodology, a testgraph partially models the states and transitions of the Class-Under-Test (CUT) state/transition graph. To determine the expected behavior for the test cases generated from the testgraph, the tester develops an oracle class, providing essentially the same operations as the CUT but supporting only the testgraph states and transitions. Surprisingly thorough testing is achievable with simple testgraphs and oracles. The ClassBench framework supports the tester by providing a testgraph editor, automated testgraph traversal, and a variety of utility classes. Test suites can be easily configured for regression testing–where many test cases are run–and debugging–where a few test cases are selected to isolate the bug. We present the ClassBench methodology and framework in detail, illustrated on both simple examples and on test suites from commercial collection class libraries. © 1997 John Wiley & Sons, Ltd.     

Test diversity

This paper describes a novel method for measuring the degree to which a set of test cases executes a given program in diverse ways with respect to the two fundamental programming concepts: control and data. Test diversity is a method for measuring the variety of software control flow and data flow, comprising of four new measures: conditional diversity, data diversity, standard deviation of diversity, and test orthogonality. These closely related measures could be used to evaluate the test effectiveness and the test-effort distribution of a test suite.The Diversity Analyzer is a novel industrial-strength testing tool that can currently perform diversity analysis on software written in C/C++/C#/VB/Java in Windows and .NET environments. The Diversity Analyzer is used to evaluate the fault-detection effectiveness of Test Diversity on various types of industrial projects.     

MySQL数据库系统自动组合测试

MySQL Test Framework对MySQL的自动化测试提供了测试方法和测试用例。应用自动化的理念,将其测试用例进行拆分并根据测试要求重新组合,以节省运行资源。