Constructing checking sequences for distributed testing

The objective of testing is to determine whether an implementation under test conforms to its specification. In distributed test architectures involving multiple remote testers, this objective can be complicated by the fact that testers may encounter coordination problems relating to controllability (synchronization) and observability during the application of tests. Based on a finite state machine (FSM) specification of the externally observable behaviour of a distributed system and a distinguishing sequence, this paper proposes a method for constructing a checking sequence where there is no potential controllability or observability problems, and where the use of external coordination message exchanges among testers is minimized. The proposed method does not assume a reliable reset feature in the implementations of the given FSM to be tested by the resulting checking sequence. phone: 613-562-5800(Extn)6684 Received May 2004 Revised March 2005 Accepted April 2005 by J. Derrick, M. Harman and R. M. Herons     

Distributed testing without encountering controllability and observability problems

The objective of testing is to determine whether a system under test conforms to its specification. In distributed test architectures that utilize remote testers, this objective can be complicated by the fact that testers may encounter problems relating to controllability and observability during the application of a test sequence. Existing solutions to these problems involve first constructing a test sequence from the specification of an implementation under test, and then inserting coordination messages or appending selected test subsequences that prevent the occurrences of controllability and observability problems during the application of the resulting test sequence. This paper proposes a method that utilizes a set of transformation rules to construct an auxiliary directed graph from a given specification, and constructs a rural Chinese postman tour in this graph to yield a minimum-length test sequence where there is no potential controllability or observability problems, and where the use of coordination messages is minimized.     

Using status messages in the distributed test architecture

If the system under test has multiple interfaces/ports and these are physically distributed then in testing we place a tester at each port. If these testers cannot directly communicate with one another and there is no global clock then we are testing in the distributed test architecture. If the distributed test architecture is used then there may be input sequences that cannot be applied in testing without introducing controllability problems. Additionally, observability problems can allow fault masking. In this paper we consider the situation in which the testers can apply a status message: an input that causes the system under test to identify its current state. We show how such a status message can be used in order to overcome controllability and observability problems.     

Overcoming controllability problems with fewest channels between testers

When testing a system that has multiple physically distributed ports/interfaces it is normal to place a tester at each port. Each tester observes only the events at its port and it is known that this can lead to additional controllability problems. While such controllability problems can be overcome by the exchange of external coordination messages between the testers, this requires the deployment of an external network and may thus increase the costs of testing. The problem studied in this paper is finding a minimum number of coordination channels to overcome controllability problems in distributed testing. Three instances of this problem are considered. The first problem is to find a minimum number of channels between testers in order to overcome the controllability problems in a given test sequence to be applied in testing. The second problem is finding a minimal set of channels that allow us to overcome controllability problems in any test sequence that may be selected from the specification of the system under test. The last problem is to find a test sequence that achieves a particular test objective and in doing so allows fewest channels to be used.     

Overcoming controllability problems in distributed testing from an input output transition system

This paper concerns the testing of a system with physically distributed interfaces, called ports, at which it interacts with its environment. We place a tester at each port and the tester at port p observes events at p only. This can lead to controllability problems, where the observations made by the tester at a port p are not sufficient for it to be able to know when to send an input. It is known that there are test objectives, such as executing a particular transition, that cannot be achieved if we restrict attention to test cases that have no controllability problems. This has led to interest in schemes where the testers at the individual ports send coordination messages to one another through an external communications network in order to overcome controllability problems. However, such approaches have largely been studied in the context of testing from a deterministic finite state machine. This paper investigates the use of coordination messages to overcome controllability problems when testing from an input output transition system and gives an algorithm for introducing sufficient messages. It also proves that the problem of minimising the number of coordination messages used is NP-hard.     

多单元协议一致性测试中的同步序列的生成

有限状态机模型一般被用来描述通信协议和其它各类的分布式系统,对于一个多端口的有限状态机,需要多个测试单元进行测试,使用一个包括K个（K≥2）测试单元的测试系统可以检查一个多单元通信协议软件的收发行为是否与协议规格一致,在测试过程中,K个测试单元之间可能会出珊步问题,目前,主要是通过增加外部同步操作来解决同步问题,提出了一种新的同步测试序列生成模型－－同步有向图,它可以判断一个给定的协议规格是否可以在不需要外部同步操作的情况下,产生同步测试序列;如果可以产生,则此生成中以将非同步测试相应的同步测试序列;另外此生成模型还可以用来选择为测试系统增加外部同步通道的方法。     

Coordination Algorithm for Distributed Testing

With the emergence of new models, architectures and middleware such as ODP, TINA and CORBA, for developing open distributed systems, testing technology requires adaptation for use within conformance assessment in such systems. All these frameworks are object-based and aim at creating open distributed environments supporting interworking, interoperability, and portability, in spite of heterogeneity and autonomy of the related systems. In this context, an open distributed system may be viewed as a system providing standardized distributed interfaces for interacting with other systems. Conformance of such a system can be assessed by attaching a related tester at each provided interface. However, many problems of controllability and observability influencing fault detection during the testing process, arise if there is no coordination between the testers. In this paper, we show how to cope with these problems by using test coordination procedures in a distributed testing architecture.     

UIO sequence based checking sequences for distributed test architectures

This study addresses the construction of a preset checking sequence that will not pose controllability (synchronization) and observability (undetectable output shift) problems when applied in distributed test architectures that utilize remote testers. The controllability problem manifests itself when a tester is required to send the current input and because it did not send the previous input nor did it receive the previous output it cannot determine when to send the input. The observability problem manifests itself when a tester is expecting an output in response to either the previous input or the current input and because it is not the one to send the current input, it cannot determine when to start and stop waiting for the output. Based on UIO sequences, a checking sequence construction method is proposed to yield a sequence that is free from controllability and observability problems.     

Architecture-based testing of service-oriented applications in distributed systems

ContextTesting distributed service-oriented applications (SOAs) is more challenging than testing monolithic applications since these applications have complex interactions between participant services. Test engineers can observe test results only through a front service that handles request messages sent by test engineers. Message exchanges between participant services are hidden behind the front service and cannot be easily observed or controlled through the front service. For this reason, testing SOAs suffer from limited observability and controllability problem.ObjectiveThis paper proposes a new test method that is architecture-based and exploits interaction architecture of a SOA. The proposed test method alleviates the limited observability and controllability problem by employing test architecture, thereby facilitating test execution and analysis through monitoring and controlling message exchanges.MethodOur proposed method derives an interaction architecture from the specification of a SOA. Test architectures can be designed from the derived interaction architecture by extending it with additional test elements. At the same time, architecture-neutral test scenarios are automatically generated from the test model that is constructed from the specification. Our method combines test architecture information with the test scenarios to obtain architecture-enabled test scenarios under the selected test architectures. Finally, architecture-enabled test execution and analysis are conducted in the real network environment.ResultsThe efficacy of the proposed method is demonstrated with an industrial case study, which shows that it is practical and effective for testing SOAs. Even though our method increases an additional test generation effort owing to test architecture, it is counterbalanced by higher fault detection rate and faster fault locating time.ConclusionThe main benefit of our approach is that using test architecture it enhances testability of SOA by increasing observability and controllability through monitoring and controlling message exchanges. Our architecture-based test method enables test engineers to detect faults efficiently and also reduce fault locating time significantly.     

基于np-FSM的分布式协议的虚拟多端口测试方法

在协议工程领域,协议测试是非常重要的工作;对于分布式协议的测试多采用多端口有限状态机模型进行描述;由于分布式协议测试存在控制观察问题,需要对测试序列进行同步和协调;通过对现有分布式测试模型的改进,引入虚拟多端口测试的方法,极大地简化了多个测试器之间的同步和协调问题,并且不会显著增加系统的构建成本,提高了测试效率.     

基于同步有向图的同步测试序列生成方法

使用多测试单元的测试系统可以对多端口协议实现进行一致性测试，但是在进行这种一致性测试时，测试系统各个端口之间可能会出现同步问题，现在，解决同步问题常用的办法是在测试单元相应端口之间增加同步连接，然后通过此同步连接相互发送同步消息来进行同步，多端口协议和其它类型的分布式系统可以用有限状态机模型来描述，目前，同步问题被分为双端口同步问题，多端口同步问题，紧同步问题等多种类型，该文考虑两种有限状态机测试问题，第一种是面向端口的测试，不考虑有限状态机测试单元之间的通信问题，第二种面向组的测试，有限状态机中的各个端口被分成互不相关的多个组，属于不同组中的测试单元之间互不通信，该文提出了一种基于同步有向图的同步测试序列生成方法，这种生成方法适用于Pair同步，Port同步和组同步问题，并且，这种方法也可以用来判断如何在非同步测试序列中增加同步通信，将非同步测试序列转化为同步测试序列。     

Testing the Schedulability of Synchronous Traffic for the Timed Token Medium Access Control Protocol

One of the key issues related to guaranteeing synchronous message deadlines in a timed token network (such as fiber distributed data interface) where the timed token medium access control protocol is used is the schedulability test of synchronous traffic (i.e., testing whether or not all synchronous messages can be transmitted before their deadlines, under a given setting of network parameters). Much work has been done on how to assign network parameters appropriately in order to guarantee timely transmission of synchronous traffic. As a result quite a few synchronous bandwidth allocation schemes and some good guidelines on selection of the target token rotation time have been proposed. In contrast, limited research has been conducted on how to effectively test whether or not given network parameters can guarantee timely transmission of all synchronous messages (of a considered synchronous message set) before their deadlines. The previous testing methods for synchronous message schedulability only provide a sufficient (but not necessary) test and therefore fail to always keep effective for any synchronous message set considered. In this paper, we propose two testing methods for determining the schedulability of a synchronous message set with message deadlines no longer than periods. The proposed tests perform better than any previous test in the sense that they are both sufficient and necessary. Some numerical examples are given to compare different testing methods, all of which have demonstrated the superiority of the proposed tests to other existing testing methods.     

A Kleene theorem and model checking algorithms for existentially bounded communicating automata

The behavior of a network of communicating automata is called existentially bounded if communication events can be scheduled in such a way that the number of messages in transit is always bounded by a value that depends only on the machine, not the run itself. We show a Kleene theorem for existentially bounded communicating automata, namely the equivalence between communicating automata, globally cooperative compositional message sequence graphs, and monadic second order logic. Our characterization extends results for universally bounded models, where for each and every possible scheduling of communication events, the number of messages in transit is uniformly bounded. As a consequence, we give solutions in spirit of Madhusudan (2001) for various model checking problems on networks of communicating automata that satisfy our optimistic restriction.     

Detecting Coordination Problems in Collaborative Software Development Environments

Software development is rarely an individual effort and generally involves teams of developers collaborating to generate good reliable code. Among the software code there exist technical dependencies that arise from software components using services from other components. The different ways of assigning the design, development, and testing of these software modules to people can cause various coordination problems among them. We claim that the collaboration of the developers, designers and testers must be related to and governed by the technical task structure. These collaboration practices are handled in what we call Socio-Technical Patterns.

The TESNA project (Technical Social Network Analysis) we report on in this paper addresses this issue. We propose a method and a tool that a project manager can use in order to detect the socio-technical coordination problems. We test the method and tool in a case study of a small and innovative software product company.     

Specifying and Validating Data-Aware Temporal Web Service Properties

Most works that extend workflow validation beyond syntactical checking consider constraints on the sequence of messages exchanged between services. These constraints are expressed only in terms of message names and abstract away their actual data content. We provide examples of real-world "data-aware" Web service constraints where the sequence of messages and their content are interdependent. To this end, we present CTL-FO⁺, an extension over computation tree logic that includes first-order quantification on message content in addition to temporal operators. We show how CTL-FO⁺ is adequate for expressing data-aware constraints, give a sound and complete model checking algorithm for CTL-FO⁺, and establish its complexity to be PSPACE-complete. A "naive" translation of CTL-FO⁺ into CTL leads to a serious exponential blowup of the problem that prevents existing validation tools to be used. We provide an alternate translation of CTL-FO⁺ into CTL, where the construction of the workflow model depends on the property to validate. We show experimentally how this translation is significantly more efficient for complex formulas and makes model checking of data-aware temporal properties on real-world Web service workflows tractable using off-the-shelf tools.     

基于混沌序列的最大和分散式协调算法

大规模信息系统的协调问题正成为新兴的研究热点,最大和协调算法显示了在该领域的应用前景,然而其收敛速度和鲁棒性有待进一步提高。针对以上问题,提出一种可用于合作系统中的基于混沌的分散式信息传递算法,并通过图形着色问题验证了该算法的有效性。在该算法中,利用混沌序列丰富的时空动态性,产生最大和协调算法的结点信息。然后依据更新规则对结点信息进行交互,完成算法的迭代。通过与传统最大和协调算法和遗传算法的对比实验结果分析,该算法在收敛速度及鲁棒性方面有着更好的表现。     

Strong types for coordinating active objects

An object type is usually regarded as a contract between an object and each of its clients. However, in concurrent (and sometimes also in sequential) systems it is more useful to regard a type as a contract between an object and the set of all clients: when the object acts as a shared resource, the clients must be coordinated before sending messages to the object. Process types of active objects specify constraints on the ordering of messages. Static type checking ensures proper coordination of clients so that objects receive messages only in acceptable orderings. As presented in this article, the process type system is based on a simple computation model where active objects communicate via asynchronous message passing. Copyright © 2001 John Wiley & Sons, Ltd.     

State inference for dynamically changing interfaces

Types in current programming languages specify constant sets of messages always acceptable throughout the lifetime of the types’ instances. However, especially in concurrent object-oriented systems, the acceptability of messages often changes with the objects’ states. We propose a typed concurrent object calculus where static type checking ensures that users send only acceptable messages although message acceptability may change dynamically. The programmer specifies in types predictable state changes and dependences of message acceptance on states; a compiler infers the needed state information. This state inference has polynomial time complexity and can be used together with subtyping.     

基于MSC的TTCN-3测试例生成研究

提出了从消息序列图(MSC)生成TTCN-3测试例的方法.测试者可以先针对被测系统(SUT)中感兴趣的部分利用MSC描述出具体的测试场景,再从MSC映射成TTCN-3测试例.将MSC映射成TTCN.3的方法主要包括对MSC文档、基本MSC和高级MSC这3部分的映射.