Checking experiments for stream X-machines

Stream X-machines are a state based formalism that has associated with it a particular development process in which a system is built from trusted components. Testing thus essentially checks that these components have been combined in a correct manner and that the orders in which they can occur are consistent with the specification. Importantly, there are test generation methods that return a checking experiment: a test that is guaranteed to determine correctness as long as the implementation under test (IUT) is functionally equivalent to an unknown element of a given fault domain Ψ. Previous work has show how three methods for generating checking experiments from a finite state machine (FSM) can be adapted to testing from a stream X-machine. However, there are many other methods for generating checking experiments from an FSM and these have a variety of benefits that correspond to different testing scenarios. This paper shows how any method for generating a checking experiment from an FSM can be adapted to generate a checking experiment for testing an implementation against a stream X-machine. This is the case whether we are testing to check that the IUT is functionally equivalent to a specification or we are testing to check that every trace (input/output sequence) of the IUT is also a trace of a nondeterministic specification. Interestingly, this holds even if the fault domain Ψ used is not that traditionally associated with testing from a stream X-machine. The results also apply for both deterministic and nondeterministic implementations.     

Testing a deterministic implementation against a non-controllable non-deterministic stream X-machine

A stream X-machine (SXM) is a type of extended finite state machine with an associated development approach that consists of building a system from a set of trusted components. One of the great benefits of using SXMs for the purpose of specification is the existence of test generation techniques that produce test suites that are guaranteed to determine correctness as long as certain well-defined conditions hold. One of the conditions that is traditionally assumed to hold is controllability: this insists that all paths through the SXM are feasible. This restrictive condition has recently been weakened for testing from a deterministic SXM. This paper shows how controllability can be replaced by a weaker condition when testing a deterministic system against a non-deterministic SXM. This paper therefore develops a new, more general, test generation algorithm for testing from a non-deterministic SXM.     

Generating Test Sets from Non-Deterministic Stream X-Machines

X-machines were proposed by Holcombe as a possible specification language and since then a number of further investigations have demonstrated that the model is intuitive and easy to use as well as general enough to cater for a wide range of applications. In particular (generalised) stream X-machines have been found to be extremely useful as a specification method and most of the theory developed so far has concentrated on this particular class of X-machines. Furthermore, a method for testing systems specified by stream X-machines exists and is proved to detect all faults of the implementation provided that the system meets certain initial requirements. However, this method can only be used to generate test sequences from deterministic X-machine specifications. In this paper we present the theoretical basis for a method for generating test sets from non-deterministic generalised stream X-machines. Received November 1999 / Accepted in revised form September 2000     

Generalised Stream X-Machines with Output Delimited Type

Some conditions relating to the automata involved in the W-testing method are discussed. It is also shown how to use the method for reduced automata instead of minimal automata. New design test conditions (weak output distinguishable, strong test-complete and output delimited type) are considered for the generalised stream X-machines (stream X-machines with basic functions replaced by relations and having as output strings of symbols rather than single symbols). It is proved that testing methods similar to those already developed for ordinary deterministic stream X-machines may be applied for generalised stream X-machines with output delimited types. A particular case of generalised stream X-machine with output delimited type is the X-machine with output delimiter, which produces outputs having a distinct right end character. Received October 2000 / Accepted in revised form January 2001     

Complete deterministic stream X-machine testing

One of the strengths of using stream X-machines to specify a system is that, under certain well defined conditions, it is possible to produce a test set that is guaranteed to determine the correctness of an implementation. However, the existing method assumes that the implementation of each processing function is proved to be correct before the actual testing can take place, so it only test the system integration. This paper presents a new method for generating test sets from a deterministic stream X-machine specification that generalises the existing integration testing method. This method no longer requires the implementations of the processing functions to be proved correct prior to the actual testing. Instead, the testing of the processing functions is performed along with the integration testing.Accepted in revised form 27 February 2004 by D.A. Duce     

Generalised Stream X-Machines and Cooperating Distributed Grammar Systems

Stream X-machines are a general and powerful computational model. By coupling the control structure of a stream X-machine with a set of formal grammars a new machine called a generalised stream X-machine with underlying distributed grammars, acting as a translator, is obtained. By introducing this new mechanism a hierarchy of computational models is provided. If the grammars are of a particular class, say regular or context-free, then finite sets are translated into finite sets, when ?k, = k derivation strategies are used, and regular or context-free sets, respectively, are obtained for ?k, * and terminal derivation strategies. In both cases, regular or context-free grammars, the regular sets are translated into non-context-free languages. Moreover, any language accepted by a Turing machine may be written as a translation of a regular set performed by a generalised stream X-machine with underlying distributed grammars based on context-free rules, under = k derivation strategy. On the other hand the languages generated by some classes of cooperating distributed grammar systems may be obtained as images of regular sets through some X-machines with underlying distributed grammars. Other relations of the families of languages computed by generalised stream X-machines with the families of languages generated by cooperating distributed grammar systems are established. At the end, an example dealing with the specification of a scanner system illustrates the use of the introduced mechanism as a formal specification model. Received September 1999 / Accepted in revised form October 2000     

Testing Conditions for Communicating Stream X-machine Systems

X-machines were proposed by Holcombe as a possible specification language and since then a number of further investigations have demonstrated that the model is intuitive and easy to use. In particular, stream X-machines (SXM), a particular class of X-machines, have been found to be extremely useful in practice. Furthermore, a method of testing systems specified as SXMs exists and is proved to detect all faults of the implementation provided that the system meets certain “design for test conditions”. Recently, a system of communicating SXMs was introduced as a means of modelling parallel processing. This paper proves that each communicating machine component can be transformed in a straightforward manner so that the entire system will behave like a single stream X-machine - the equivalent SXM of the system. The paper goes on to investigate the applicability of the SXM testing method to a system of communicating SXMs and identifies a class of communicating SXMs for which the equivalent SXM of the system meets the “design for test conditions”. Received November 1999 / Accepted in revised form June 2001     

Testing methods for X-machines: a review

The X-machine testing method has been developed as an application of the W-method to testing the control structure of an implementation, against a specification. The method was proven to demonstrate the equivalence of the behaviour of the two, subject to a number of conditions both a specification and an implementation are expected to satisfy, such as (1) determinism of the two and (2) that functions labelling arcs on a transition diagram of a specification control structure have been tested in advance. Since the original publication of the testing method, a number of extensions have been published, removing the restrictions mentioned above. This paper surveys the extensions of the X-machine testing method, for (1) testing of functions together with testing of a transition diagram, (2) equivalence testing of a non-deterministic implementation against a non-deterministic specification, (3) conformance testing of a deterministic implementation against a non-deterministic specification and (4) equivalence testing of a system of concurrently executing and communicating X-machines, against a specification. Received June 2004 Revised March 2005 Accepted March 2005 by J. Derrick, M. Harman and R. M. Herons     

A unified integration and component testing approach from deterministic stream X-machine specifications

One of the great benefits of using a stream X-machine to specify a system is its associated testing method. Under certain design for test conditions, this method produces a test suite that can determine the correctness of the implementation under test (IUT), provided that the basic components of the stream X-machine model have been correctly implemented. However, such an approach implies that each component can be tested in isolation from the rest of the system. This is a limitation that, in practice, can be resolved by developing stubs and drivers. However, this adds complexity to the testing process and, furthermore, these new pieces of software can introduce faults that can invalidate the theoretical results of the aforementioned testing method. This paper extends the approach by allowing component testing to be performed in parallel with integration testing, while still guaranteeing the IUT correctness under the given design for test conditions. It also shows how the integration test suite, produced in previous publications, can be reduced.     

Editorial

Formal Aspects of Computing marks the end of the first year of our re-launched format. It has not been an easy year either for the editors or for the ever-patient staff at Springer-Verlag; but it has certainly been successful with first class papers being published soon after acceptance. As with most computing journals, refereeing poses a (potential) bottleneck to getting an author's ideas into print but even here our colleagues at Springer-Verlag have come up with an incentive scheme from which our future referees will benefit and hopefully speed the refereeing process to everyone's advantage. Recently, most editions of the journal have been standard issues with a number of submitted papers. It has been our stated intention since the journal began to have special editions with whole editions on a single topic. We are currently planning such a special edition to mark Rod Burstall's retirement (in fact we have so many excellent papers that there will probably have to be a double edition). This special issue collects a number of papers on X-machines edited by Mike Holcombe and myself. Mike has written a brief introduction and there follow five papers which have all been refereed by experts in the area (I took personal charge of having Mike's paper refereed). The generalisation of the testing theory to non-deterministic stream X-machines is the focus of two articles. Non-determinism can be generalised in several ways. R. Hierons and M. Harman look at quasi-non-deterministic machines and describe an approach to dealing with the generation of test sets for such machines. F. Ipate and M. Holcombe look at another way to view non-determinism and also focus on test set generation, the issue of fairness becomes important if the strong claims about fault detection by the test sets are to be achieved. M. Gheorghe has investigated how a collection of formal grammars can be controlled by a type of generalised stream X-machine so that the languages generated by such a system of grammars can be determined. He has shown that relatively simple grammars can generate very complex languages using this approach. T. Balanescu explores further generalisations of Stream X-machines and discusses how the design for test conditions can be adapted for a specific type of machine. A. Cowling et al. look at communicating X-machine systems and consider how this approach can be used to model message passing using a simple communicating matrix metaphor. Models built this way can be used to generate, automatically, concurrent programs. In a paper to appear in Volume 13, F. Ipate and M. Holcombe look at how the test theory can be adapted to apply to the communicating X-machines systems case. Indeed, Volume 13 already looks to be an exciting mix of scientific contributions – we also expect to back on a more regular publication schedule by the end of 2001.