Traffic-aware stress testing of distributed real-time systems based on UML models using genetic algorithms

This paper presents a model-driven, stress test methodology aimed at increasing chances of discovering faults related to network traffic in distributed real-time systems (DRTS). The technique uses the UML 2.0 model of the distributed system under test, augmented with timing information, and is based on an analysis of the control flow in sequence diagrams. It yields stress test requirements that are made of specific control flow paths along with time values indicating when to trigger them. The technique considers different types of arrival patterns (e.g., periodic) for real-time events (common to DRTSs), and generates test requirements which comply with such timing constraints. Though different variants of our stress testing technique already exist (that stress different aspects of a distributed system), they share a large amount of common concepts and we therefore focus here on one variant that is designed to stress test the system at a time instant when data traffic on a network is maximal. Our technique uses genetic algorithms to find test requirements which lead to maximum possible traffic-aware stress in a system under test. Using a real-world DRTS specification, we design and implement a prototype DRTS and describe, for that particular system, how the stress test cases are derived and executed using our methodology. The stress test results indicate that the technique is significantly more effective at detecting network traffic-related faults when compared to test cases based on an operational profile.     

A state-based approach to integration testing based on UML models

Correct functioning of object-oriented software depends upon the successful integration of classes. While individual classes may function correctly, several new faults can arise when these classes are integrated together. In this paper, we present a technique to enhance testing of interactions among modal classes. The technique combines UML collaboration diagrams and statecharts to automatically generate an intermediate test model, called SCOTEM (State COllaboration TEst Model). The SCOTEM is then used to generate valid test paths. We also define various coverage criteria to generate test paths from the SCOTEM model. In order to assess our technique, we have developed a tool and applied it to a case study to investigate its fault detection capability. The results show that the proposed technique effectively detects all the seeded integration faults when complying with the most demanding adequacy criterion and still achieves reasonably good results for less expensive adequacy criteria.     

Improving scalability of event‐driven distributed objects architectures

The goal of this paper is to present practical techniques for improving the scalability of a distributed objects architecture built around an event‐driven paradigm and the evaluation of their effectiveness in the case of location systems. As a case study, the ABng location system constructed using a publisher/subscriber pattern over a CORBA compliant middleware is presented. The main contribution of this paper is the elaboration and evaluation of three main concepts that play a crucial role in improving the scalability of event‐driven architectures: events dissemination quality of service, notification tree based events‐routing, and transparent multicast event‐channel notification service. The applicability and efficiency of these techniques have been verified in practice by the ABng system implementation and performance testing. Copyright © 2000 John Wiley & Sons, Ltd.     

基于UML的软件使用模型生成技术研究

软件可靠性测试是高可靠软件质量保证的重要过程,而软件使用模型则是进行软件可靠性测试的基础。UML(Unified Modeling Language)作为事实上的工业标准,在软件开发中得到了日益广泛的应用。研究基于UML的软件可靠性测试具有很强的现实意义。利用在软件开发早期阶段生成的UML模型,如用例图、顺序图、活动图,可以生成软件使用模型,进而可据此产生测试用例。     

Search‐based testing using constraint‐based mutation

Many modern automated test generators are based on either metaheuristic search techniques or use constraint solvers. Both approaches have their advantages, but they also have specific drawbacks: Search‐based methods may get stuck in local optima and degrade when the search landscape offers no guidance; constraint‐based approaches, on the other hand, can only handle certain domains efficiently. This paper describes a method that integrates both techniques and delivers the best of both worlds. On a high‐level view, the proposed method uses a genetic algorithm to generate tests, but the twist is that during evolution, a constraint solver is used to ensure that mutated offspring efficiently explores different control flow. Experiments on 20 case study programmes show that on average the combination improves branch coverage by 28% over search‐based techniques while reducing the number of tests by 55%, and improves coverage by 13% over constraint‐based techniques while reducing the number of tests by 73%. Copyright © 2013 John Wiley & Sons, Ltd.     

A transformation‐based approach to testing concurrent programs using UML activity diagrams

Unified Modeling Language (UML) activity diagrams are widely used to model concurrent interaction among multiple objects. In this paper, we propose a transformation‐based approach to generating scenario‐oriented test cases for applications modeled by UML activity diagrams. Using a set of transformation rules, the proposed approach first transforms a UML activity diagram specification into an intermediate representation, from which it then constructs test scenarios with respect to the given concurrency coverage criteria. The approach then finally derives a set of test cases for the constructed test scenarios. The approach resolves the difficulties associated with fork and join concurrency in the UML activity diagram and enables control over the number of the resulting test cases. We further implemented a tool to automate the proposed approach and studied its feasibility and effectiveness using a case study. Experimental results show that the approach can generate test cases on demand to satisfy a given concurrency coverage criterion and can detect up to 76.5% of seeded faults when a weak coverage criterion is used. With the approach, testers can not only schedule the software test process earlier, but can also better allocate the testing resources for testing concurrent applications. Copyright © 2015 John Wiley & Sons, Ltd.     

Composing and scheduling service‐oriented applications in time‐triggered distributed real‐time Java environments

During the last decade, the number of distributed application domains with temporal requirements has significantly augmented, arising the necessity of exploring new concepts and paradigms that allow, on the one hand, the development of dynamic and flexible distributed applications and, on the other hand, the reusability of code. Service‐oriented paradigms have been successfully applied to distributed environments, increasing their flexibility and allowing the reusability of their components. Besides, distributed real‐time Java technologies have shown to be a good candidate to deploy real‐time distributed applications. This paper presents a model for service‐oriented applications on a time‐triggered distributed real‐time Java environment, focusing on the definition of the temporal model of an application and its schedulability, applying and evaluating this model in real‐time service‐oriented composition algorithms. Copyright © 2012 John Wiley & Sons, Ltd.     

Simulation of resource synchronization in a dynamic real‐time distributed computing environment

Today, more and more distributed computer applications are being modeled and constructed using real‐time principles and concepts. In 1989, the Object Management Group (OMG) formed a Real‐Time Special Interest Group (RT SIG) with the goal of extending the Common Object Request Broker Architecture (CORBA) standard to include real‐time specifications. This group's most recent efforts have focused on the requirements of dynamic distributed real‐time systems. One open problem in this area is resource access synchronization for tasks employing dynamic priority scheduling. This paper presents two resource synchronization protocols that the authors have developed which meet the requirements of dynamic distributed real‐time systems as specified by Dynamic Scheduling Real‐Time CORBA (DSRT CORBA). The proposed protocols can be applied to both Earliest Deadline First (EDF) and Least Laxity First (LLF) dynamic scheduling algorithms, allow distributed nested critical sections, and avoid unnecessary runtime overhead. In order to evaluate the performance of the proposed protocols, we analyzed each protocol's schedulability. Since the schedulability of the system is affected by numerous system configuration parameters, we have designed simulation experiments to isolate and illustrate the impact of each individual system parameter. Simulation experiments show the proposed protocols have better performance than one would realize by applying a schema that utilizes dynamic priority ceiling update. Copyright © 2004 John Wiley & Sons, Ltd.     

Killing strategies for model‐based mutation testing

This article presents the techniques and results of a novel model‐based test case generation approach that automatically derives test cases from UML state machines. The main contribution of this article is the fully automated fault‐based test case generation technique together with two empirical case studies derived from industrial use cases. Also, an in‐depth evaluation of different fault‐based test case generation strategies on each of the case studies is given and a comparison with plain random testing is conducted. The test case generation methodology supports a wide range of UML constructs and is grounded on the formal semantics of Back's action systems and the well‐known input–output conformance relation. Mutation operators are employed on the level of the specification to insert faults and generate test cases that will reveal the faults inserted. The effectiveness of this approach is shown and it is discussed how to gain a more expressive test suite by combining cheap but undirected random test case generation with the more expensive but directed mutation‐based technique. Finally, an extensive and critical discussion of the lessons learnt is given as well as a future outlook on the general usefulness and practicability of mutation‐based test case generation. Copyright © 2014 John Wiley & Sons, Ltd.     

The software architecture of a distributed problem‐solving environment

This paper describes the functionality and software architecture of a generic problem‐solving environment (PSE) for collaborative computational science and engineering. A PSE is designed to provide transparent access to heterogeneous distributed computing resources, and is intended to enhance research productivity by making it easier to construct, run, and analyze the results of computer simulations. Although implementation details are not discussed in depth, the role of software technologies such as CORBA, Java, and XML is outlined. An XML‐based component model is presented. The main features of a Visual Component Composition Environment for software development, and an Intelligent Resource Management System for scheduling components, are described. Some prototype implementations of PSE applications are also presented. Copyright © 2000 John Wiley & Sons, Ltd.     

Search‐based software test data generation: a survey

The use of metaheuristic search techniques for the automatic generation of test data has been a burgeoning interest for many researchers in recent years. Previous attempts to automate the test generation process have been limited, having been constrained by the size and complexity of software, and the basic fact that, in general, test data generation is an undecidable problem. Metaheuristic search techniques offer much promise in regard to these problems. Metaheuristic search techniques are high‐level frameworks, which utilize heuristics to seek solutions for combinatorial problems at a reasonable computational cost. To date, metaheuristic search techniques have been applied to automate test data generation for structural and functional testing; the testing of grey‐box properties, for example safety constraints; and also non‐functional properties, such as worst‐case execution time. This paper surveys some of the work undertaken in this field, discussing possible new future directions of research for each of its different individual areas. Copyright © 2004 John Wiley & Sons, Ltd.     

Generating transition probabilities to support model‐based software testing

Markov chain usage models support test planning, test automation, and analysis of test results. In practice, transition probabilities for Markov chain usage models are often specified using a cycle of assigning, verifying, and revising specific values for individual transition probabilities. For large systems, such an approach can be difficult for a variety of reasons. We describe an improved approach that represents transition probabilities by explicitly preserving the information concerning test objectives and the relationships between transition probabilities in a format that is easy to maintain and easy to analyze. Using mathematical programming, transition probabilities are automatically generated to satisfy test management objectives and constraints. A more mathematical treatment of this approach is given in References [ 1 ] (Poore JH, Walton GH, Whittaker JA. A constraint‐based approach to the representation of software usage models. Information and SoftwareTechnology 2000; at press) and [ 2 ] (Walton GH. Generating transition probabilities for Markov chain usage models. PhD Thesis, University of Tennessee, Knoxville, TN, May 1995.). In contrast, this paper is targeted at the software engineering practitioner, software development manager, and test manager. This paper also adds to the published literature on Markov chain usage modeling and model‐based testing by describing and illustrating an iterative process for usage model development and optimization and by providing some recommendations for embedding model‐based testing activities within an incremental development process. Copyright © 2000 John Wiley & Sons, Ltd.     

Towards a goal‐driven approach to action selection in self‐adaptive software

Self‐adaptive software is a closed‐loop system, since it continuously monitors its context (i.e. environment) and/or self (i.e. software entities) in order to adapt itself properly to changes. We believe that representing adaptation goals explicitly and tracing them at run‐time are helpful in decision making for adaptation. While goal‐driven models are used in requirements engineering, they have not been utilized systematically yet for run‐time adaptation. To address this research gap, this article focuses on the deciding process in self‐adaptive software, and proposes the Goal‐Action‐Attribute Model (GAAM). An action selection mechanism, based on cooperative decision making, is also proposed that uses GAAM to select the appropriate adaptation action(s). The emphasis is on building a light‐weight and scalable run‐time model which needs less design and tuning effort comparing with a typical rule‐based approach. The GAAM and action selection mechanism are evaluated using a set of experiments on a simulated multi‐tier enterprise application, and two sample ordinal and cardinal action preference lists. The evaluation is accomplished based on a systematic design of experiment and a detailed statistical analysis in order to investigate several research questions. The findings are promising, considering the obtained results, and other impacts of the approach on engineering self‐adaptive software. Although, one case study is not enough to generalize the findings, and the proposed mechanism does not always outperform a typical rule‐based approach, less effort, scalability, and flexibility of GAAM are remarkable. Copyright © 2011 John Wiley & Sons, Ltd.     

It really does matter how you normalize the branch distance in search‐based software testing

The use of search algorithms for test data generation has seen many successful results. For structural criteria like branch coverage, heuristics have been designed to help the search. The most common heuristic is the use of approach level (usually represented with an integer) to reward test cases whose executions get close (in the control flow graph) to the target branch. To solve the constraints of the predicates in the control flow graph, the branch distance is commonly employed. These two measures are linearly combined. Since the approach level is more important, the branch distance is normalized, often in the range [0, 1]. In this paper, different types of normalizing functions are analyzed. The analyses show that the one that is usually employed in the literature has several flaws. The paper presents a different normalizing function that is very simple and does not suffer from these limitations. Empirical and analytical analyses are carried out to compare these two functions. In particular, their effect is studied on commonly used search algorithms, such as Hill Climbing, Simulated Annealing and Genetic Algorithms. Copyright © 2011 John Wiley & Sons, Ltd.     

Software release management for component‐based software

Software release management is the process through which software is made available to and obtained by its users. Until now, this process has been relatively straightforward. However, the emergence of component‐based software is complicating software release management. Increasingly, software is constructed via the assembly of pre‐existing, independently produced, and independently released components. Both developers and users of such software are affected by these complications. Developers need to accurately document the complex and changing dependencies among the components constituting the software. Users must be involved in locating, retrieving, and assembling components in order to appropriately bring the software into their particular environment. In this paper, we introduce the problem of release management for component‐based software and discuss SRM, a prototype software release management tool we have developed that supports both developers and users in the software release management process. Copyright © 2002 John Wiley & Sons, Ltd.     

HARTEX—a safe real‐time kernel for distributed computer control systems

A hard real‐time kernel is presented for distributed computer control systems (DCCS), highlighting a number of novel features, such as integrated scheduling of hard and soft real‐time tasks as well as tasks and resources; high‐performance time management supporting safe DCCS operation in a hard real‐time environment; synchronization and communication featuring event notification via vector semaphores and transparent communication through implicit (content‐oriented) message addressing. Conventional queues have been substituted by Boolean vectors and vector processing techniques throughout the kernel, resulting in efficient and highly deterministic behaviour, which is characterized by very low overhead and constant execution time of kernel operations, independent of the number of tasks involved. Copyright © 2001 John Wiley & Sons, Ltd.     

Verification of real‐time systems design

The main objective of this paper is to present an approach to accomplish verification in the early design phases of a system, which allows us to make the system verification easier, specifically for those systems with timing restrictions. For this purpose we use RT‐UML sequence diagrams in the design phase and we translate these diagrams into timed automata for performing the verification by using model checking techniques. Specifically, we use the Object Management Group's UML Profile for Schedulability, Performance, and Time and from the specifications written using this profile we obtain the corresponding timed automata. The ‘RT‐UML Profile’ is used in conjunction with a very well‐known tool to perform validation and verification of the timing needs, namely, the UPPAAL tool, which is used to simulate and analyze the behaviour of real‐time dynamic systems described by timed automata. Copyright © 2009 John Wiley & Sons, Ltd.     

基于关键字驱动的自动化测试研究与实现

为有效解决目前回归测试测试效率低以及手工测试操作复杂的问题,研究了自动化测试技术的原理和方法,提出了一种基于关键字驱动的自动化测试方法;介绍了Web自动化测试实施的过程和自动化测试框架的主要功能.在遵从保证产品质量和提高测试效率的原则基础上,对系统进行自动化测试的可行性分析,再进行自动化测试开发工具的选择、制作Demo和自动化测试框架开发,实现了以关键字驱动为核心,Ruby为开发语言的自动化测试框架.实验结果表明,该方法提高了回归测试效率和测试系统的可移植性.     

Exploring risk‐based testing and its implications

If software cannot be tested exhaustively, it must be tested selectively. But, on what should selection be based in order to maximize test effectiveness? It seems sensible to concentrate on the parts of the software where the risks are greatest, but what risks should be sought and how can they be identified and analysed? ‘Risk‐based testing’ is a term in current use, for example in an accredited test‐practitioner's course syllabus, but there is no broadly accepted definition of the phrase and no literature or body of knowledge to underpin the subject implied by it. Moreover, there has so far been no suggestion that it requires an understanding of the subject of risk. This paper examines what is implied by risk‐based testing, shows that its practice requires an understanding of risk, and points to the need for research into the topic and the development of a body of knowledge to underpin it. Copyright © 2004 John Wiley & Sons, Ltd.