Scalable System-level CTI Testing through Lightweight Coarse-grained Coordination

We propose a solution to the problem of system-level testing of functionally complex communication systems based on lightweight coordination. The enabling aspect is here the adoption of a coarse-grained approach to test design, which is central to the scalability of the overall testing environment. This induces an understandable modelling paradigm of system-wide test cases which is adequate for the needs and requirements of industrial test engineers. The approach is coarse-grained in the sense that it renounces a detailed model of the system functionality (which would be unfeasible in the considered industrial setting). The coordination is lightweight in the sense that it allows a programming-free definition of system-level behaviours (in this case complex test cases) based on the coarse models of the functionalities. These features enable test engineers to graphically design complex test cases, which, in addition, can even be automatically checked for their intended purposes via model checking.     

基于用户会话的Web应用性能测试方法的研究

近年来,Web应用成为互联网时代基础设施中最重要的组成部分。随着互联网技术的不断进步以及使用用户数量的不断增长,Web应用也越发变得复杂。为了保证Web应用性能测试的有效性和真实性,提出一种利用服务器日志中的用户会话信息、结合经过序列匹配算法改进的层次凝聚算法来自动产生Web应用性能测试脚本的方法。提出的方法通过从服务器日志中提取出用户会话信息,分析用户真实的访问模式,通过聚类算法将同类型的用户访问模式进行聚集,自动产生相应的测试请求序列,生成测试集。相比于传统的基于录制/回访的测试方法,基于用户会话的性能测试方法通过对用户会话进行聚类分析,还原实际的用户访问模式,减少了设计测试用例时的人工参与,保证了测试结果的有效性和真实性。     

Dynamic systems modelling and development of robotic applications in a visual programming environment

In this paper we describe our approach to modelling dynamic systems behaviour and its application to an industrial environment. A complete programming environment is described through an account of the rationale and modus operandi of the system. Our objective is to help managers, designers, engineers and software developers to understand dynamic systems behaviour through modelling. Examples of systems modelling are given and an industrial application of the programming environment for automatic food handling involving machine vision, virtual sensors and robot manipulation control is described.     

Database design with user-definable modelling concepts

Modelling is an integral part of engineering processes. Consequently, database design for engineering applications should take into account the modelling concepts used by engineers. On the other hand, these applications exhibit a wide diversity of modelling concepts. Rather than consolidating these into one single semantic data model one should aim for correspondingly specialized semantic models. This paper takes a constructive approach to developing such specialized models by proposing an Extensible Semantic Model (ESM) as the basis for declaring specialized semantic data models. The paper introduces a computerized environment for database design based on an ESM, and discusses the consequences of the ESM for a number of design tools: the need for a formal definition of the notion of modelling concept in order to have reliable and precise foundation for the extensions, declarative techniques for quickly introducing graphical representations for new concepts and for using them during schema design, conceptual-level test data generation for a designer-oriented evaluation of designs, and optimization techniques to control the wide latitude in mapping a conceptual schema to a logical schema. First experiences seem to point to considerable productivity gains during database design.     

Performance modelling of a network processor using POOSL

The increasing complexity of innovative real-time hardware/software systems forced industry to consider system-level design methods. Before actually implementing a system with hardware and software components, system-level design methods enable analysing the performance of different design alternatives that realise the required functionality. In order to develop performance models early in the design process, the parallel object-oriented specification language (POOSL) can be used. POOSL is an expressive modelling language for analysing complex real-time distributed hardware/software systems. Being equipped with a formal semantics, POOSL ensures unambiguous execution of models and proper application of performance analysis techniques. This paper discusses the use of POOSL for analysing the performance of a network processor. A network processor consists of components that perform their behaviour in a synchronously concurrent way, whereas POOSL is based on an asynchronous modelling paradigm. In this paper, we illustrate that constructing abstract models of synchronous systems for the purpose of performance analysis may benefit from an asynchronous modelling approach.     

Flying Together: Modelling Air Mission Teams

The problem of modelling air missions is part of a larger problem—simulating possible war-like scenarios in the air, sea, and on land. In modelling such military systems one is required to model the behaviour of various actors and the resources that are available to them. One aspect of this problem is the modelling of a group of actors as a team and then modelling the coordinated behaviour of such a team to achieve a joint goal.In the domain of air mission modelling the actors are pilots who control aircraft and their behaviour is referred to as tactics. In this paper we present the approach we adopted in modelling teams and team tactics as part of the development of the Smart Whole AiR Mission Model (SWARMM) for the Air Operations Division of the Australian Defence Science and Technology Organization. In our approach teams are composed of sub-teams and adopt organizational structures. Such structures define the responsibilities of the sub-teams towards the mission to be achieved as well as towards the control and coordination of the sub-teams. We also describe how communication is used when adopting a variety of control and coordination strategies and how one could reason about the choice of organizational structures for a given mission and situation.     

An initial industrial evaluation of interactive search-based testing for embedded software

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so.Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies.For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system.This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search.In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ‘domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions.The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently.     

Illustrative Multivariate Visualization for Geological Modelling

In this paper, we present a novel illustrative multivariate visualization for geological modelling to assist geologists and reservoir engineers in visualizing multivariate datasets in superimposed representations, in contrast to the single‐attribute visualizations supported by commercial software. Our approach extends the use of decals from a single surface to 3D irregular grids, using the layering concept to represent multiple attributes. We also build upon prior work to augment the design and implementation of different geological attributes (namely, rock type, porosity, and permeability). More specifically, we propose a new sampling strategy to generate decals for porosity on the geological grid, a hybrid visualization for permeability which combines 2D decals and 3D ellipsoid glyphs, and a perceptually‐based design that allows us to visualize additional attributes (e.g., oil saturation) while avoiding visual interference between layers. Furthermore, our visual design draws from traditional geological illustrations, facilitating the understanding and communication between interdisciplinary teams. An evaluation by domain experts highlights the potential of our approach for geological modelling and interpretation in this complex domain.     

Event‐based modelling,analysis and testing of user interactions: approach and case study

With the growing complexity of computer‐based systems, their graphical user interfaces have also become more complex. Accordingly, the test and analysis process becomes more tedious and costly. This paper introduces a holistic view of fault modelling that is carried out as a complementary step to system modelling, enabling a scalability of the test process, and providing considerable potential for automation. Event‐based notions and tools are used to generate and select test cases systematically. The elements of the approach are illustrated and validated by a case study. This paper does not claim to introduce a novel theoretic approach; rather, it makes use of graph‐theoretic results for a practical and simple, but nevertheless powerful, view of modelling, analysis and testing of graphical user interfaces. Copyright © 2005 John Wiley & Sons, Ltd.     

Constraint programming for type inference in flexible model-driven engineering

Domain experts typically have detailed knowledge of the concepts that are used in their domain; however they often lack the technical skills needed to translate that knowledge into model-driven engineering (MDE) idioms and technologies. Flexible or bottom-up modelling has been introduced to assist with the involvement of domain experts by promoting the use of simple drawing tools. In traditional MDE the engineering process starts with the definition of a metamodel which is used for the instantiation of models. In bottom-up MDE example models are defined at the beginning, letting the domain experts and language engineers focus on expressing the concepts rather than spending time on technical details of the metamodelling infrastructure. The metamodel is then created manually or inferred automatically. The flexibility that bottom-up MDE offers comes with the cost of having nodes in the example models left untyped. As a result, concepts that might be important for the definition of the domain will be ignored while the example models cannot be adequately re-used in future iterations of the language definition process. In this paper, we propose a novel approach that assists in the inference of the types of untyped model elements using Constraint Programming. We evaluate the proposed approach in a number of example models to identify the performance of the prediction mechanism and the benefits it offers. The reduction in the effort needed to complete the missing types reaches up to 91.45% compared to the scenario where the language engineers had to identify and complete the types without guidance.     

Aerodynamic shape optimization of supersonic aircraft configurations via an adjoint formulation on distributed memory parallel computers

This work describes the application of a control theory-based aerodynamic shape optimization method to the problem of supersonic aircraft design. A high fidelity computational fluid dynamics (CFD) algorithm modelling the Euler equations is used to calculate the aerodynamic properties of complex three-dimensional aircraft configurations. The design process is greatly accelerated through the use of both control theory and parallel computing. Control theory is employed to derive the adjoint differential equations whose solution allows for the evaluation of design gradient information at a fraction of the computational cost required by previous design methods. The resulting problem is then implemented in parallel using a domain decomposition approach, an optimized communication schedule, and the Message Passing Interface (MPI) Standard for portability and efficiency. In our earlier studies, the serial implementation of this design method, was shown to be effective for the optimization of airfoils, wings, wing–bodies, and complex aircraft configurations using both the potential equation and the Euler equations. In this work, our concern will be to extend the methodologies such that the combined capabilities of these new technologies can be used routinely and efficiently in an industrial design environment. The aerodynamic optimization of a supersonic transport configuration is presented as a demonstration test case of the capability. A particular difficulty of this test case is posed by the close coupling of the propulsion/airframe integration.     

Consistent Modelling of Users, Devices and Sensors in a Ubiquitous Computing Environment

This paper describes the use of an accretion-resolution user modelling representation to model people, places and objects. We explain the motivation for the key properties of the representation, especially those of particular importance for ubiquitous computing: firstly, for flexibility in interpreting the typically noisy and potentially conflicting evidence about users’ locations; secondly, to support users in scrutinising their user model, the processes that determine its contents and the way that it is used in the ubiquitous computing environment. A novel and important aspect of this work is our extension of the representation beyond modelling just users, using it also to represent the other elements such as devices, sensors, rooms and buildings. We illustrate our approach in terms of models we have been building for a system which enables users to gain personalised information about the sensors and services in a ubiquitous computing environment. We report experiments on the scalability and the management of inconsistency in modelling of location, based on accretion-resolution     

Testing interruptions in reactive systems

Modelling and testing of reactive systems with interruptions are discussed. These systems are commonly found in portable devices, where interruptions to a running application can be demanded at any time, due to concurrent execution of processes sharing a single resource, such as screen, as well as arrival of calls from network distributed services. Since the possible number of combinations of allowed interruptions is large, proper test case selection activities need to be performed. But, in order to systematically investigate and select test cases, it is fundamental to explicitly model interruption behaviour in a compositional way, avoiding the need for explicit enumeration. This work presents a strategy for testing interruptions in reactive systems that covers modelling for testing of systems with interruptions, generation and selection of sound test cases. The strategy is supported by the LTS-BT tool. Moreover, a formal model of an environment devoted to execution of test cases with interruptions is presented. Finally, a case study illustrates its applicability in the mobile phone application domain.     

A framework for evolution of modelling languages

In model-driven engineering, evolution is inevitable over the course of the complete life cycle of complex software-intensive systems and more importantly of entire product families. Not only instance models, but also entire modelling languages are subject to change. This is in particular true for domain-specific languages, whose language constructs are tightly coupled to an application domain.The most popular approach to evolution in the modelling domain is a manual process, with tedious and error-prone migration of artefacts such as instance models as a result. This paper provides a taxonomy for evolution of modelling languages and discusses the different evolution scenarios for various kinds of modelling artefacts, such as instance models, meta-models, and transformation models. Subsequently, the consequences of evolution and the required remedial actions are decomposed into primitive scenarios such that all possible evolutions can be covered exhaustively. These primitives are then used in a high-level framework for the evolution of modelling languages.We suggest that our structured approach enables the design of (semi-)automatic modelling language evolution solutions.     

A framework for evolution of modelling languages

In model-driven engineering, evolution is inevitable over the course of the complete life cycle of complex software-intensive systems and more importantly of entire product families. Not only instance models, but also entire modelling languages are subject to change. This is in particular true for domain-specific languages, whose language constructs are tightly coupled to an application domain.The most popular approach to evolution in the modelling domain is a manual process, with tedious and error-prone migration of artefacts such as instance models as a result. This paper provides a taxonomy for evolution of modelling languages and discusses the different evolution scenarios for various kinds of modelling artefacts, such as instance models, meta-models, and transformation models. Subsequently, the consequences of evolution and the required remedial actions are decomposed into primitive scenarios such that all possible evolutions can be covered exhaustively. These primitives are then used in a high-level framework for the evolution of modelling languages.We suggest that our structured approach enables the design of (semi-)automatic modelling language evolution solutions.     

Model-Based Design, Development and Validation for UAS Critical Software

This paper focuses on the major phases present in the development of critical software for UAS: design, development, testing and validation with flying experiments. A model-based approach is the backbone of all these development phases. The presented approach enables the researchers or engineers to work over the whole development cycle with the same tools, and produces an improvement over the classical design cycle. The model-based approach is illustrated with the development of guidance, navigation and control algorithms applied to rotary-wing UAVs. The paper discusses several implementation issues, including the integration of a hardware in the loop (HWIL) test environment within the model-based development cycle. Finally, several simulations and experimental results for this application are shown.     

A DFT Approach for Testing Embedded Systems Using DC Sensors

Today's consumer electronics must be portable, reliable at various operating environments, and power efficient. Thus, semiconductor manufacturers constantly upgrade their production technologies and incorporate intelligent circuit design techniques. With widespread advances in system integration techniques, manufacturers can bundle multiple functionalities onto a single chip, reducing the end product's form factor. However, with higher levels of integration and reduced pin count, test issues are becoming more critical. During high-volume production, variations in process parameters cause devices to vary significantly from their performance metrics, and test engineers have only limited test resources to perform at-speed testing. Generating diagnosis information is also challenging during product ramp-up, as very little information is available from the output pins about the different modules' functionalities. DFT seems to be the only viable solution in such a scenario. DFT can address various issues related to at-speed testing and high-speed test response capture by performing signal conditioning to more easily capture information at lower speeds. The authors present a method that uses embedded DC sensors at test observation nodes to simplify data capture and enhance test quality while performing at-speed tests during production testing. Experiments show that monitoring sensor outputs provides a very good estimate of complex, system-level specifications.     

Automated product line test case selection: industrial case study and controlled experiment

Automated test case selection for a new product in a product line is challenging due to several reasons. First, the variability within the product line needs to be captured in a systematic way; second, the reusable test cases from the repository are required to be identified for testing a new product. The objective of such automated process is to reduce the overall effort for selection (e.g., selection time), while achieving an acceptable level of the coverage of testing functionalities. In this paper, we propose a systematic and automated methodology using a feature model for testing (FM_T) to capture commonalities and variabilities of a product line and a component family model for testing (CFM_T) to capture the overall structure of test cases in the repository. With our methodology, a test engineer does not need to manually go through the repository to select a relevant set of test cases for a new product. Instead, a test engineer only needs to select a set of relevant features using FM_T at a higher level of abstraction for a product and a set of relevant test cases will be selected automatically. We evaluated our methodology via three different ways: (1) We applied our methodology to a product line of video conferencing systems called Saturn developed by Cisco, and the results show that our methodology can reduce the selection effort significantly; (2) we conducted a questionnaire-based study to solicit the views of test engineers who were involved in developing FM_T and CFM_T. The results show that test engineers are positive about adapting our methodology and models (FM_T and CFM_T) in their current practice; (3) we conducted a controlled experiment with 20 graduate students to assess the performance (i.e., cost, effectiveness and efficiency) of our automated methodology as compared to the manual approach. The results showed that our methodology is cost-effective as compared to the manual approach, and at the same time, its efficiency is not affected by the increased complexity of products.