Model-based testing for concurrent systems: unfolding-based test selection

Model-based testing has mainly focused on models where concurrency is interpreted as interleaving (like the ioco theory for labeled transition systems), which may be too coarse when one wants concurrency to be preserved in the implementation. In order to test such concurrent systems, we choose to use Petri nets as specifications and define a concurrent conformance relation named co-ioco. We present a test generation algorithm based on Petri net unfolding able to build a complete test suite w.r.t our co-ioco conformance relation. In addition, we propose several coverage criteria that allow to select finite prefixes of an unfolding in order to build manageable test suites.     

Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today??s testing approaches for SPLs validate products individually using different kinds of reuse techniques for testing. Because of their reusability and adaptability capabilities, model-based approaches are suitable to describe variability and are therefore frequently used for implementation and testing purposes of SPLs. Due to the enormous number of possible products, individual product testing becomes more and more infeasible. Pairwise testing offers one possibility to test a subset of all possible products. However, according to the best of our knowledge, there is no contribution discussing and rating this approach in the SPL context. In this contribution, we provide a mapping between feature models describing the common and variable parts of an SPL and a reusable test model in the form of statecharts. Thereby, we interrelate feature model-based coverage criteria and test model-based coverage criteria such as control and data flow coverage and are therefore able to discuss the potentials and limitations of pairwise testing. We pay particular attention to test requirements for feature interactions constituting a major challenge in SPL engineering. We give a concise definition of feature dependencies and feature interactions from a testing point of view, and we discuss adequacy criteria for SPL coverage under pairwise feature interaction testing and give a generalization to the T-wise case. The concept and implementation of our approach are evaluated by means of a case study from the automotive domain.     

Model-based testing for real

Model-based testing relies on abstract behavior models for test case generation. These models are abstractions, i.e., simplifications. For deterministic reactive systems, test cases are sequences of input and expected output. To bridge the different levels of abstraction, input must be concretized before being applied to the system under test. The systems output must then be abstracted before being compared to the output of the model.The concepts are discussed along the lines of a feasibility study, an inhouse smart card case study. We describe the modeling concepts of the CASE tool AutoFocus and an approach to model-based test case generation that is based on symbolic execution with Constraint Logic Programming.Different search strategies and algorithms for test case generation are discussed. Besides validating the model itself, generated test cases were used to verify the actual hardware with respect to these traces.     

Model-based software diagnosis

Software development is a complex process in which it is all too easy to introduce errors or faults that can be extremely difficult to identify . However , there has been little research into the use of intelligent diagnostic techniques for software ( as compared to hardware , for example ). Therefore , applying such techniques to software is potentially extremely useful . In many cases the presence of a bug is only identified when an invalid result is observed . However , the task of identifying the software elements that may have contributed to this result can be a time - consuming and tedious task , but must be performed before the actual root cause can be identified and a repair effected . This operation is akin to fault localization within hardware diagnosis and has been the subject of extensive research within the model - based community . This article presents a technique that minimizes the tedious task of fault localization with the software system , leaving the developer to concentrate on identifying the root cause and remedial action . In particular , we treat diagnosis as a trial with witnesses for the prosecution and defense . The result is a diagnostic trial that uses the source code of a system as its model and knowledge of valid and invalid results ( victims ) to identify a set of suspects for a developer to investigate further .     

Model-based software synthesis

The knowledge-representation and compilation techniques used in a model-based, automatic software synthesis environment are discussed. The environment was used to build Caddmus, a system with more than 250 cooperating processes. The real-time execution environment automatically generates a macro-dataflow computation from declarative models. Central to the approach is the Multigraph Architecture, which provides the framework for model-based synthesis in real-time, parallel-computing environments. Application of Caddmus to analysis of all data related to testing new and redesigned turbine engines is described     

Model-based testing of software and systems: recent advances and challenges

Model-based testing is focused on testing techniques which rely on the use of models. The diversity of systems and software to be tested implies the need for research on a variety of models and methods for test automation. We briefly review this research area and introduce several papers selected from the 22nd International Conference on Testing Software and Systems (ICTSS).     

Pairwise testing for software product lines: comparison of two approaches

Software Product Lines (SPL) are difficult to validate due to combinatorics induced by variability, which in turn leads to combinatorial explosion of the number of derivable products. Exhaustive testing in such a large products space is hardly feasible. Hence, one possible option is to test SPLs by generating test configurations that cover all possible t feature interactions (t-wise). It dramatically reduces the number of test products while ensuring reasonable SPL coverage. In this paper, we report our experience on applying t-wise techniques for SPL with two independent toolsets developed by the authors. One focuses on generality and splits the generation problem according to strategies. The other emphasizes providing efficient generation. To evaluate the respective merits of the approaches, measures such as the number of generated test configurations and the similarity between them are provided. By applying these measures, we were able to derive useful insights for pairwise and t-wise testing of product lines.     

Risk management for software projects

There is little to instruct software project managers on how to handle risk in a way that ensures the success of contingency planning and avoids crisis. This seven-step procedure describes how to identify risk factors, calculate their probability and effect on a project, and plan for and conduct risk management     

Model-based testing leveraged for automated web tests

Software Quality Journal - Agile methods and their practices have fostered the widespread presence of automated test cases. Such test cases have been successfully and extensively adopted to test...     

一种面向AADL架构的模型测试方法

鉴于模型在软件系统开发中日趋重要的地位和AADL模型在嵌入式软件建模中的良好应用前景,为了在嵌入式软件系统开发前期保证AADL模型的质量,提出了一种基于模型测试的AADL架构验证方法;该方法应用马尔可夫链描述AADL架构的行为,然后根据得到的马尔可夫链模型以及系统设计要求标准生成相应的测试用例和测试预言,并通过测试用例执行输出和期望值的比较判断AADL模型的正确性,实现对系统AADL模型的测试;最后通过案例分析证明了该方法的有效性。     

Curating GitHub for engineered software projects

Software forges like GitHub host millions of repositories. Software engineering researchers have been able to take advantage of such a large corpora of potential study subjects with the help of tools like GHTorrent and Boa. However, the simplicity in querying comes with a caveat: there are limited means of separating the signal (e.g. repositories containing engineered software projects) from the noise (e.g. repositories containing home work assignments). The proportion of noise in a random sample of repositories could skew the study and may lead to researchers reaching unrealistic, potentially inaccurate, conclusions. We argue that it is imperative to have the ability to sieve out the noise in such large repository forges. We propose a framework, and present a reference implementation of the framework as a tool called reaper, to enable researchers to select GitHub repositories that contain evidence of an engineered software project. We identify software engineering practices (called dimensions) and propose means for validating their existence in a GitHub repository. We used reaper to measure the dimensions of 1,857,423 GitHub repositories. We then used manually classified data sets of repositories to train classifiers capable of predicting if a given GitHub repository contains an engineered software project. The performance of the classifiers was evaluated using a set of 200 repositories with known ground truth classification. We also compared the performance of the classifiers to other approaches to classification (e.g. number of GitHub Stargazers) and found our classifiers to outperform existing approaches. We found stargazers-based classifier (with 10 as the threshold for number of stargazers) to exhibit high precision (97%) but an inversely proportional recall (32%). On the other hand, our best classifier exhibited a high precision (82%) and a high recall (86%). The stargazer-based criteria offers precision but fails to recall a significant portion of the population.     

Model-based reasoning: a principled approach for software engineering

The software engineering industry suffers from almost unmanageable complexity both in the products it produces and in the processes of production. One of the current shortcomings in the software production process is the weakness of the models used. This paper makes observations on the role of knowledge in engineering and examines the central role of models and simulation. We develop an argument for the application of certain new forms of modelling methods in software engineering in order to impose more discipline and give a principled framework for building models that can support the software life-cycle. The concept of a model is examined in depth and different characteristics and types of model are defined. This introduces the relatively new concept of qualitative models and their use in the field known as model-based reasoning. Unlike previous knowledge-based methods, model-based reasoning has several important advantages. Although very few model-based software projects exist, we illustrate how this approach can be developed by drawing on applications from traditional engineering. It is argued that, because qualitative modelling offers great power for addressing the issue of complexity, such models have considerable potential as high-level abstractions of software products. These could form the core of tools for the management and support of the software development process through the whole product life-cycle.     

Model-based cache-aware dispatching of object-oriented software for multicore systems

In recent years, processor technology has evolved towards multicore processors, which include multiple processing units (cores) in a single package. Those cores, having their own private caches, often share a higher level cache memory dedicated to each processor die. This multi-level cache hierarchy in multicore processors raises the importance of cache utilization problem. Assigning parallel-running software components with common data to processor cores that do not share a common cache increases the number of cache misses. In this paper we present a novel approach that uses model-based information to guide the OS scheduler in assigning appropriate core affinities to software objects at run-time. We build graph models of software and cache hierarchies of processors and devise a graph matcher algorithm that provides mapping between these two graphs. Using this mapping we obtain candidate core sets that each software object can be affiliated with at run-time. These affiliations are determined based on the idea that software components that have the potential to share common data at run-time should run on cores that share a common cache. We also develop an object dispatcher algorithm that keeps track of object affiliations at run-time and dispatches objects by using the information from the compile-time graph matcher. We apply our approach on design pattern implementations and two different application program running on servers using CFS scheduling. Our results show that cache-aware dispatching based on information obtained from software model, decreases number of cache misses significantly and improves CFS’ scheduling performance.     

Model-based software configuration: patterns and languages

The common presupposition of enterprise systems (ES) is that they lead to significant efficiency gains. However, this is only the case for well-implemented ES that are well-aligned with the organisation. The list of ES implementation failures is significant which is partly attributable to the insufficiently addressed fundamental problem of adapting an ES efficiently. As long as it is not intuitively possible to configure an ES, this problem will prevail because organisations have a non-generic character. A solution to this problem consists in re-thinking current practices of ES provision. This paper proposes a new approach based on configurable process models, which reflect ES functionalities. We provide in this paper a taxonomy of situations that can occur from a business perspective during process model configuration. This taxonomy is represented via so-called semantic configuration patterns. In the next step, we discuss so-called syntactic configuration patterns. This second type of configuration patterns implements the semantic configuration patterns for specific modelling techniques. We chose two popular process modelling languages in order to illustrate our approach.     

A software development process for small projects

The authors' software development process integrates portions of an iterative, incremental process model with a quality assurance process and a measurement process that is used for process improvement. Their process aims to produce high-quality and timely results with less overhead     

基于模型的软件安全预测与分析

为了有效表示和分析软件中存在的安全缺陷和隐患,基于模型的软件安全分析技术采用多层次建模技术实现安全特征的描述,在评价软件及软件组件间安全性的过程中提出软件安全预测技术.通过分析软件组成成分之间的关联度获得相关的安全距离,在此基础之上生成安全依赖图,最后根据安全依赖图进行安全预测和分析.基于模型的安全分析技术能够针对可能存在的安全隐患给出预测和分析,为软件的测试和维护提供依据和手段.     

Web-based issue tracking for large software projects

Many problems are found and fixed during the development of a software system. The Project Issue Tracking System toolkit, a Web-based issue-management tool, can be used to organize issue reports during development and to communicate with different project teams around the world. The Project Issue Tracking System (PITS) is a tool that supports the IV&V effort for two major NASA projects: the Earth Observation System Data and Information System (EOSDIS) and the Earth Science Data and Information System (ESDIS). The EOSDIS IV&V effort teams several companies and organizations at several sites with Intermetrics, Inc, serving as the lead contractor. We examine the PITS Web-based mechanisms for tracking issue reports