Tools: An environment for time-shared computing and programming

The Yale Tools environment provides integrated modern computing facilities to a mixed group of programmers and users on time-shared machines. Hundreds of people use the environment every day for systems and applications programming, word-processing and university business. The success of the environment is in its integration and engineering detail. Building such a system without a high-level language and a run-time library would have been impossible.     

Global software testing under deadline pressure: Vendor-side experiences

ContextIn the era of globally-distributed software engineering, the practice of global software testing (GST) has witnessed increasing adoption. Although there have been ethnographic studies of the development aspects of global software engineering, there have been fewer studies of GST, which, to succeed, can require dealing with unique challenges.ObjectiveTo address this limitation of existing studies, we conducted, and in this paper, report the findings of, a study of a vendor organization involved in one kind of GST practice: outsourced, offshored software testing.MethodWe conducted an ethnographically-informed study of three vendor-side testing teams over a period of 2 months. We used methods, such as interviews and participant observations, to collect the data and the thematic-analysis approach to analyze the data.FindingsOur findings describe how the participant test engineers perceive software testing and deadline pressures, the challenges that they encounter, and the strategies that they use for coping with the challenges. The findings reveal several interesting insights. First, motivation and appreciation play an important role for our participants in ensuring that high-quality testing is performed. Second, intermediate onshore teams increase the degree of pressure experienced by the participant test engineers. Third, vendor team participants perceive productivity differently from their client teams, which results in unproductive-productivity experiences. Lastly, participants encounter quality-dilemma situations for various reasons.ConclusionThe study findings suggest the need for (1) appreciating test engineers’ efforts, (2) investigating the team structure’s influence on pressure and the GST practice, (3) understanding culture’s influence on other aspects of GST, and (4) identifying and addressing quality-dilemma situations.     

Automatic testing environment for multi-core embedded software—ATEMES

Software testing during the development process of embedded software is not only complex, but also the heart of quality control. Multi-core embedded software testing faces even more challenges. Major issues include: (1) how demanding efforts and repetitive tedious actions can be reduced; (2) how resource restraints of embedded system platform such as temporal and memory capacity can be tackled; (3) how embedded software parallelism degree can be controlled to empower multi-core CPU computing capacity; (4) how analysis is exercised to ensure sufficient coverage test of embedded software; (5) how to do data synchronization to address issues such as race conditions in the interrupt driven multi-core embedded system; (6) high level reliability testing to ensure customer satisfaction. To address these issues, this study develops an automatic testing environment for multi-core embedded software (ATEMES). Based on the automatic mechanism, the system can parse source code, instrument source code, generate testing programs for test case and test driver, support generating primitive, structure and object types of test input data, multi-round cross-testing, and visualize testing results. To both reduce test engineer's burden and enhance his efficiency when embedded software testing is in process, this system developed automatic testing functions including unit testing, coverage testing, multi-core performance monitoring. Moreover, ATEMES can perform automatic multi-round cross-testing benchmark testing on multi-core embedded platform for parallel programs adopting Intel TBB library to recommend optimized parallel parameters such as pipeline tokens. Using ATEMES on the ARM11 multi-core platform to conduct testing experiments, the results show that our constructed testing environment is effective, and can reduce burdens of test engineer, and can enhance efficiency of testing task.     

Design and implementation of an adaptable software environment

In any major software project, a large volume of project-related information must be maintained and manipulated throughout the software life-cycle. Often this information is scattered, in different media ranging from scraps of papers to program listings, poorly cross-referenced, and inaccessible. TRIAD, an adaptable, integrated software development environment, has been designed to integrate and manage all this information in an organized and structured manner.

The TRIAD tool-box environment supports different methodologies to guide the users during development and maintenance of project related information. The environment can be customized by selecting a set of forms which enforce a particular methodology. These forms help develop, integrate, and maintain project information. Forms for a methodology can be tuned to reflect a project-team's experience. When customized by the tuned methodology, TRIAD can provide better project oriented support.

TRIAD uses grammar forms as its organizational model. This paper will discuss the advantages of using grammar forms in the modeling and implementation of an adaptable software information editor, within the TRIAD tool-box environment.     

A requirements engineering environment within a tightly integrated SDE

This paper describes the outside functionality of an RE environment within an integrated software development environment. Furthermore, an integrator tool for the transition to software system architecture modelling is presented. The tools discussed are editors, analysers, executors, monitors, and integration tools of different characteristics for horizontal integration (within RE) and vertical integration (to architecture modelling). All tools are tightly integrated and work incrementally, therefore allowing different forms of construction and modification processes and giving substantial support.     

An initial industrial evaluation of interactive search-based testing for embedded software

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so.Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies.For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system.This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search.In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ‘domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions.The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently.     

Model-based testing of global properties on large-scale distributed systems

ContextLarge-scale distributed systems are becoming commonplace with the large popularity of peer-to-peer and cloud computing. The increasing importance of these systems contrasts with the lack of integrated solutions to build trustworthy software. A key concern of any large-scale distributed system is the validation of global properties, which cannot be evaluated on a single node. Thus, it is necessary to gather data from distributed nodes and to aggregate these data into a global view. This turns out to be very challenging because of the system’s dynamism that imposes very frequent changes in local values that affect global properties. This implies that the global view has to be frequently updated to ensure an accurate validation of global properties.ObjectiveIn this paper, we present a model-based approach to define a dynamic oracle for checking global properties. Our objective is to abstract relevant aspects of such systems into models. These models are updated at runtime, by monitoring the corresponding distributed system.MethodWe conduce real-scale experimental validation to evaluate the ability of our approach to check global properties. In this validation, we apply our approach to test two open-source implementations of distributed hash tables. The experiments are deployed on two clusters of 32 nodes.ResultsThe experiments reveal an important defect on one implementation and show clear performance differences between the two implementations. The defect would not be detected without a global view of the system.ConclusionTesting global properties on distributed software consists of gathering data from different nodes and building a global view of the system, where properties are validated. This process requires a distributed test architecture and tools for representing and validating global properties. Model-based techniques are an expressive mean for building oracles that validate global properties on distributed systems.     

嵌入式实时系统软件测试实践

软件质量是软件的生命,软件测试是保证软件产品质量的重要手段。本文从嵌入式系统的特点出发,阐述了嵌入式系统软件测试的实现方法。并结合某无线通信电台软件测试的实例,详细叙述了如何利用Testbed和RTInsight搭建嵌入式实时系统的软件测试平台,说明了在此平台上进行软件覆盖率测试和性能分析的全过程,同时给出了部分测试结果。     

嵌入式实时系统软件测试实践

软件质量是软件的生命,软件测试是保证软件产品质量的重要手段。本文从嵌入式系统的特点出发,阐述了嵌入式系统软件测试的实现方法。并结合某无线通信电台软件测试的实例,详细叙述了如何利用Testbed和RTInsight搭建嵌入式实时系统的软件测试平台,说明了在此平台上进行软件覆盖率测试和性能分析的全过程,同时给出了部分测试结果。     

Testing and verification aspects of Pascal-like languages

This paper addresses aspects of programming language design that affect the ease with which programs written in a language can be subjected to systematic testing and/or program verification. The discussion focuses of Pascal and on several languages that have been derived primarily from Pascal, particularly Euclid and PLAIN. Specific language issues addressed include translation-time checking, program readability, flow of control, support for program modularity, data flow, and program immutability. The relative ease of validating such programs is then determined by the style in which the programs are written. The paper presents some guidelines for writing programs in Pascal-like languages for testability and verifiability.     

An experimental study of adaptive testing for software reliability assessment

Adaptive testing is a new form of software testing that is based on the feedback and adaptive control principle and can be treated as the software testing counterpart of adaptive control. Our previous work has shown that adaptive testing can be formulated and guided in theory to minimize the variance of an unbiased software reliability estimator and to achieve optimal software reliability assessment. In this paper, we present an experimental study of adaptive testing for software reliability assessment, where the adaptive testing strategy, the random testing strategy and the operational profile based testing strategy were applied to the Space program in four experiments. The experimental results demonstrate that the adaptive testing strategy can really work in practice and may noticeably outperform the other two. Therefore, the adaptive testing strategy can serve as a preferable alternative to the random testing strategy and the operational profile based testing strategy if high confidence in the reliability estimates is required or the real-world operational profile of the software under test cannot be accurately identified.     

An adaptive and trustworthy software testing framework on the grid

Grid computing, which is characterized by large-scale sharing and collaboration of dynamic distributed resources has quickly become a mainstream technology in distributed computing and is changing the traditional way of software development. In this article, we present a grid-based software testing framework for unit and integration test, which takes advantage of the large-scale and cost-efficient computational grid resources to establish a testbed for supporting automated software test in complex software applications. Within this software testing framework, a dynamic bag-of-tasks model using swarm intelligence is developed to adaptively schedule unit test cases. Various high-confidence computing mechanisms, such as redundancy, intermediate value checks, verification code injection, and consistency checks are employed to verify the correctness of each test case execution on the grid. Grid workflow is used to coordinate various test units for integration test. Overall, we expect that the grid-based software testing framework can provide efficient and trustworthy services to significantly accelerate the testing process with large-scale software testing.

Embedded software verification in hardware–software codesign

Concurrent Embedded Real-Time Software (CERTS) is intrinsically different from traditional, sequential, independent, and temporally unconstrained software. The verification of software is more complex than hardware due to inherent flexibilities (dynamic behavior) that incur a multitude of possible system states. The verification of CERTS is all the more difficult due to its concurrency and embeddedness. The work presented here shows how the complexity of CERTS verification can be reduced significantly through answering common engineering questions such as when, where, and how one must verify embedded software. First, a new Schedule-Verify-Map strategy is proposed to answer the when question. Second, verification under system concurrency is proposed to answer the where question. Finally, a complete symbolic model checking procedure is proposed for CERTS verification. Several application examples illustrate the usefulness of our technique in increasing verification scalability.     

基于MVC模式的嵌入式软件测试开发环境设计

嵌入式软件仿真测试开发环境(ESTDE,Embedded Software Testing Development Environment)是嵌入式软件仿真测试平台(ESSTP,Embedded Software Simulation Testing Platform)的重要组成部分,是有效实现嵌入式软件实时、闭环测试的基础。MVC(Model-View-Controller)是一种开发交互式软件系统的典型体系结构模式,在分析和研究ESTDE功能、组成和工作原理的基础上,将MVC应用于ESTDE的设计当中,不仅保证了ESSTP的成功开发,而且提高了系统的可复用性和可适应性。     

An overview of TEDIUM

TEDIUM was developed to manage the entire life cycle of a software application from initial analysis through final retirement. In this sense, TEDIUM is different from a Programming Support Environment (PSE), which applies automated tools to aid in program implementation and project control within the context of an established process model. In the case of the PSE, computer technology is used to aid in the management of large, complex software projects. TEDIUM, on the other hand, uses its own software process model and is constrained to moderate-sized information system applications. After a brief history of the environment, the author offers an evaluation of TEDIUM, and some observations that may be of value for other development paradigms or application classes.     

面向实时嵌入式机载软件的测试技术研究

文章对实时嵌入式机载软件的测试方法进行了研究，地对机载计算软件的时空约束，面向机器和容错计算每一系列特点，提出了一个适用于实时嵌入式机载系统的ＲＥＡＳＴ软件测试模型，并地该模型实现中的４个核心技术进行了简要的阐明，最后，介绍了以上软件测试方法在实际工程项目中的应用情况。     

Model checking, testing and verification working together

We present a symbolic model checking approach that allows verifying a unit of code, e.g., a single procedure or a collection of procedures that interact with each other. We allow temporal specifications that assert over both the program counters and the program variables. We decompose the verification into two parts: (1) a search that is based on the temporal behavior of the program counters, and (2) the formulation and refutation of a path condition, which inherits conditions constraining the program variables from the temporal specification. This verification approach is modular, as we do not require that all the involved procedures are provided. Furthermore, we do not request that the code is based on a finite domain. The presented approach can also be used for automating the generation of test cases for unit testing.A preliminary version of the paper, with the title Unit Checking: Symbolic Model Checking for a Unit of Code appears in the Lecture Notes in Computer Science volume 2772, Verification– Theory and Practice, celebrating Zohar Manna’s 64th birthdayThis research was partially supported by US Army Research Office Grant number DAAAD19-01-1-0473This research was partially supported by Subcontract UTA03-031 to The University of Warwick under University of Texas at Austin’s prime National Science Foundation Grant #CCR-0205483Received February 2004Revised September 2004 and April 2005Accepted April 2005 by M. Leuschel and D. J. Cooke     

企业机房建设与管理

作为企业,如何建设适合新时代的机房？如何科学管理机房？如何发挥机房的作用？如何使机房建设与管理更加科学、规范、高效益？这些都是企业迫切需要解决的问题。本文结合自己单位目前机房建设与管理现状和近几年来的操作管理经验,提出一些个人看法,供大家参考。