AUTOSAR标准一致性测试研究

分析AUTOSAR一致性测试过程,采用eclipse、TTthree、μTTman等工具,基于TTCN-3测试系统架构,实现AUTOSAR一致性测试。针对简单信号灯系统,研究测试系统中测试套、编解码器和适配器的功能及相关实现方法,并验证采用TTCN-3进行AUTOSAR一致性测试的可行性。     

Standardized TTCN-3 specifications for SIP-ISUP/ISDN interworking testing

ETSI technical committee TISPAN currently funds the development of TTCN-3 network integration test suites using TTCN-3. Specialist task forces (STF) have already been initiated to implement and validate abstract test suites focusing on, e.g., SIP/ISUP and SIP/ISDN interworking units. In this paper, we report on our implementation approach and experiences made in the STF on testing interworking between SIP and ISDN/ISUP. Special attention has been given to the reuse of know-how and of existing TTCN-3 code available due to the existing SIP conformance test suite and the TTCN-3 test framework developed by ETSI for IPv6. Validation has been done using the test laboratory of a telecom network provider.     

Quantifying the evolution of TTCN-3 as a language

Ten years of maintenance, nine published revisions of the standards for the Testing and Test Control Notation version 3 (TTCN-3), more than 500 change requests since 2006, and 10 years of activity on the official TTCN-3 mailing list add up to a rich history, not unlike that of many successful Open Source Software (OSS) projects. In this article, we contemplate TTCN-3 in the context of software evolution and examine its history quantitatively. We mined the changes in the textual content of the standards, the data in change requests from the past 5 years, and the mailing list archives from the past 10 years. In addition, to characterize the use of the TTCN-3 we investigated the meta-data of the contributions at the TTCN-3 User Conference, and the use of language constructs in a large-scale TTCN-3 test suite. Based on these data sets, we first analyze the amount, density, and location of changes within the different parts of the standard. Then, we analyze the activity and focus of the user community and the maintenance team in both the change request management system and the official TTCN-3 mailing list. Finally, we analyze the distribution of contributions at the TTCN-3 User Conference across different topics over the past 8 years and construct use anomalies during the development of a large-scale test suite. Our findings indicate that the TTCN-3 is becoming increasingly stable as the overall change density and intensity, as well as the number of change requests are decreasing, despite the monotonous increase in the size of the standards.     

Test suite development with TTCN-3 libraries

This paper presents a methodology for test suite implementation based on Testing and test control notation (TTCN)-3 libraries which has been developed and successfully applied for the implementation of complex, standardized Internet Protocol version 6 (IPv6) conformance test suites at ETSI. It introduces the concept of a TTCN-3 library, discusses benefits and challenges, and examines the effect of some important TTCN-3 language constructs on reuse. Finally, it explains in detail a layered library design approach to test implementation in the context of a IPv6 test suite.     

TTCN-3 语言非侵入式调试技术

TTCN-3(Testing and Test Control Notation version 3)是一种形式化的测试描述语言,被广泛应用于协议测试等领域。由于测试套的复杂度越来越高,测试人员迫切需要能够调试TTCN-3语言的技术。本文提出了一种非侵入式的TTCN-3语言调试方法,通过驱动第三方调试器对TTCN-3语言编译后生成的目标代码进行调试,同时结合语句映射和符号解析等算法,实现了基本的调试功能。实验结果表明,该调试技术对测试系统的执行性能影响非常小,能更好的满足对大型测试套的调试需求。     

Automatic CoDec generation to reduce test engineering cost

TTCN-3 is an abstract language for specification of Abstract Test Suites. Coding of TTCN-3 values into physically transmittable messages and decoding of bitstrings into their TTCN-3 representation has been removed from the language itself and relayed to external and specialized components, called CoDec. CoDec development, either implicitly or explicitly, is a must in any TTCN-3 testing activity. Field experience showed that there is a high cost associated with CoDec development and maintenance. To achieve adequate software engineering practices, a set of types, tools and definitions were developed. This paper unveils gray areas in TTCN-3 architecture and presents a methodological approach to minimize the complexity of CoDec development. Even though the initial field of application is IPv6 testing, the main tool introduced—the CoDec Generator—is a valuable tool in any testing application domain. It is designed to lower the CoDec maintenance costs in all test case lifecycle stages, from development to maintenance. This work has been partly supported by the IST Go4IT European project: .     

Automating test automation

When trying to introduce TTCN-3 and thus improving test automation we are regularly faced with the issue of legacy test automation. These existing approaches to test automation range from small code developed for a very specific task to large applications with substantial development effort or even purchased solution from third party vendors that might include hardware components. Often these solutions cater for a specific need and their combined application is controlled by humans. A higher degree of test automation can be achieved if these solutions are combined in a test system to enable batch runs for example. TTCN-3 can be a good choice for implementing a future-oriented solution. However, recreating all of the existing parts in TTCN-3 might not be a feasible solution from a business perspective. Instead of refraining from creating an advanced solution, the idea is to use those existing parts to substantially reduce the effort necessary to introduce an improved test automation solution. In this paper, we will report on the successful integration of legacy test solutions using TTCN-3 and creating a new, advanced test system that enables overnight test runs. The paper will illustrate our challenges and experiences from this project.     

基于TTCN-3表格形式的OSP测试例

为了满足日益增长的测试需求,欧洲电信标准协会推出了新一代测试描述语言TTCN-3。TTCN-3的表格形式不仅是其核心语言的一种表现形式,而且强调了如何满足协议一致性测试的需求。文中研究了TTCN-3的表格形式,并提出了一个用表格形式描述的OSP协议的测试例。该测试例可以被应用于OSP协议的一致性测试。     

Innovation and evolution in integrated web application testing with TTCN-3

Over the last 10 years there has been continuous innovation and evolution in the technology of web applications. While originally designed as a telecom testing platform, TTCN-3 has proven to be a flexible and powerful platform for web application testing throughout this period. Major challenges to testing have been integration with unit test frameworks, service-oriented architecture, rich client interfaces, and security vulnerabilities. Through careful analysis and practical experience in industrial projects we have developed mechanisms to address each of these aspects of web application testing in TCN-3. These mechanisms are summarized here with examples drawn from our industrial experience to show how TTCN-3 provides significant advantages for testing web applications in comparison to other tools typically used in industry. We also present a significant extension to the model architecture for TTCN-3 which greatly facilitates web application testing that has been implemented by a major vendor and is now under consideration by the ETSI standards committee.     

基于TTCN-3的LDP一致性测试研究

为了满足LDP的一致性测试需求,并突破原有LDP一致性测试中测试方式简单、测试程序不易于理解和测试语言描述能力弱的局限,基于TTCN-3测试描述语言研究了LDP一致性测试;在自行研制的分布式并发测试系统PITSv3的基础上设计扩展了针对LDP的测试适配器模块;设计了LDP的测试集并使用TTCN-3语言实现了该测试集,从而完成了PITSv3系统对LDP的功能扩展。最后进行了PITSv3系统对于LDP的一致性测试实践。     

History,status, and recent trends of the testing and test control notation version 3 (TTCN-3)

This overview article presents the Testing and Test Control Notation (TTCN-3) success story and serves as an introduction to this Special Section that contains five articles selected from the TTCN-3 user conference in 2011. The article sketches the development of TTCN-3 from its very beginning. It summarizes the current status of the language by reviewing its standardization process, available test suites, tools, and services as well as its training program. In addition, the article puts the articles selected for this Special Section into perspective, with regard to the evolution of TTCN-3 and the testing methodology in general. Last but not least, it discusses indicators for possible future developments of TTCN-3.     

TTCN-3研究综述

随着网络技术的发展，TTCN-3在协议测试领域的运用越来越广泛。文章介绍了TTCN-3的发展过程和应用状况，阐述了其核心语言、表格及图形表示格式，重点分析了TTCN-3标准对比TTCN前版本改进的主要特性，并以此为基础提出TTCN-3的应用前景。     

Refactoring and converting a TTCN-2 test suite

When deploying TTCN-3 at Nokia, we converted two TTCN-2 test systems to TTCN-3. We explain to which extend we have been able to do the conversion automatically. The conversion tool used provided a syntactically and semantically correct conversion of the TTCN-2 code. We define some improvements made on the tool to increase readability and maintainability of the resulting code. We cover aspects of converting also the non-TTCN-2 parts and describe the experiences we made as a set of lessons learnt.     

Testing hybrid systems with TTCN-3 embedded

A testing language typically provides a set of test automation statements that allows for a systematic definition and automatic application of stimulation data (i.e. messages or signals) to a system under test. Moreover, it eases the assessment of the system’s reaction by providing customizable evaluation statements and functions. TTCN-3, the Testing and Test Control Notation, already provides universal and powerful concepts to describe tests for discrete, message-based systems. However, software-based control systems that are used to control physical processes often show continuous quantities that can be only poorly stimulated and assessed by means of the currently available language constructs in TTCN-3. In this article, we show how this problem can be solved by extending the TTCN-3 language. We introduce an extension of TTCN-3, namely TTCN-3 embedded, that provides concepts and constructs that directly address the specification of tests for continuous and hybrid real time systems. The extension includes the notion of streams that can be used to represent continuous quantities over time. In addition, TTCN-3 has been extended with the concepts of stream-based ports, sampling, equation systems, and with additional control flow structures. The concepts are integrated with standard TTCN-3 and allow for defining test cases that handle continuous quantities, as well as discrete state changes and the exchange of messages within the same concept space. The feasibility of the approach is shown by providing a small example from the automotive industry.     

基于CPN对系统的并发行为进行测试

首先对基本并发行为进行CPN建模及状态空间生成,逐渐增大模型的复杂性,展示了CPN建模并发行为可能存在的状态空间快速增大问题。在保证并发覆盖的前提下,将测试序列的生成划分为3个阶段,重点讨论了覆盖并发行为的测试序列生成方法。对于并发开始(末)库所按照CPN执行产生的状态空间中的节点进行了映射,得到状态空间中对应并发的开始(末)库所的开始(末)节点集合。通过对并发的开始(末)节点集内的节点间的关系进行分析,依据它们在状态空间中的前驱后继关系 ,生成一个由开始(末)节点的序列构成的序列的集合,然后以此序列集内的序列作为覆盖并发行为的测试序列的开始部分或者结尾部分,生成覆盖并发的测试序列。通过一个自行实现的P2P软件,使用了提出的建模方法及测试序列生成算法。最后通过得到的测试序列设计了测试场景及测试方案,包括Tracker并发行为测试系统结构、服务器测试方案及典型的测试场景设计。将P2P软件和TTCN-3测试机部署在一起实现了TTCN-3测试套的执行,测试结果表明测试工作的设计与实现是正确的。     

基于Eclipse平台的TTCN-3测试系统

分析TTCN-3标准规范中的TTCN-3测试系统通用结构,提出一种基于Eclipse平台的TTCN-3测试系统体系架构。使用Eclipse的插件机制,实现TTCN-3测试环境与Eclipse平台的集成。讨论测试系统中各模块的功能及相关实现方法,阐述测试系统的具体测试流程。实际应用表明了该测试系统的有效性。     

TTCN-3测试系统中模板匹配机制的一种实现方法

TTCN-3拥有一个功能强大且易于掌握的模板匹配机制。本文提出一种TTCN-3测试系统开发过程中模板匹配机制通用的、开放的实现方案。通过对模板匹配功能在TTCN-3测试系统中的作用进行深入分析,采用面向对象的程序设计语言C++实现TTCN-3测试系统中带有匹配机制的模板表示和模板匹配操作,并给出一个对于DNS协议实现功能测试的抽象测试套的翻译方案验证本文设计的合理性。     

基于TTCN-3的Web应用功能测试方法

针对日益增长的Web应用功能测试需求与测试重用需求,提出了一种基于TTCN-3的Web应用功能测试方法。该方法结合分层测试的思想以及TTCN-3的特点,提出将Web应用功能测试划分为页面内容测试、会话管理测试、使用场景测试等3个层次,并分别从这3个层次给出了采用TTCN-3进行测试的过程。该方法应用TTCN-3参数化设计测试用例的思想,有效分离了测试设计与测试实现,使测试用例具有高可重用性。结合实例,表明了该方法的有效性。