Software testing processes as a linear dynamic system

Software testing is essential for software reliability improvement and assurance, and the processes of software testing are intrinsically dynamic. However they are seldom investigated in a mathematically rigorous manner. In this paper a theoretical study is presented to examine the dynamic behavior of software testing. More specifically, a set of simplifying assumptions is adopted to formulate and quantify the software testing processes. The mathematical formulae for the expected number of observed software failures are rigorously derived, the bounds and trends of the expected number of observed software failures are analyzed, and the variance of the number of observed software failures is examined. On the other hand, it is demonstrated that under the simplifying assumptions, the software testing processes can be treated as a linear dynamic system. This suggests that the software testing processes could be classified as linear or non-linear, and there be intrinsic link between software testing and system dynamics.     

Product backlog rating: a case study on measuring test quality in scrum

Agile software development methodologies focus on software projects which are behind schedule or highly likely to have a problematic development phase. In the last decade, Agile methods have transformed from cult techniques to mainstream methodologies. Scrum, an Agile software development method, has been widely adopted due to its adaptive nature. This paper presents a metric that measures the quality of the testing process in a Scrum process. As product quality and process quality correlate, improved test quality can ensure high-quality products. Also, gaining experience from 8 years of successful Scrum implementation at SoftwarePeople, we describe the Scrum process emphasizing the testing process. We propose a metric Product Backlog Rating (PBR) to assess the testing process in Scrum. PBR considers the complexity of the features to be developed in an iteration of Scrum, assesses test ratings and offers a numerical score of the testing process. This metric is able to provide a comprehensive overview of the testing process over the development cycle of a product. We present a case study which shows how the metric is used at SoftwarePeople. The case study explains some features that have been developed in a Sprint in terms of feature complexity and potential test assessment difficulties and shows how PBR is calculated during the Sprint. We propose a test process assessment metric that provides insights into the Scrum testing process. However, the metric needs further evaluation considering associated resources (e.g., quality assurance engineers, the length of the Scrum cycle).     

Validation of software in measuring instruments

Most measuring instruments of today are based on embedded systems or personal computers. The software embedded into the instruments controls functions that must be possible to verify and validate. Validation of software is certainly performed by the developing companies, but also the independent test laboratories must be able to perform the validation. Traditional evaluation of measuring accuracy and resistance to environmental stress is not enough to confirm the functionality of the instrument.An efficient validation procedure often combines different kinds of validation methods. The set of validation methods can be collected as a “tool box” which can be applied differently for different measuring instruments. The risk classes, the validation guidance and some possible validation methods are discussed. The paper is based on the work carried out by the European project “MID-Software”.     

软件单元测试应用研究及案例分析

单元测试是软件测试过程的第一站,高质量的单元测试是软件质量保证的必要条件,本文就单元测试的概念、测试类型及规程作了简单的阐述,同时就单元测试进行了案例分析。     

A testing data validity assessment method and testing data validation platform based on SOA

In modern manufacturing, ensuring the quality of component testing data is highly valued by both product manufacturers and component suppliers. However, in common component quality analysis processes, testing data are assumed to be valid, which might not be true. Therefore, assessing the validity of component testing data would be important. Many existing data analysis platforms are separated from enterprises’ own systems, which makes the inspection data analysis incoherent to their business process. In this paper, we propose a testing data quality assessment method and a testing data validation platform based on SOA. The platform provides reliable third-party testing data validation service via RESTful APIs, so that the services can be seamlessly integrated to enterprise systems. The testing data validity assessment method, which is the core of the platform, is implemented by detecting illegal behavior in data recording. The detection is a combination of behavior analysis and a positive and unlabeled learning process.     

基于C++的软件编码阶段的质量保证方法和工具

在基于C++的软件产品开发过程中,如何保证编码阶段的质量,是软件开发人员关注的关键问题之一。文章以面向网络海量空间信息大型GIS的开发中的编码质量保证实践为背景和案例,说明了基于C++的软件编码过程的质量保证关键方法和工具。     

Conformance testing of software

People who buy software want a guarantee that it works. When they cannot satisfy themselves, and do not wholly trust the programmer, it is natural to seek the help of a third party who will certify the software. So far, formal certification is available only for compilers. This paper discusses some of the theoretical problems and reviews existing compiler validation schemes. Finally there is a brief examination of other methods of quality assurance that might provide a better solution.     

An approach to applying SOFL for agile process and its application in developing a test support tool

Structured Object-Oriented Formal Language (SOFL) is a representative formal engineering method for software development. It offers a three-step specification approach to constructing formal specifications, and specification-based inspection and testing for verification and validation. In this paper, we describe a novel approach to applying the SOFL method to achieve agile development process. This approach results from our experience in several collaboration projects with industry, and aims to strike a balance between the fast delivery of software product and the assurance of its quality. We have tested the approach in developing a prototype test support tool.     

Evaluating testing methods by delivered reliability [software]

There are two main goals in testing software: (1) to achieve adequate quality (debug testing), where the objective is to probe the software for defects so that these can be removed, and (2) to assess existing quality (operational testing), where the objective is to gain confidence that the software is reliable. Debug methods tend to ignore random selection of test data from an operational profile, while for operational methods this selection is all-important. Debug methods are thought to be good at uncovering defects so that these can be repaired, but having done so they do not provide a technically defensible assessment of the reliability that results. On the other hand, operational methods provide accurate assessment, but may not be as useful for achieving reliability. This paper examines the relationship between the two testing goals, using a probabilistic analysis. We define simple models of programs and their testing, and try to answer the question of how to attain program reliability: is it better to test by probing for defects as in debug testing, or to assess reliability directly as in operational testing? Testing methods are compared in a model where program failures are detected and the software changed to eliminate them. The “better” method delivers higher reliability after all test failures have been eliminated. Special cases are exhibited in which each kind of testing is superior. An analysis of the distribution of the delivered reliability indicates that even simple models have unusual statistical properties, suggesting caution in interpreting theoretical comparisons     

Using a defined and measured Personal Software Process

Improved software processes lead to improved product quality. The Personal Software Process (PSP) is a framework of techniques to help engineers improve their performance-and that of their organizations-through a step-by-step, disciplined approach to measuring and analyzing their work. This article explains how the PSP is taught and how it applies to different software engineering tasks. The author reports some promising early results     

Beyond Brute Force: Testing Financial Software

Large software development projects that follow formal processes generally divide the work among separate roles on structured teams that tend to employ development staff whose technical expertise is domain-neutral. This is especially true of large outsourcing vendors. Yet, the end users' needs are expressed in the applied domain's specific concepts, terms, and logic. Testing financial software's functionality requires deep subject expertise and an understanding of quality assurance methods and processes. Thorough domain analysis, efficient test designs, specific methods, and well-prepared quality assurance staff are key elements for success.     

Tool electrode geometry and process parameters influence on different feature geometry and surface quality in electrical discharge machining of AISI H13 steel

Electro discharge machining process (EDM) is frequently used when machining of high complex and accurate features is required. Indeed, it is specially recommended for hard materials and micro-machined features. However, due to the process nature, there is still incomprehension on process parameters influence at the final quality features, ending up by lower productivity and quality ratios. On the other hand, fashioning and re-shaping of required electrodes for each feature are time consuming phases and the number of stored electrodes is very high. Therefore, in order to increase the global EDM process productivity, quality and flexibility, standardized simple electrode shapes, capable to machine different features, must be found. This study presents the influence of the main EDM process parameters and different tool geometries on basic process performance measures. A set of designed experiments with varying parameters such as pulsed current, open voltage, pulse time and pulse pause time are carried out in H13 steel using different geometries of copper electrodes. In addition, material removal rate , surface roughness and different dimensional and geometrical micro-accuracies are analyzed through statistical methods. Results help to select appropriate EDM process parameters to machine parts depending on product requirements.     

On assessing spatial uniformity of particle distributions in quality control of manufacturing processes

There are many situations in quality control of manufacturing processes in which the quality of a process is characterized by the spatial distribution of certain particles in the product, and the more uniform the particle distribution is, the better the quality is. To realize quality control and guide process improvement efforts, the degree of spatial uniformity of particle distributions needs to be assessed. On the other hand, many quantitative metrics have been developed in areas outside manufacturing for measuring uniformity of point patterns, which can be applied for this purpose. However, critical issues exist in applying existing metrics for quality control relating to which metrics to choose and how to use them in specific situations. To provide general guidelines on these issues, this research identifies popular uniformity metrics scattered in different areas and compares their performance in detecting nonuniform particle distributions under various practical scenarios through a comprehensive numerical study. Effects of different factors on the performance of the metrics are revealed and the best metric is found. The use and effectiveness of the selected metric is also demonstrated in a case study where it is applied to data from emerging material fabrication processes in nanomanufacturing and biomanufacturing.     

Reducing test effort: A systematic mapping study on existing approaches

ContextQuality assurance effort, especially testing effort, is often a major cost factor during software development, which sometimes consumes more than 50% of the overall development effort. Consequently, one major goal is often to reduce testing effort.ObjectiveThe main goal of the systematic mapping study is the identification of existing approaches that are able to reduce testing effort. Therefore, an overview should be presented both for researchers and practitioners in order to identify, on the one hand, future research directions and, on the other hand, potential for improvements in practical environments.MethodTwo researchers performed a systematic mapping study, focusing on four databases with an initial result set of 4020 articles.ResultsIn total, we selected and categorized 144 articles. Five different areas were identified that exploit different ways to reduce testing effort: approaches that predict defect-prone parts or defect content, automation, test input reduction approaches, quality assurance techniques applied before testing, and test strategy approaches.ConclusionThe results reflect an increased interest in this topic in recent years. A lot of different approaches have been developed, refined, and evaluated in different environments. The highest attention was found with respect to automation and prediction approaches. In addition, some input reduction approaches were found. However, in terms of combining early quality assurance activities with testing to reduce test effort, only a small number of approaches were found. Due to the continuous challenge of reducing test effort, future research in this area is expected.     

Application of a Methodology for the Development and Validation of Reliable Process Control Software

This paper discusses the necessity of a good methodology for the development of reliable software, especialy with respect to the final software validation and testing activities. A formal specification development and validation methodology is proposed. This methodology has been applied to the development and validation of a pilot software, incorporating typical features of critical software for nuclear power plant safety protection. The main features of the approach indude the use of a formal specification language and the independent development of two sets of specifications. Analyses on the specifications consists of three-parts: validation against the functional requirements consistency and integrity of the specifications, and dual specification comparison based on a high-level symbolic execution technique. Dual design, implementation, and testing are performed. Automated tools to facilitate the validation and testing activities are developed to support the methodology. These includes the symbolic executor and test data generator/dual program monitor system. The experiences of applying the methodology to the pilot software are discussed, and the impact on the quality of the software is assessed.     

软件测评能力成熟度模型研究

软件测评作为软件质量保证的重要手段,其重要性已越来越受到各方的高度关注。要在有限的时间、人员和经费等约束条件下,验证软件是否满足要求并尽可能多地发现其中存在的缺陷,就必须找到提高测评效率和质量的有效途径,为此不仅要研究软件测评的技术、方法和工具,还必须注重软件测评过程的管理和改进。本文在借鉴CMM、CMMI、TMMi、GJB5000、GJB5000A思想的基础上,提出了适用于软件第三方测评机构的软件测评能力成熟度模型,有助于第三方测评机构改进其测评过程、主管部门遴选测评机构。     

Modelling and optimization of grinding processes

The paper describes different methods for modelling and optimization of grinding processes. First the process and product quality characterizing quantities have to be measured. Afterwards different model types, e.g. physical–empirical basic grinding models as well as empirical process models based on neural networks, fuzzy set theory and standard multiple regression methods, are discussed for an off-line process conceptualization and optimization using a genetic algorithm. The assessment of grinding process results, which build the individuals in the genetic algorithm's population, is carried out using a target tree method. The methods presented are integrated into an existing grinding information system, which is part of a three control loop system for quality assurance.     

软件确认测试概述

软件确认测试在软件测试生命周期中占有重要地位。尽管在整个开发过程中经历了各种测试和检验，但是，在其正式发布和投入运行之前仍必需通过正式的和系统全面的测试，来确认其是否符合需求。本文介绍了软件确认测试的主要测试内容及活动，分析了确认测试在软件开发过程中的作用和地位，归纳了确认测试的基本测试模型，总结了确认测试的基本测试过程和方法。     

Some bibliometric procedures for analyzing and evaluating research fields

Nowadays, measuring the quality and quantity of the scientific production is an important necessity since almost every research assessment decision depends, to a great extent, upon the scientific merits of the involved researchers. To do that, many different indicators have been proposed in the literature. Two main bibliometric procedures to explore a research field have been defined: performance analysis and science mapping. On the one hand, performance analysis aims at evaluating groups of scientific actors (countries, universities, departments, researchers) and the impact of their activity on the basis of bibliographic data. On the other hand, the extraction of knowledge from the intellectual, social or conceptual structure of a research field could be done by means of science mapping analysis based on bibliographic networks. In this paper, we introduce some of the most important techniques and software tools to analyze the impact of a research field and its scientific structures. Particularly, four bibliometric indices (h, g, hg and q²), the h-classics approach to identify the classic papers of a research field and three free science mapping software tools (CitNetExplorer, SciMAT and VOSViewer) are shown.     

软件质量功能展开法

质量功能展开作为对新产品开发进行质量保证的方法论在日本已被广泛应用。最近，在欧美也被相继引入，笔者们把质量功能开应用到软件开发中进行软件产品的质量保证。本文首先用质量展开的概念模型解释了质量功能展开的基本想法，其次，分析了软件质量保证的问题点；然后，介绍了软件质量功能展开法，最后，总结了软件质量功能展开法的应用效果和今后的课题。