Assuring software quality assurance

The term quality assurance (or QA) has a variety of interpretations. The most common one is that it ensures that developers, testers, or independent auditors have performed some form of scrutiny on a system to validate that it will work as required. Software quality assurance is similar but applies to the code or noncode artifacts.     

Efficiency of CAMEtools in software quality assurance

Our paper describes the requirements and possibilities of integration of metrics tools in the field of software quality assurance. Tools for the support of the measurement process are herein classified as Computer Assisted Software Measurement and Evaulation Tools (CAMETools). Software measurement regarded as a special type of metrics application provides a great amount of basic information for the evaluation of the software development process or the software product itself. Our paper examines the effectiveness and destination of software measurement in tool-based software development and is based on an analysis of more than 20 CAME tools in the Software Measurement Laboratory at the University of Magdeburg. CAMEtools are useable for the process, product, and resources evaluation in all phases of the software life cycle (including the problem definition) for different development paradigms. The efficiency of CAME tools is described on the basis of a general measurement framework. This framework includes all steps in the software measurement and evaulation process: metrics definition, selection of the evaluation criteria, tool-based modelling and measurement, value presentation and statistical analysis. The framework includes the main aspects of the process evaluation techniques (Capability Maturity Model, ISO 9000-3 etc.) and product evaluation (ISO 9126, etc.). It is not a disjointed set of aspects: our measurement framework represents an incremental technique for the application of quantification of quality aspects in a required quality assurance     

The role of inspection in software quality assurance

Due to the complexity of the code, software is released with many errors. In response, both software practitioners and software researchers need to improve the reputation of the software. Inspection is the only way to improve the quality of software. Inspection methods can be more effective but success depends on having a sound and systematic procedure for conducting the inspection. The Workshop on Inspection in Software Engineering (WISE), a satellite event of the 2001 Computer Aided Verification (CAV '01) Conference, brought together researchers, practitioners, and regulators in the hope of finding effective approaches to software inspection. The workshop included invited lectures and paper presentations in the form of panel discussions on all aspects of software inspection. Submissions explained how practitioners and researchers were performing inspections, discussed the relevance of inspections, provided evidence of how inspections could be improved through refinement of the inspection process and computer aided tool support and explained how careful design of software could make inspections easier or more effective.     

The use of mathematics in software quality assurance

The use of mathematics for documenting, inspecting, and testing software is explained and illustrated. Three measures of software quality are described and discussed. Then three distinct complementary approaches to software quality assurance are presented. A case study, the testing and inspection of a safety-critical system, is discussed in detail.     

Formal engineering methods for software quality assurance

Conventional software engineering on the basis of informal or semi-formal methods is facing tremendous challenges in ensuring software quality.Formal methods have attempted to address these challenges by introducing mathematical notation and calculus to support formal specification,refinement,and verification in software development.     

浅谈对日软件外包保证项目质量的几点体会

软件外包是近几年国内发展迅速的产业。一般是委托方担当系统的概要设计,中方担当详细设计、编程、单体测试以及集成测试。由于地域、语言、文化等差异,如何保证项目的质量,时常成为困扰企业的难题。在实际的面向中小企业统合管理系统项目的开发基础上,通过分析影响实际项目质量的主要因素,总结并提出了在不写详细设计文档的情况下,加强概要设计的复审,加强沟通环节以保证软件项目质量的一些观点。这种方式下开发的系统其品质得到了较好的控制并取得了客户的认可。     

Integrating security activities into the software development life cycle and the software quality assurance process

Security concerns should be an integral part of the entire planning, development, and operation of a computer application. Inadequacies in the design and operation of computer applications are very frequent source of security vulnerabilities associated with computers. In most cases, the effort to improve security should concentrate on the application software. The system development life cycle (SDLC) technique provides the structure to assure that security safeguards are planned, designed, developed and tested in a manner that is consistent with the sensitivity of the data and/or the application. The software quality assurance process provides the reviews and audits to assure that the activities accomplished during the SDLC produce operationally effective safeguards.This paper addresses two issues of concern to those responsible for ensuring that the safeguards incorporated into application software are adequate and appropriate. The first issue addresses the integration of specific security activities into the SDLC. The discussion of this issue addresses the following security activities in the SDLC; determination of the sensitivity of the application and data; determination of security objectives; assessment of the security risks; conduct of the security feasibility study; definition of security requirements; development of the security test plan; design of the security specifications; development of the security test procedures; writing of the security-relevant code; writing of the security-relevant documentation; conduct of the security test and evaluation; writing on the security test analysis report; and, preparation of the security certification report.The second security issue addresses the security reviews and audits that should be integrated into the software quality assurance process to ensure that the security activities in the SDLC are accomplished. The security reviews and audits discussed include: the security requirements review; the security design review; the security specifications review; the security test readiness review; and the security test and evaluation review. Also addressed is how quality software is defined and achieved and why and how the concept of quality should be applied to application software security safeguards.     

安全关键软件的安全性保障工作研究

安全关键软件如果发生故障,可能会对国家财产和人民安全造成巨大的损失,所以需要重点考虑它们的安全性.但是由于当前还无法精确地定量评估软件安全性,而只能在软件生命周期中从安全性角度对开发行为进行规范和保障.概述了安全性相关的概念,并给出了一个完整的安全关键软件安全性保障工作流程.     

An essay on software testing for quality assurance – Editor’s introduction

This volume resulted from a call for papers to “... explore the state of the art of software quality assurance, with particular emphasis on testing to measure quality.” It is my belief that software testing as a discipline is ripe for theoretical breakthroughs. Researchers are considering the right questions, and there are promising new approaches and exciting new results. It seems that new understanding of the testing process can lead to practical tools and techniques that revolutionize software development. I don’t believe that testing will become easier or cheaper; rather, it will be more rational in the sense that expending effort will more dependably lead to better software. In this introductory essay I provide a personal view of testing, testing research, and their roles in software quality assurance.     

Statistical games and software tools for quality assurance based on statistical process control

This paper proposes two kinds of statistical games constructed to show how to achieve quality assurance system based on SPC(statistical process control) by using simple models and software tools. Proposed games are Coin Shooting Game and Paper Glider Releasing Game. These games can be played on a table using simple materials, and are easy to play. These are described by showing actual data.

Participants of these games get necessary outputs timely by using prepared software tools so that they can execute an effective and efficient decision. These save time and also raise the level of understanding how to achieve quality assurance system based on SPC.     

Software quality assurance in practice

This paper presents the key findings of a survey of a representative sample of the Australian software industry as to the actual use and application of quality assurance techniques in the development of software. The survey aims to confirm previous survey findings through a management questionnaire and to investigate the software development practices at the screen face through a developer questionnaire. The separate responses are analysed to rate the extent to which quality management practices have penetrated the information systems department. The project commenced in July 1994 and the results of this initial survey support the hypothesis that software quality assurance programmes have not yet penetrated to the systems developers at lower levels of organizations.     

Stretching the paradigm for software reliability assurance

Reliability of a software product is best assured by the same means as reliability of any other product is assured: by a systematic, end-to-end, design for reliability process that begins at product concept and follows through to customer service after delivery. Steps in this process are coordinated with the phases of the product realization (software development) process and include specific reliability improvement activities that complement each phase. This represents a new approach to software reliability assurance needs, contrasted with the traditional inspection and rework approach commonly taken to software reliability assurance via reliability growth testing and modelling. The program we describe draws on contemporary principles of quality management and reliability engineering for its overall structure. Also, several important and promising new areas of software reliability research devolving from this expanded paradigm are sketched.     

Using argumentation to evaluate software assurance standards

ContextMany people and organisations rely upon software safety and security standards to provide confidence in software intensive systems. For example, people rely upon the Common Criteria for Information Technology Security Evaluation to establish justified and sufficient confidence that an evaluated information technology product’s contributions to security threats and threat management are acceptable. Is this standard suitable for this purpose?ObjectiveWe propose a method for assessing whether conformance with a software safety or security standard is sufficient to support a conclusion such as adequate safety or security. We hypothesise that our method is feasible and capable of revealing interesting issues with the proposed use of the assessed standard.MethodThe software safety and security standards with which we are concerned require evidence and discuss the objectives of that evidence. Our method is to capture a standard’s evidence and objectives as an argument supporting the desired conclusion and to subject this argument to logical criticism. We have evaluated our method by case study application to the Common Criteria standard.ResultsWe were able to capture and criticise an argument from the Common Criteria standard. Review revealed 121 issues with the analysed use of the standard. These range from vagueness in its text to failure to require evidence that would substantially increase confidence in the security of evaluated software.ConclusionOur method was feasible and revealed interesting issues with using a Common Criteria evaluation to support a conclusion of adequate software security. Considering the structure of similar assurance standards, we see no reason to believe that our method will not prove similarly valuable in other applications.     

Experiences from introducing UML-based development in a large safety-critical project

UML and UML-based development methods have become de facto standards in industry, and there are many claims for the positive effects of modelling object-oriented systems using methods based on UML. However, there is no reported empirical evaluation of UML-based development in large, industrial projects. This paper reports a case study in ABB, a global company with 120,000 employees, conducted to identify immediate benefits as well as difficulties and their causes when introducing UML-based development in large projects. ABB decided to use UML-based development in the company’s system development projects as part of an effort to enable certification according to the IEC 61508 safety standard. A UML-based development method was first applied in a large, international project with 230 system developers, testers and managers. The goal of the project was to build a new version of a safety-critical process control system. Most of the software was embedded. The project members were mostly newcomers to the use of UML. Interviews with 16 system developers and project managers at their sites in Sweden and Norway were conducted to identify the extent to which the introduction of UML-based development had improved their development process. The interviewees had experienced improvements with traceability from requirements to code, design of the code, and development of test cases as well as in communication and documentation. These results thus support claims in the literature regarding improvements that may be obtained through the use of UML. However, the results also show that the positive effects of UML-based development were reduced due to (1) legacy code that it was not feasible to reverse engineer into UML, (2) the distribution of requirements to development teams based on physical units and not on functionality, (3) training that was not particularly adapted to this project and considered too expensive to give to project members not directly involved in development with UML, and (4) a choice of modelling tools with functionality that was not in accordance with the needs of the project. The results from this study should be useful in enabling other UML adopters to have more realistic expectations and a better basis for making project management decisions.

Software quality assurance economics

ContextSoftware companies invest in quality assurance in order to lower software development and maintenance cost, and to increase revenue and profit margins. To contribute to increase of net income, a quality assurance organization has to consider cost and value of the testware involved in assuring quality of software artifacts, such as requirements, specifications, designs, and code.ObjectiveThis paper proposes a set of economic metrics: testware return on investment, inflation, and cost and value sensitivity to artifact changes and time passage. The paper proposes a set of guidelines on lowering testware cost, on increasing value, on maximizing return on investment, and on when to release.MethodThis paper presents an industrial case study data on the relation between test case cost and value, and on cost and value sensitivity to time passage and artifact changes.ResultsThe industrial case study showed return on investment on test cases of up to 200%, deflation of up to −2% per month, undesirable economic effects, such as test case cost outpacing test case value and rapid test case value depreciation based on time passage.ConclusionA viable QA organization should measure and improve test case return on investment, inflation, and cost and value sensitivity to artifact changes and time passage.     

Experiences of a software reuse project

Despite the large number of successful software reuse cases reported, there are more than a few negative views that reuse has not yet delivered as expected. This is because of the lack of consensus on the applicative conditions of reuse technology and the size of the benefits that can be expected. This article describes a software reuse project conducted at Nippon Telegraph and Telephone Corporation (NTT) Software Laboratories. Based on this experience, the article discusses such important aspects of software reuse as domain selection and analysis, impediments, incentives, and library tools. It concludes that successful software reuse requires the selection of appropriate domains, systematic development of reusable modules based on domain analysis, and the commitment of senior management.