Quality specification and metrication,results from a case-study in a mission-critical software domain

Software quality is of increasing importance in mission-critical embedded software systems. Due to the fast growing complexity and accompanying risks of failures of these systems, software quality needs to be addressed explicitly by software developers, preferably with a systematic method for an optimal implementation of software qualities, such as reliability, time-behavior and usability. At the Centre of Automation of Mission-critical Systems (CAMS) of the Dutch Royal Navy, a new approach has been defined for software developers to improve the way that they deal with software quality in the process of mission-critical systems engineering. The stepwise approach is based on both an international quality standard for software product quality, i.e. ISO9126, and on Multi-Criteria Decision Making techniques, i.e. analytical hierarchy process (AHP). The stepwise approach has been validated in a case study. In particular, the tailoring of the ISO9126 standard toward the specific CAMS development situation, and the applicability of AHP techniques, from the perspective of software developers, has been investigated. The case study is carried out in a representative software development project, i.e. the software for combat management systems (CMS) of warships. Results of the case study show that software developers can explicitly deal with quality on the basis of both the ISO9126 standard and the AHP techniques, respectively regarding the specification, prioritization and metrication of software product quality.     

Specifying software quality with the extended ISO model

Specifying the quality of software products is a valuable addition to functional specification, clarifying product properties such as learnability and availability. Specifying such properties is considered difficult due to the different parties involved and the implicit nature of the requirements. The QUINT project gathered experience with product specification by means of the Extended ISO model: an extension to the ISO 9126 model of software quality. By defining indicators and specifying how they should be measured, quality specifications can make requirements explicit. Recommendations and pitfalls for composing a specification are grouped by the context in which quality specifications can be used.     

Efficiency of CAMEtools in software quality assurance

Our paper describes the requirements and possibilities of integration of metrics tools in the field of software quality assurance. Tools for the support of the measurement process are herein classified as Computer Assisted Software Measurement and Evaulation Tools (CAMETools). Software measurement regarded as a special type of metrics application provides a great amount of basic information for the evaluation of the software development process or the software product itself. Our paper examines the effectiveness and destination of software measurement in tool-based software development and is based on an analysis of more than 20 CAME tools in the Software Measurement Laboratory at the University of Magdeburg. CAMEtools are useable for the process, product, and resources evaluation in all phases of the software life cycle (including the problem definition) for different development paradigms. The efficiency of CAME tools is described on the basis of a general measurement framework. This framework includes all steps in the software measurement and evaulation process: metrics definition, selection of the evaluation criteria, tool-based modelling and measurement, value presentation and statistical analysis. The framework includes the main aspects of the process evaluation techniques (Capability Maturity Model, ISO 9000-3 etc.) and product evaluation (ISO 9126, etc.). It is not a disjointed set of aspects: our measurement framework represents an incremental technique for the application of quantification of quality aspects in a required quality assurance     

Agile requirements engineering practices and challenges: an empirical study

This paper describes empirical research into agile requirements engineering (RE) practices. Based on an analysis of data collected in 16 US software development organizations, we identify six agile practices. We also identify seven challenges that are created by the use of these practices. We further analyse how this collection of practices helps mitigate some, while exacerbating other risks in RE. We provide a framework for evaluating the impact and appropriateness of agile RE practices by relating them to RE risks. Two risks that are intractable by agile RE practices emerge from the analysis. First, problems with customer inability and a lack of concurrence among customers significantly impact agile development. Second, risks associated with the neglecting non‐functional requirements such as security and scalability are a serious concern. Developers should carefully evaluate the risk factors in their project environment to understand whether the benefits of agile RE practices outweigh the costs imposed by the challenges.     

Defining a Requirements Process Improvement Model

Both software organisations and the academic community are aware that the requirements phase of software development is in need of further support. We address this problem by creating a specialised Requirements Capability Maturity Model (R-CMM¹). The model focuses on the requirements engineering process as defined within the established Software Engineering Institute’s (SEI’s) software process improvement framework. Our empirical work with software practitioners is a primary motivation for creating this requirements engineering process improvement model. Although all organisations in our study were involved in software process improvement (SPI), they all showed a lack of control over many requirement engineering activities.This paper describes how the requirements engineering (RE) process is decomposed and prioritised in accordance with maturity goals set by the SEI’s Software Capability Maturity Model (SW CMM). Our R-CMM builds on the SEI’s framework by identifying and defining recommended RE sub-processes that meet maturity goals. This new focus will help practitioners to define their RE process with a view to setting realistic goals for improvement.Sarah Beecham is a research fellow in the Department of Maths and Computing in The Open University in the UK. She is currently working on the EPSRC funded CRESTES project () looking into modelling resource estimation for long-lived software. She has recently completed her PhD for a program of work entitled “A Requirements-based Software Process Maturity Model”. Current research interests are in estimation for software evolution and maintenance and in the general areas of software process improvement. Her particular research interests are in empirical methods in software engineering and requirements engineering.Tracy Hall leads the Systems & Software Research Group in the Department of Computer Science at the University of Hertfordshire. She specialises in the empirical investigation of technical and non-technical issues within software engineering. During the past ten years Tracy has successfully collaborated with many companies on a variety of research projects. She is very active in the Empirical Software Engineering community and is regularly invited to talk about empirical methods both in the UK and abroad. Tracy is an accomplished researcher having published over twenty high quality journal papers.Austen Rainer Austen Rainer is a senior lecturer at the University of Hertfordshire. He studied for his PhD at Bournemouth University, in conjunction with IBM Hursley Park. His current research interests include open source software development, longitudinal case study research, and the credibility of empirical evidence for researchers and software practitioners.     

可信软件非功能需求形式化表示与可满足分析

可信软件的可信性由其功能需求和非功能需求共同来体现,其中,非功能需求的实现是可信软件获得用户对其行为实现预期目标能力的信任程度的客观依据.针对可信软件的重要性以及对可信软件的迫切需求,在可信软件的早期需求工程阶段,提出可信软件非功能需求驱动的过程策略选取方法.首先,对可信软件需求进行定义,提出由功能需求和非功能需求中的可信关注点构成可信需求,非可信关注点的非功能需求则定义为软目标,用于表达质量需求,基于模糊集合论和信息熵对可信软件非功能需求进行排序并获取可信关注点和软目标.在此基础上,提出可信软件非功能需求驱动的过程策略选取方法.传统的软件早期需求工程阶段的目标是为了获取满足需求的技术及设计决策,与此不同,本文对可信软件非功能需求进行分析的目标是获取过程策略,从过程角度解决可信软件生产问题.由于非功能需求间复杂的相关关系,尤其是因为存在冲突关系,故提出了基于可满足性问题求解方法推理过程策略的方法,选取满足可信软件非功能需求的过程策略.最后,通过第三方可信认证中心软件的案例,说明所提出方法的可行性.     

A methodology for the selection of requirements engineering techniques

The complexity of software projects as well as the multidisciplinary nature of requirements engineering (RE) requires developers to carefully select RE techniques and practices during software development. Nevertheless, the selection of RE techniques is usually based on personal preference or existing company practice rather than on characteristics of the project at hand. Furthermore, there is a lack of guidance on which techniques are suitable for a certain project context. So far, only a limited amount of research has been done regarding the selection of RE techniques based on the attributes of the project under development. The few approaches that currently exist for the selection of RE techniques provide only little guidance for the actual selection process. We believe that the evaluation of RE techniques in the context of an application domain and a specific project is of great importance. This paper describes a Methodology for Requirements Engineering Techniques Selection (MRETS) as an approach that helps requirements engineers select suitable RE techniques for the project at hand. The MRETS has three aspects: Firstly, it aids requirements engineers in establishing a link between the attributes of the project and the attributes of RE techniques. Secondly, based on the evaluation schema proposed in our research, MRETS provides an opportunity to analyze RE techniques in detail using clustering. Thirdly, the objective function used in our approach provides an effective decision support mechanism for the selection of RE techniques. This paper makes contributions to RE techniques analysis, the application of RE techniques in practice, RE research, and software engineering in general. The application of the proposed methodology to an industrial project provides preliminary information on the effectiveness of MRETS for the selection of RE techniques.     

关于软件需求中的不一致性管理

复杂软件系统开发的一个关键问题是分析和处理可能存在的不一致的需求描述.这个问题解决得好坏直接影响到需求规格说明的质量,进而影响到最终软件产品的质量.在目前公认的一个不一致需求管理框架的基础上,就需求不一致性管理方面的有代表性的工作,进行了较为系统的分析,以期建立对当前需求工程中,关于不一致的需求管理方法和技术的全面认识.最后,对需求不一致性管理方面的研究进行了展望.     

Dealing with nonfunctional requirements in large software systems

Quality improvement such as increased reliability and maintainability are of utmost importance in software development. In this field, which was previously ad hoc and unpredictable rather than customer‐oriented, increasing competition and focus on customer satisfaction have motivated management to put more emphasis on quality issues. This paper provides insight in techniques for dealing with what customers call generic quality attributes and what software engineers call nonfunctional requirements. Since requirements management more than many other disciplines in software engineering need practical insight, examples are provided for dealing with four nonfunctional requirements in large telecommunication systems, namely performance, usability, reliability, and maintainability. Guidelines are presented for specifying, constructing, analyzing, measuring and tracing nonfunctional requirements. Many examples from telecommunication system development show how to specifically and pragmatically deal with nonfunctional requirements. Extracts from three common standards (SEI CMM, ISO 9000–3, BELLCORE suite) are added in the form of a checklist for underlining the importance of proper requirements management from a customer viewpoint.     

A Framework for Improving the Requirements Engineering Process Management

This paper presents a system dynamics model for improving the requirements engineering process management. The paper argues that improving RE process management improves the quality of the specification produced. It uses a simulation modelling approach to capture the complex and dynamic nature of quality and also the cost of resources and time needed to complete the process. Current claims by various researchers and empirical evidence has led to our proposition that “the earlier in the requirements engineering phase that system dynamics simulation modelling is used, the more effective the RE process management is and the better its product quality will be.” In developing such a model, the paper fills an important gap in the RE process management literature and has potential to provide requirement engineers, managers and software development organisations with a model-based process framework to aid quality assessment and improvement. The paper concludes by suggesting that the framework makes a useful contribution both in providing the foundations for theory building in RE process management and quality improvement by aiding shared understanding through learning and training situations. This revised version was published online in August 2006 with corrections to the Cover Date.     

浅谈软件质量度量和软件产品评价

软件质量度量和软件产品评价系列标准是国际标准化组织ISO／IEC JTC1近年来在软件工程标准方面的研究重点之一,对于通过量化手段进行软件产品的度量和评价,规范软件产品的质量管理,这两个系列标准提供了一条可以参考的实施途径。本文在多年跟踪研究国际上软件工程标准和制定软件工程国家标准的基础上,对ISO／IEC JTC1近年推出的ISO／IEC 9126和ISO／IEC 14598系列,以及正在研制的ISO／IEC 25000系列标准进行综合介绍。     

Effective Hybrid Content-Based Collaborative Filtering Approach for Requirements Engineering

Requirements engineering (RE) is among the most valuable and critical processes in software development. The quality of this process significantly affects the success of a software project. An important step in RE is requirements elicitation, which involves collecting project-related requirements from different sources. Repositories of reusable requirements are typically important sources of an increasing number of reusable software requirements. However, the process of searching such repositories to collect valuable project-related requirements is time-consuming and difficult to perform accurately. Recommender systems have been widely recognized as an effective solution to such problem. Accordingly, this study proposes an effective hybrid content-based collaborative filtering recommendation approach. The proposed approach will support project stakeholders in mitigating the risk of missing requirements during requirements elicitation by identifying related requirements from software requirement repositories. The experimental results on the RALIC dataset demonstrate that the proposed approach considerably outperforms baseline collaborative filtering-based recommendation methods in terms of prediction accuracy and coverage in addition to mitigating the data sparsity and cold-start item problems.     

An evaluation of timed scenario notations

There is a general consensus on the importance of good Requirements Engineering (RE) for achieving high quality software. The modeling and analysis of requirements have been the main challenges during the development of complex systems. Although semi-formal, scenario driven approaches have raised the awareness and use of requirement engineering techniques, mostly because of their intuitive representation. Scenarios are a well established approach to describe functional requirements, uncovering hidden requirements and trade-offs, as well as validating and verifying requirements.The ability to perform quantitative analysis at the requirements level supports the detection of design errors during the early stages of a software development life cycle, and helps reduce the cost of later redesign activities. In order to achieve this goal, non-functional aspects and in particular time-related aspects have to be incorporated at the software requirement phase. This is essential in order to correctly model and analyze time dependent applications at early stages in system development.The widespread interest in time modeling and analysis techniques provides the major motivation for our paper. The objective of the article is to provide readers with sufficient knowledge about existing timed scenario approaches to guide them in making informed decisions to when and how time aspects can be incorporated in their development process. In order to support this process, we present a comprehensive classification, evaluation and comparison of time-based scenario notations. In order to evaluate these existing notations, we introduce a set of eleven time-related criteria and apply them to categorize and compare forty seven scenario construction approaches.     

A case study validation of a knowledge-based approach for the selection of requirements engineering techniques

Requirements engineering (RE) is a critical phase in the software engineering process and plays a vital role in ensuring the overall quality of a software product. Recent research has shown that industry increasingly recognizes the importance of good RE practices and the use of appropriate RE techniques. However, due to the large number of RE techniques, requirements engineers find it challenging to select suitable techniques for a particular project. Unfortunately, technique selection based on personal experience has limitations with regards to the scope, effectiveness and suitability of the RE techniques for the project at hand. In this paper, a Knowledge-based Approach for the Selection of Requirements Engineering Techniques (KASRET) is proposed that helps during RE techniques selection. This approach has three major features. First, a library of requirements techniques was developed which includes detailed knowledge about RE techniques. Second, KASRET integrates advantages of different knowledge representation schemata and reasoning mechanisms. Thus, KASRET provides mechanisms for the management of knowledge about requirements techniques and support for RE process development. Third, as a major decision support mechanism, an objective function evaluates the overall ability and cost of RE techniques, which is helpful for the selection of RE techniques. This paper makes not only a contribution to RE but also to research and application of knowledge management and decision support in process development. A case study using an industrial project shows the support of KASRET for RE techniques selection.

基于人机交互的计算机辅助软件需求分析工具的研发

软件需求工程在软件开发项目中起着至关重要的作用,而需求分析活动的主角是人,因此计算机辅助需求分析工具的设计要做到以用户为中心,即以人为本。本原型作品在已有的面向问题的计算机辅助软件需求工程(Computer-Aided Requirements Engineering,CARE)工具的基础上,进一步在支持的平台及易用性等方面进行了扩展。选择安卓平台来开发此工具,理由是移动端应用软件相较于PC端具有一些优势(功能多样性、便捷性等)以及安卓的开源性质,它能够为提高软件质量以及用户体验带来新的技术支持。此外,还具体展示并介绍了该工具是如何基于人机交互的理论模型及设计原则设计的。     

Quality evaluation for Model-Driven Web Engineering methodologies

ContextThere are lots of approaches or methodologies in the Model-Driven Web Engineering (MDWE) context to develop Web Applications without reaching a consensus on the use of standards and scarcity of both, practical experience and tool support.ObjectiveModel-Driven Web Engineering (MDWE) methodologies are constantly evolving. Moreover, Quality is a very important factor to identify within a methodology as it defines processes, techniques and artifacts to develop Web Applications. For this reason, when analyzing a methodology, it is not only necessary to evaluate quality, but also to find out how to improve it. The main goal of this paper is to develop a set of Quality Characteristics and Sub-Characteristics for MDWE approaches based on ISO/IEC standards.MethodFrom the software products context, some widely standards proposed, such as ISO/IEC 9126 or ISO/IEC 25000, suggest a Quality Model for software products, although up to now, there are no standard methods to assess quality on MDWE methodologies. Such methodologies can be organized into Properties, thus, a methodology has artifacts, processes and techniques. Then, each item is evaluated through a set of appropriate Quality Characteristics, depending on its nature. This paper proposes to evaluate a methodology as a product itself.ResultsThis paper recommends a set of Quality Characteristics and Sub-Characteristics based on these standards in order to evaluate MDWE methodologies quality. Additionally, it defines an agile way to relate these Quality Sub-Characteristics to Properties with the sole purpose of not only analyzing, but also assessing and improving MDWE methodologies.ConclusionsThe application of these Quality Characteristics and Sub-Characteristics could promote efficiency in methodologies since this kind of assessment enhances both the understanding of strengths and weaknesses of approaches.     

复杂软件需求开发方法研究

软件需求开发方法研究是当前复杂软件系统开发中最重要的问题之一。本文首先基于软件开发生命周期模型,对软件需求开发工作进行了介绍。进而总结了软件需求所必须包含的内容,分析了软件需求的特性。基于迭代改进的开发方法,本文将需求开发工作分解为需求捕获、需求分析、需求编写和需求确认四部分活动,并对每部分工作的详细活动和目的进行了分析和总结。软件开发工程实践证明,科学的软件需求开发方法是提高软件开发效率,提高软件开发质量的重要手段。     

Integrated requirements engineering: a tutorial

This short tutorial introduces the fundamental activities of RE (requirements engineering) and discusses how it has evolved as a part of the software engineering process. However, rather than focusing on the established RE techniques, the author discusses how the changing nature of software engineering has led to the new challenges in RE. The author then introduces a number of new techniques that helps us to meet these challenges by integrating RE more closely with other systems implementation activities.     

基于软件完整性级别的软件可靠性度量选取框架

选取合适的软件可靠性度量,对于软件质量保证及项目管理有着重要意义。现有的软件可靠性度量选取方法没有考虑软件完整性级别这个重要的设计属性。完整性级别表示软件特性的取值范围,该范围对将系统风险保持在可容忍的限度内是必需的,其对软件可靠性水平有显著影响。提出了一种基于完整性级别的可靠性度量选取框架:首先给出基于完整性级别的度量选取体系;然后在选取体系的基础上,给出相应的度量选取方法;最后,将提出的度量选取框架应用于ISO/IEC 9126质量模型中的外部软件可靠性度量,根据度量的特点将每种度量不同程度(基本、条件及参考)地推荐给不同的完整性级别。实例表明,基于完整性级别的度量选取技术是系统且有效的,所推荐的度量可以满足软件尤其是安全关键软件在不同完整性级别上的需求。