Using error abstraction and classification to improve requirement quality: conclusions from a family of four empirical studies

Achieving high software quality is a primary concern for software development organizations. Researchers have developed many quality improvement methods that help developers detect faults early in the lifecycle. To address some of the limitations of fault-based quality improvement approaches, this paper describes an approach based on errors (i.e. the sources of the faults). This research extends Lanubile et al.’s, error abstraction process by providing a formal requirement error taxonomy to help developers identify both faults and errors. The taxonomy was derived from the software engineering and psychology literature. The error abstraction and classification process and the requirement error taxonomy are validated using a family of four empirical studies. The main conclusions derived from the four studies are: (1) the error abstraction and classification process is an effective approach for identifying faults; (2) the requirement error taxonomy is useful addition to the error abstraction process; and (3) deriving requirement errors from cognitive psychology research is useful.     

A taxonomy and catalog of runtime software-fault monitoring tools

A goal of runtime software-fault monitoring is to observe software behavior to determine whether it complies with its intended behavior. Monitoring allows one to analyze and recover from detected faults, providing additional defense against catastrophic failure. Although runtime monitoring has been in use for over 30 years, there is renewed interest in its application to fault detection and recovery, largely because of the increasing complexity and ubiquitous nature of software systems. We present taxonomy that developers and researchers can use to analyze and differentiate recent developments in runtime software fault-monitoring approaches. The taxonomy categorizes the various runtime monitoring research by classifying the elements that are considered essential for building a monitoring system, i.e., the specification language used to define properties; the monitoring mechanism that oversees the program's execution; and the event handler that captures and communicates monitoring results. After describing the taxonomy, the paper presents the classification of the software-fault monitoring systems described in the literature.     

X-ware reliability and availability modeling

The problem of modeling a system's reliability and availability with respect to the various classes of faults (physical and design, internal and external) which may affect the service delivered to its users is addressed. Hardware and software models are currently exceptions in spite of the user's requirements; these requirements are expressed in terms of failures independently of their sources, i.e., the various classes of faults. The causes of this situation are analyzed; it is shown that there is no theoretical impediment to deriving such models, and that the classical reliability theory can be generalized in order to cover both hardware and software viewpoints that are X-Ware     

Predicting risk as a function of risk factors

In previous work, we showed that risk factors have a significant negative effect on reliability (e.g., occurrence of failure). In this paper, we show that it is feasible to predict risk (i.e., the probability of risk factors being related to discrepancy reports occurring on the release of software). This is an important advancement over the previous research because discrepancy reports are available in the requirements phase—when the cost and labor required to correct faults is low—whereas failure data only becomes available in the test phase—when the cost and labor required for correcting faults is high. Although using historical failure data to drive traditional software reliability models would produce greater prediction accuracy, the opportunity to provide early prediction of reliability using risk factors outweighs this advantage.     

软件开发活动数据的数据质量问题

问题追踪系统和版本控制系统等软件开发支持工具已被广泛应用于开源和商业软件的开发中,产生了大量的数据,即软件开发活动数据.软件开发活动数据被广泛应用于科学研究和开发实践,为智能化开发提供支持.然而数据质量对相关的研究和实践有重大影响,却还没有得到足够的重视.为了能够更好地警示数据使用者潜在的数据质量问题,通过文献调研和访谈,并基于自有经验对数据进行分析,总结出了9种数据质量问题,覆盖了数据产生、数据收集和数据使用这3个不同的阶段.进一步地,提出了相应的方法以帮助发现和解决数据问题.发现问题是指加强对数据上下文的理解和通过统计分析及数据可视化发现潜在的数据质量问题,解决问题是指利用冗余数据或者挖掘用户行为模式进行修正.     

SERP-test: a taxonomy for supporting industry–academia communication

This paper presents the construction and evaluation of SERP-test, a taxonomy aimed to improve communication between researchers and practitioners in the area of software testing. SERP-test can be utilized for direct communication in industry academia collaborations. It may also facilitate indirect communication between practitioners adopting software engineering research and researchers who are striving for industry relevance. SERP-test was constructed through a systematic and goal-oriented approach which included literature reviews and interviews with practitioners and researchers. SERP-test was evaluated through an online survey and by utilizing it in an industry–academia collaboration project. SERP-test comprises four facets along which both research contributions and practical challenges may be classified: Intervention, Scope, Effect target and Context constraints. This paper explains the available categories for each of these facets (i.e., their definitions and rationales) and presents examples of categorized entities. Several tasks may benefit from SERP-test, such as formulating research goals from a problem perspective, describing practical challenges in a researchable fashion, analyzing primary studies in a literature review, or identifying relevant points of comparison and generalization of research.     

An empirical study of the impact of modern code review practices on software quality

Software code review, i.e., the practice of having other team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that formal code inspections tend to improve the quality of delivered software. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process, little is known about the relationship between modern code review practices and long-term software quality. Hence, in this paper, we study the relationship between post-release defects (a popular proxy for long-term software quality) and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, (2) code review participation, i.e., the degree of reviewer involvement in the code review process, and (3) code reviewer expertise, i.e., the level of domain-specific expertise of the code reviewers. Through a case study of the Qt, VTK, and ITK projects, we find that code review coverage, participation, and expertise share a significant link with software quality. Hence, our results empirically confirm the intuition that poorly-reviewed code has a negative impact on software quality in large systems using modern reviewing tools.     

静态软件缺陷预测方法研究

静态软件缺陷预测是软件工程数据挖掘领域中的一个研究热点.通过分析软件代码或开发过程,设计出与软件缺陷相关的度量元;随后,通过挖掘软件历史仓库来创建缺陷预测数据集,旨在构建出缺陷预测模型,以预测出被测项目内的潜在缺陷程序模块,最终达到优化测试资源分配和提高软件产品质量的目的.对近些年来国内外学者在该研究领域取得的成果进行了系统的总结.首先,给出了研究框架并识别出了影响缺陷预测性能的3个重要影响因素:度量元的设定、缺陷预测模型的构建方法和缺陷预测数据集的相关问题;接着,依次总结了这3个影响因素的已有研究成果;随后,总结了一类特殊的软件缺陷预测问题(即,基于代码修改的缺陷预测)的已有研究工作;最后,对未来研究可能面临的挑战进行了展望.     

The role of information structures in game-theoretic multi-agent learning

Multi-agent learning (MAL) studies how agents learn to behave optimally and adaptively from their experience when interacting with other agents in dynamic environments. The outcome of a MAL process is jointly determined by all agents’ decision-making. Hence, each agent needs to think strategically about others’ sequential moves, when planning future actions. The strategic interactions among agents makes MAL go beyond the direct extension of single-agent learning to multiple agents. With the strategic thinking, each agent aims to build a subjective model of others decision-making using its observations. Such modeling is directly influenced by agents’ perception during the learning process, which is called the information structure of the agent’s learning. As it determines the input to MAL processes, information structures play a significant role in the learning mechanisms of the agents. This review creates a taxonomy of MAL and establishes a unified and systematic way to understand MAL from the perspective of information structures. We define three fundamental components of MAL: the information structure (i.e., what the agent can observe), the belief generation (i.e., how the agent forms a belief about others based on the observations), as well as the policy generation (i.e., how the agent generates its policy based on its belief). In addition, this taxonomy enables the classification of a wide range of state-of-the-art algorithms into four categories based on the belief-generation mechanisms of the opponents, including stationary, conjectured, calibrated, and sophisticated opponents. We introduce Value of Information (VoI) as a metric to quantify the impact of different information structures on MAL. Finally, we discuss the strengths and limitations of algorithms from different categories and point to promising avenues of future research.     

Leveraging user impact: an architecture for secondary screens usage in interactive television

This paper reports on an architecture, and a working implementation, for using secondary screens in the interactive television environment. While there are specific genres and programs that immerse the viewer into the television experience, there are situations in which people perform as well a secondary task, whilst watching. In the living room, people surf the web, use email, and chat using one or many secondary screens. Instead of focusing on unrelated activities to television watching, the architecture presented in this paper aims at related activities, i.e., to leverage the user impact on the content being watched. After a comprehensive literature review and working systems analysis, the requirements for the secondary screen architecture are identified and modelled in the form of a taxonomy. The taxonomy is divided into three high-level categories: control, enrich, and share content. By control we refer to the decision what to consume and where to render it. In addition, the viewer can use the secondary screen for enriching media content and for sharing the enriched material. The architecture is validated based on the taxonomy and by an inspection of the available services. The final intention of our work is to leverage the viewers’ control over the consumed content in our multi-person, multi-device living rooms.     

An Experimental Investigation into the Effectiveness of OOA for Specifying Requirements

The application of object oriented concepts (OO) to the requirements phase of information systems (IS) and software development has been adopted by many proponents of IS and software development methodologies. Although many claims have been made about the effectiveness of OO techniques for improving requirements analysis, very few experimental studies have been done to substantiate these claims. This paper addresses this gap in the literature by conducting an experimental study that attempts to validate the effectiveness of object-oriented analysis (OOA) by comparing it to structured analysis (SA) for producing requirements. We argue that the quality of the requirements specification can be measured and that measurement can be used to compare the effectiveness of OOA and SA. We present an overview of the basic models and principles associated with OOA and SA, a discussion of quality in requirements definition, and a detailed discussion of the research methodology used. A review of relevant research is also presented and directions for further research are suggested. Our findings suggest that the OOA methodology does not necessarily produce better requirements statements.     

A taxonomy of reasoning mechanisms and data synchronization framework for road excavation productivity monitoring

Project management tasks, such as productivity monitoring and cost estimation, require data to be fused from multiple data sources, which are typically spatial and temporal in nature. In order to fuse a pair of spatial and temporal data sources, a number of different types of reasoning mechanisms are needed. This paper presents a taxonomy of spatial and temporal reasoning mechanisms needed to fuse spatial and temporal data sources to support construction productivity monitoring. In addition, the paper also describes two different approaches (i.e., interpolation and nearest neighbor approaches) that can be used to synchronize the temporal and/or spatial data sources. The developed taxonomy has been validated based on representative queries of construction engineers and managers that are identified in previous research studies. The interpolation and nearest neighbor approaches have been validated with real and simulated construction data sources.     

The measurement of software design quality

Software quality involves the conformance of a software product to some predefined set of functional requirements at a specified level of quality. The software is considered valid when it conforms to these “quality factors” at some acceptable level. There are a large number of quality factors against which software may be validated. This paper discusses the development of traditional software metrics in relation to the anticipated structure of a software system. The taxonomy of a software system primarily relies upon the dissection of the software system into modules. Modular design is the cornerstone of quality software, and metrics that can predict an optimum modular structure are critical. By examining the theoretical bases on quality metrics, a base set of common quantitative metrics can be devised and mapped to quality metrics in which they reside. This paper surveys existing metrics and suggests the derivation of software design metrics from software quality factors. Measurable software attributes are identified and suggested as potential design metrics.     

Emulation of Software Faults: A Field Data Study and a Practical Approach

The injection of faults has been widely used to evaluate fault tolerance mechanisms and to assess the impact of faults in computer systems. However, the injection of software faults is not as well understood as other classes of faults (e.g., hardware faults). In this paper, we analyze how software faults can be injected (emulated) in a source-code independent manner. We specifically address important emulation requirements such as fault representativeness and emulation accuracy. We start with the analysis of an extensive collection of real software faults. We observed that a large percentage of faults falls into well-defined classes and can be characterized in a very precise way, allowing accurate emulation of software faults through a small set of emulation operators. A new software fault injection technique (G-SWFIT) based on emulation operators derived from the field study is proposed. This technique consists of finding key programming structures at the machine code-level where high-level software faults can be emulated. The fault-emulation accuracy of this technique is shown. This work also includes a study on the key aspects that may impact the technique accuracy. The portability of the technique is also discussed and it is shown that a high degree of portability can be achieved     

Application of multivariate analysis for software fault prediction

Prediction of fault-prone modules provides one way to support software quality engineering through improved scheduling and project control. The primary goal of our research was to develop and refine techniques for early prediction of fault-prone modules. The objective of this paper is to review and improve an approach previously examined in the literature for building prediction models, i.e. principal component analysis (PCA) and discriminant analysis (DA). We present findings of an empirical study at Ericsson Telecom AB for which the previous approach was found inadequate for predicting the most fault-prone modules using software design metrics. Instead of dividing modules into fault-prone and not-fault-prone, modules are categorized into several groups according to the ordered number of faults. It is shown that the first discriminant coordinates (DC) statistically increase with the ordering of modules, thus improving prediction and prioritization efforts. The authors also experienced problems with the smoothing parameter as used previously for DA. To correct this problem and further improve predictability, separate estimation of the smoothing parameter is shown to be required.     

An improved strategic information management plan for medical institutes

The driving force behind software development of the Electronic Medical Record (EMR) has been gradually changing. Heterogeneous software requirements have emerged, so how to correctly carry out development project has become a complex task. This paper adopts the knowledge engineering and management mechanism, i.e., CommonKADS, and software quality engineering to improve existing strategic information management (SIM) plan as a design methodology to help software implementation for medical institutes. We evaluate the adopting performance by a real case that examines the maturity level of the architecture alignment between the target solution in the proposed SIM plan and the built medical system.     

从用户需求到软件规约：一种问题变换的方法

研究的目的是在获取用户需求和领域描述的基础上规约出对软件规格的描述.提供了一种实现从用户需求到软件规约的平滑和可推理的变换方法.在深入研究问题框架方法的基础上,采用Hoare 的通信顺序进程语言CSP及Lai的最弱环境演算符实现了整个问题图的变换,且导出的软件规格是具有高抽象粒度的程序代码模型,能够被FDR模型检测工具所验证.该工作为实现嵌入式软件开发从需求到软件代码、文档的自动转化及验证等奠定了理论基础.此外,把该理论与模型检测工具FDR联合起来会有助于提高嵌入式软件开发的效率和准确性.     

Artificial Intelligence and Environmental Decision Support Systems

An effective protection of our environment is largely dependent on the quality of the available information used to make an appropriate decision. Problems arise when the quantities of available information are huge and nonuniform (i.e., coming from many different disciplines or sources) and their quality could not be stated in advance. Another associated issue is the dynamical nature of the problem. Computers are central in contemporary environmental protection in tasks such as monitoring, data analysis, communication, information storage and retrieval, so it has been natural to try to integrate and enhance all these tasks with Artificial Intelligence knowledge-based techniques. This paper presents an overview of the impact of Artificial Intelligence techniques on the definition and development of Environmental Decision Support Systems (EDSS) during the last fifteen years. The review highlights the desirable features that an EDSS must show. The paper concludes with a selection of successful applications to a wide range of environmental problems.     

面向软件黑箱测试的仿真环境嵌入故障研究

故障注入作为软件测试的一种有效技术已进入实用阶段, 然而如何在软件黑箱测试中运用故障注入技术目前尚少有文献加以论述.文章提出了软件黑 箱测试中故障外围注入的思想,通过嵌入故障的仿真环境,实现对被测软件输入级的故障引入 ,改变软件的运行状态,诱发内在的失效模式,导致错误的输出,从而达到预期的测试目的.这 一方法已应用于铁路车站信号控制系统软件的测试中,并取得了良好的效果.