Avoiding metric monsters: A design metrics approach

Metric monsters are stumbling blocks that prevent software metrics-guided methodologies from attaining product and process improvement. Metric monsters can occur during the identification, collection or application of software metrics. In our research, we have developed and tested our design metrics over a five-year period and have found them to be excellent predictors of error-prone modules. Based on this research, we will identify some of the monsters that occur in the quantitative analyses of software and its development processes, and present our approach in formulating a design metrics model that avoids these monsters. This model consists of software tools, guidelines and actions for the application of software design metrics.     

An evaluation of the MOOD set of object-oriented software metrics

This paper describes the results of an investigation into a set of metrics for object-oriented design, called the MOOD metrics. The merits of each of the six MOOD metrics is discussed from a measurement theory viewpoint, taking into account the recognized object-oriented features which they were intended to measure: encapsulation, inheritance, coupling, and polymorphism. Empirical data, collected from three different application domains, is then analyzed using the MOOD metrics, to support this theoretical validation. Results show that (with appropriate changes to remove existing problematic discontinuities) the metrics could be used to provide an overall assessment of a software system, which may be helpful to managers of software development projects. However, further empirical studies are needed before these results can be generalized     

Measuring where it matters: Determining starting points for metrics collection

Defining useful metrics to measure the goals of a software organisation is difficult. Defining useful metrics to measure the causes of the (failure) to fulfil those organisational goals is even more difficult, as the diversity of potential causes makes their measurement illusive. In this article, we describe a method to select useful software metrics based on findings from qualitative research. In a case study, we apply this method to a previously conducted study of project post-mortem reviews to assess the validity of our prior claims. For this we collected data on 109 new software projects in the organisation in which we conducted the previous case study.     

The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process

Many empirical studies have found that software metrics can predict class error proneness and the prediction can be used to accurately group error-prone classes. Recent empirical studies have used open source systems. These studies, however, focused on the relationship between software metrics and class error proneness during the development phase of software projects. Whether software metrics can still predict class error proneness in a system’s post-release evolution is still a question to be answered. This study examined three releases of the Eclipse project and found that although some metrics can still predict class error proneness in three error-severity categories, the accuracy of the prediction decreased from release to release. Furthermore, we found that the prediction cannot be used to build a metrics model to identify error-prone classes with acceptable accuracy. These findings suggest that as a system evolves, the use of some commonly used metrics to identify which classes are more prone to errors becomes increasingly difficult and we should seek alternative methods (to the metric-prediction models) to locate error-prone classes if we want high accuracy.     

An empirical research agenda for understanding formal methods productivity

ContextFormal methods, and particularly formal verification, is becoming more feasible to use in the engineering of large highly dependable software-based systems, but so far has had little rigorous empirical study. Its artefacts and activities are different to those of conventional software engineering, and the nature and drivers of productivity for formal methods are not yet understood.ObjectiveTo develop a research agenda for the empirical study of productivity in software projects using formal methods and in particular formal verification. To this end we aim to identify research questions about productivity in formal methods, and survey existing literature on these questions to establish face validity of these questions. And further we aim to identify metrics and data sources relevant to these questions.MethodWe define a space of GQM goals as an investigative framework, focusing on productivity from the perspective of managers of projects using formal methods. We then derive questions for these goals using Easterbrook et al.’s (2008) taxonomy of research questions. To establish face validity, we document the literature to date that reflects on these questions and then explore possible metrics related to these questions. Extensive use is made of literature concerning the L4.verified project completed within NICTA, as it is one of the few projects to achieve code-level formal verification for a large-scale industrially deployed software system.ResultsWe identify more than thirty research questions on the topic in need of investigation. These questions arise not just out of the new type of project context, but also because of the different artefacts and activities in formal methods projects. Prior literature supports the need for research on the questions in our catalogue, but as yet provides little evidence about them. Metrics are identified that would be needed to investigate the questions. Thus although it is obvious that at the highest level concepts such as size, effort, rework and so on are common to all software projects, in the case of formal methods, measurement at the micro level for these concepts will exhibit significant differences.ConclusionsEmpirical software engineering for formal methods is a large open research field. For the empirical software engineering community our paper provides a view into the entities and research questions in this domain. For the formal methods community we identify some of the benefits that empirical studies could bring to the effective management of large formal methods projects, and list some basic metrics and data sources that could support empirical studies. Understanding productivity is important in its own right for efficient software engineering practice, but can also support future research on cost-effectiveness of formal methods, and on the emerging field of Proof Engineering.     

Efficiency of CAMEtools in software quality assurance

Our paper describes the requirements and possibilities of integration of metrics tools in the field of software quality assurance. Tools for the support of the measurement process are herein classified as Computer Assisted Software Measurement and Evaulation Tools (CAMETools). Software measurement regarded as a special type of metrics application provides a great amount of basic information for the evaluation of the software development process or the software product itself. Our paper examines the effectiveness and destination of software measurement in tool-based software development and is based on an analysis of more than 20 CAME tools in the Software Measurement Laboratory at the University of Magdeburg. CAMEtools are useable for the process, product, and resources evaluation in all phases of the software life cycle (including the problem definition) for different development paradigms. The efficiency of CAME tools is described on the basis of a general measurement framework. This framework includes all steps in the software measurement and evaulation process: metrics definition, selection of the evaluation criteria, tool-based modelling and measurement, value presentation and statistical analysis. The framework includes the main aspects of the process evaluation techniques (Capability Maturity Model, ISO 9000-3 etc.) and product evaluation (ISO 9126, etc.). It is not a disjointed set of aspects: our measurement framework represents an incremental technique for the application of quantification of quality aspects in a required quality assurance     

一种基于多变量Logistic模型的缺陷定位方法

缺陷定位是软件开发过程的重要环节。充分利用程序的结构特征和行为特征有助于提高缺陷定位效率。提出一种基于多变量Logistic回归分析的缺陷定位框架, 用于软件演化时对新版本程序进行类方法级别的缺陷定位。首先设计一组度量结构特征和行为特征的指标, 通过静态分析和测试程序搜集并构建旧版本程序的特征数据集, 同时从缺陷跟踪系统获取旧版本缺陷信息;其次, 基于所得特征数据集和缺陷信息, 应用单变量分析筛选出度量指标中与缺陷显著相关的指标, 随后用选中的显著指标展开多变量分析, 训练多变量Logistic模型;最后, 基于选出的显著指标搜集并构建新版本程序的特征数据集, 运用得到的Logistic模型预测每个类方法的出错概率, 进而按出错概率降序检查类方法以定位错误。基于一组开源程序进行缺陷定位实证研究,结果表明, 多变量Logistic模型可以提高缺陷定位的效率。关键词：     

Benchmarking library and application software with Data Envelopment Analysis

Library software is generally believed to be well-structured and follows certain design guidelines due to the need of continuous evolution and stability of the respective APIs. We perform an empirical study to investigate whether the design of open-source library software is actually superior to that of application software. By analyzing certain design principles and heuristics that are considered important for API design, we extract a set of software metrics that are expected to reflect the improved nature of libraries. An initial comparison by conventional statistical analysis confirms the overall belief that products of different software size scale should not be compared by simply examining metric values in isolation. In this paper, we propose the use of Data Envelopment Analysis (DEA), borrowed from production economics, as a means of measuring and benchmarking the quality of different object-oriented software designs captured by software metrics and apply this approach to the comparison of library and application software. The advantages offered by DEA and the differences between the application of DEA in an economic and a technological context are discussed. Results of the approach are presented for 44 open-source projects, equally divided between libraries and applications.     

Evaluation metrics and methodologies for user-centered evaluation of intelligent systems

In the past four years, we have worked with several research programs that were developing intelligent software for use by intelligence analysts. Our involvement in these programs was to develop the metrics and methodologies for assessing the impact on users; in this case, on intelligence analysts. In particular, we focused on metrics to evaluate how much the intelligent systems contribute to the users’ tasks and what the cost is to the user in terms of workload and process deviations. In this paper, we describe the approach used. We started with two types of preliminary investigations – first, collecting and analyzing data from analysts working in an instrumented environment for a period of 2 years, and second, developing and conducting formative evaluations of research software. The long-term studies informed our ideas about the processes that analysts use and provided potential metrics in an environment without intelligent software tools. The formative evaluations helped us to define sets of application-specific metrics. Finally, we conducted assessments during and after technology insertions. We describe the metrics and methodologies used in each of these activities, along with the lessons learned.     

完善验证的凝聚力测量方法以提升软件品质

无适当使用软件测量将可能引起软件低品质且高成本的窘态.凝聚力是软件品质重要因子之一如同维护度,可靠度和再利用度.软件模组品质的优劣必影响整体系统之品质的优劣.为了设计和维护高品质软件,软件专案经理人和软件工程师无可避免需引用软件凝聚力测量以衡量和产生高品质软件.提出以活路跃变量及视觉化变量纵距为分析基础之功能导向凝聚力测量方法.进而,以一系列实际案例来作实验验证,并以一组性质来作理论辩证所提的测量方法.因此一经完善定义,完善实验和完善辩证之凝聚力测量方法被提出用于当软件凝聚力强度的指标和因此增进软件品质.这凝聚力测量方法能容易嵌入CASE以帮助软件工程师确保软件品质.     

SBFS：基于搜索的软件缺陷预测特征选择框架

软件缺陷预测通过预先识别出被测项目内的潜在缺陷程序模块,有助于合理分配测试资源,并最终提高被测软件产品的质量。但在搜集缺陷预测数据集的时候,由于考虑了大量与代码复杂度或开发过程相关的度量元,造成数据集内存在维数灾难问题。借助基于搜索的软件工程思想,提出一种新颖的基于搜索的包裹式特征选择框架SBFS。该框架在实现时,首先借助SMOTE方法来缓解数据集内存在的类不平衡问题,随后借助基于遗传算法的特征选择方法,基于训练集选出最优特征子集。在实证研究中,以NASA数据集作为评测对象,以基于前向选择策略的包裹式特征选择方法FW、基于后向选择策略的包裹式特征选择BW、不进行特征选择的Origin作为基准方法。最终实证研究结果表明：SBFS方法在90%的情况下,不差于Origin法。在82.3%的情况下,不差于BW法。在69.3%的情况下,不差于FW法。除此之外,我们发现若基于决策树分类器,则应用SMOTE方法后,可以在71%的情况下,提高模型性能。而基于朴素贝叶斯和Logistic回归分类器,则应用SMOTE方法后,仅可以在47%和43%的情况下,提高模型的预测性能。     

Evaluation of software testing metrics for NASA's Mission Control Center

Software metrics are used to evaluate the software developmentprocess and the quality of the resultingproduct. We used five metrics during the testing phase of the Shuttle Mission Control Center (MCC) Upgrade (MCCU) at the National Aeronautics and Space Administration's (NASA) Johnson Space Center. All but one metric provided useful information. Based on our experience we recommend using metrics during the test phase of software development and propose additional candidate metrics for further study.     

Empirical Assessment of a Software Metric: The Information Content of Operators

This paper presents an empirical case study that predicted faults in modules based on the total information content of the operators. This metric is closely related to Harrison's average information content classification (AICC), which is the entropy of the operators. Most information theory-based metrics proposed in the literature have not been subjected to empirical predictive studies of real-world software systems. In contrast, this study shows that a simple information theory-based metric can be more useful for prediction of software quality than comparable metrics based on counts in the context of a commercial software development organization.Three models were considered, all based on operators as an abstraction of software. The model based on information content of the operators made more accurate predictions than two similar models based on the number of operators and the number of unique operators. The purpose of this paper is a fair comparison of the three metrics, rather than developing an optimal model. We have long advocated multivariate models for industrial use. The case study considered three large commercial systems, written in assembly language, and developed consecutively by professional programmers. The first system was used to estimate parameters of the models. The subsequent two were used to evaluate the accuracy of model predictions.     

A Generalized Structural Model of Structured Programs for Software Metrics Definition

There are many different types of models to describe various aspects of software from inception to completion. Some of these models are the basis for defining software metrics. In this paper, we propose a generalized formal structural model for traditional structured programs such as 3/4 GLs. We provide a formal definition using conventional notations for our proposed model covering information flow, control flow and combination of information and control flows.     

A Model-Based Approach to Object-Oriented Software Metrics

The need to improve software productivity and software quality has put forward the research on software metrics technology and the development of software metrics tool to support related activities.To support object-oriented software metrics practice efectively,a model-absed approach to object-oriented software metrics is proposed in this paper.This approach guides the metrics users to adopt the quality metrics model to measure the object-oriented software products .The development of the model can be achieved by using a top-down approach.This approach explicitly proposes the conception of absolute normalization computation and relative normalization computation for a metrics model.Moreover,a generic software metrics tool-Jade Bird Object-Oriented Metrics Tool(JBOOMT)is designed to implement this approach.The parser-based approach adopted by the tool makes the information of the source program accurate and complete for measurement.It supports various customizable hierarchical metrics models and provides a flexible user interface for users to manipulate the models.It also supports absolute and relative normalization mechanisms in different situations.     

面向对象软件度量的分析和研究

软件度量学是软件工程的一个新分支,通过度量反馈的信息来控制开发过程,以提高软件质量。传统的面向过程的软件将数据和操作分离,强调以算法为核心。近年来发展起来的面向对象软件的主要特性包括抽象性、封装性、继承性和多态性,将数据和操作封装,强调以对象为核心。因此面向对象软件度量方法也随之发展起来,著名的是C&K度量方法。还介绍了面向对象的软件可重用性的度量方法。并对面向对象软件的下一步研究阐述了看法。     

Techniques of Successful Application of Factor Analysis in Software Measurement

Metrics in a software sense can be roughly divided into two groups, internal metrics and external metrics. Internal metrics can be measured directly, while external metrics are predicted using internal metrics. Factor analysis is a statistical technique which can be applied to analyze the structure of internal metrics and form a prediction model for external metrics A prediction model generated by using factor analysis needs further validation since factor analysis has a non-unique solution for the factor-loading matrix. The computational methodology to establish a model is presented as well as the validation process. This is illustrated by an empirical study of design metrics and a prediction model for design quality.     

软件生态系统研究综述

软件生态系统加速软件产业的垂直分工和水平整合,改变了软件产业的商业生态.自2003年提出以来,软件生态系统吸引工业界和学术界的广泛关注.国内外知名IT企业深耕软件生态系统,形成许多具有广泛影响的软件生态系统.近年来,在云计算、移动应用开发、信息物理系统、区块链等新兴信息技术快速发展和软件生态系统应用领域不断拓展的双重作用下,软件生态系统新泛型不断涌现.人们对软件生态系统的认识与理解,随着研究深度和应用广度的不断变化而改变.直至2016年,学术界和工业界才对软件生态系统的定义趋于一致.软件生态系统是在特定的技术环境中软件企业、开发者、技术社区等涉众以软件产品、服务、数据和知识为媒介相互作用而形成的复杂系统.通常,研究人员使用2003年提出的软件生态系统研究内容七分类法把相关文献归类到过程与技术、定性分析、结构化分析模型、实证研究、工具及表示法、行业应用、技术报告等七大类中.由于软件生态系统的领域边界、关键特征较之2003年的情境已发生巨变,原有的七分类法有待修订.本文类比自然生态系统,建立软件生态系统元模型来描述软件生态的结构及特征.在软件生态系统研究内容七分类法的基础上,将“定性分析”和“结构化分析模型”整合为“分析方法与框架”,把“行业应用”和“实证研究”合并为“实证研究”,形成软件生态系统研究内容五分类法.然后,对2015~2017年软件生态系统的研究现状进行文献综述,从技术报告、分析方法与框架、过程与技术、实证研究、工具及表示法五个方面总结软件生态系统的研究现状,并展望研究趋势及发现面临的挑战.软件生态系统的需求工程、架构建模、模型驱动开发、能量机理、关键特征分析、信息量抽取与建模、生态网络分析、缺陷/代码坏味累积效应、CASE工具、软件生态系统新泛型等方面的研究有待进一步深入.     

构件软件的回归测试复杂性度量

基于构件的软件构建方法目前被广泛使用在软件开发中,用于减少软件开发的工程成本和加快软件开发进度.在软件维护过程中,由于构件更新或者新版本的发布,基于构件的系统会受到影响,需要进行回归测试.对于指定的软件修改需求,维护者可以实施不同的修改手段.不同的修改手段会导致不同的回归测试复杂性,这种复杂性是软件维护成本和有效性的重要因素.目前的研究没有强调构件软件的回归测试复杂性问题.基于修改影响复杂性模型和度量,提出一种回归测试的复杂性度量框架.该度量框架包括两个部分:基于图的模型和形式化度量计算.该度量可以有效表示构件软件分别在构件和系统层面的回归测试复杂性因素,可视化地体现复杂性变化.然后根据模型,提出具体的度量计算方式.最后,通过实验研究,针对同一个构件软件的相同修改需求,利用若干个实验组进行独立修改实施,然后比较回归测试的复杂性.实验结果表明,所提出的度量方式是可行和有效的.