A new calibration for Function Point complexity weights

Function Point (FP) is a useful software metric that was first proposed 25 years ago, since then, it has steadily evolved into a functional size metric consolidated in the well-accepted Standardized International Function Point Users Group (IFPUG) Counting Practices Manual – version 4.2. While software development industry has grown rapidly, the weight values assigned to count standard FP still remain same, which raise critical questions about the validity of the weight values. In this paper, we discuss the concepts of calibrating Function Point, whose aims are to estimate a more accurate software size that fits for specific software application, to reflect software industry trend, and to improve the cost estimation of software projects. A FP calibration model called Neuro-Fuzzy Function Point Calibration Model (NFFPCM) that integrates the learning ability from neural network and the ability to capture human knowledge from fuzzy logic is proposed. The empirical validation using International Software Benchmarking Standards Group (ISBSG) data repository release 8 shows a 22% accuracy improvement of mean magnitude relative error (MMRE) in software effort estimation after calibration.     

Metric Analysis and Data Validation Across Fortran Projects

The desire to predict the effort in developing or explain the quality of software has led to the proposal of several metrics in the literature. As a step toward validating these metrics, the Software Engineering Laboratory has analyzed the Software Science metrics, cyclomatic complexity, and various standard program measures for their relation to 1) effort (including design through acceptance testing), 2) development errors (both discrete and weighted according to the amount of time to locate and frix), and 3) one another. The data investigated are collected from a production Fortran environment and examined across several projects at once, within individual projects and by individual programmers across projects, with three effort reporting accuracy checks demonstrating the need to validate a database. When the data come from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither Software Science's E metric, cyclomatic complexity nor source lines of code appears to relate convincingly better with effort than the others     

A fuzzy logic based approach for phase-wise software defects prediction using software metrics

ContextThe software defect prediction during software development has recently attracted the attention of many researchers. The software defect density indicator prediction in each phase of software development life cycle (SDLC) is desirable for developing a reliable software product. Software defect prediction at the end of testing phase may not be more beneficial because the changes need to be performed in the previous phases of SDLC may require huge amount of money and effort to be spent in order to achieve target software quality. Therefore, phase-wise software defect density indicator prediction model is of great importance.ObjectiveIn this paper, a fuzzy logic based phase-wise software defect prediction model is proposed using the top most reliability relevant metrics of the each phase of the SDLC.MethodIn the proposed model, defect density indicator in requirement analysis, design, coding and testing phase is predicted using nine software metrics of these four phases. The defect density indicator metric predicted at the end of the each phase is also taken as an input to the next phase. Software metrics are assessed in linguistic terms and fuzzy inference system has been employed to develop the model.ResultsThe predictive accuracy of the proposed model is validated using twenty real software project data. Validation results are satisfactory. Measures based on the mean magnitude of relative error and balanced mean magnitude of relative error decrease significantly as the software project size increases.ConclusionIn this paper, a fuzzy logic based model is proposed for predicting software defect density indicator at each phase of the SDLC. The predicted defects of twenty different software projects are found very near to the actual defects detected during testing. The predicted defect density indicators are very helpful to analyze the defect severity in different artifacts of SDLC of a software project.     

Complexity measures for assembly language programs

This paper presents the results of a study of the software complexity characteristics of a large real-time signal processing system for which there is a 6-yr maintenance history. The objective of the study was to compare values generated by software metrics to the maintenance history in order to determine which software complexity metrics would be most useful for estimating maintenance effort. The metrics that were analyzed were program size measures, software science measures, and control flow measures. During the course of the study two new software metrics were defined. The new metrics, maximum knot depth and knots per jump ratio, are both extensions of the knot count metric. When comparing the metrics to the maintenance data the control flow measures showed the strongest positive correlation.     

Software Effort, Quality, and Cycle Time: A Study of CMM Level 5 Projects

The Capability Maturity Model (CMM) has become a popular methodology for improving software development processes with the goal of developing high-quality software within budget and planned cycle time. Prior research literature, while not exclusively focusing on CMM level 5 projects, has identified a host of factors as determinants of software development effort, quality, and cycle time. In this study, we focus exclusively on CMM level 5 projects from multiple organizations to study the impacts of highly mature processes on effort, quality, and cycle time. Using a linear regression model based on data collected from 37 CMM level 5 projects of four organizations, we find that high levels of process maturity, as indicated by CMM level 5 rating, reduce the effects of most factors that were previously believed to impact software development effort, quality, and cycle time. The only factor found to be significant in determining effort, cycle time, and quality was software size. On the average, the developed models predicted effort and cycle time around 12 percent and defects to about 49 percent of the actuals, across organizations. Overall, the results in this paper indicate that some of the biggest rewards from high levels of process maturity come from the reduction in variance of software development outcomes that were caused by factors other than software size     

Measuring effectiveness of HCI integration in software development processes

Integrating human-computer interaction (HCI) activities in software engineering (SE) processes is an often-expressed desire. Two metrics to demonstrate the impact of integrating HCI activities in SE processes are proposed. Usability Goals Achievement Metric (UGAM) is a product metric that measures the extent to which the design of a product achieves its user-experience goals. Index of Integration (IoI) is a process metric that measures the extent of integration of the HCI activities in the SE process. Both the metrics have an organizational perspective and can be applied to a wide range of products and projects. An attempt has been made to keep the metrics easy to use in the industrial context. While the two metrics were proposed mainly to establish a correlation between the two and thereby demonstrate the effectiveness of integration of HCI in SE processes, several other applications seem likely. The two metrics were evaluated in three independent studies: a classroom-based evaluation with two groups of students, a qualitative feedback from three industry projects, and a quantitative evaluation using 61 industry projects. The metrics were found to be useful, easy to use, and helpful in making the process more systematic. Our studies showed that the two metrics correlate well with each other and that IoI is a good predictor of UGAM. Regression analysis showed that IoI has a somewhat greater effect on UGAM in projects that use the agile process model than the waterfall process and in the projects that are executed as a contracted software development service than in the projects in product companies. UGAM also correlated well with the traditional usability evaluations.     

Quality assessment in competition-based software crowdsourcing

Quality assessment is a critical component in crowdsourcing-based software engineering (CBSE) as software products are developed by the crowd with unknown or varied skills and motivations. In this paper, we propose a novel metric called the project score to measure the performance of projects and the quality of products for competitionbased software crowdsourcing development (CBSCD) activities. To the best of our knowledge, this is the first work to deal with the quality issue of CBSE in the perspective of projects instead of contests. In particular, we develop a hierarchical quality evaluation framework for CBSCD projects and come up with two metric aggregation models for project scores. The first model is a modified squale model that can locate the software modules of poor quality, and the second one is a clustering-based aggregationmodel, which takes different impacts of phases into account. To test the effectiveness of the proposed metrics, we conduct an empirical study on TopCoder, which is a famous CBSCD platform. Results show that the proposed project score is a strong indicator of the performance and product quality of CBSCD projects.We also find that the clustering-based aggregation model outperforms the Squale one by increasing the percentage of the performance evaluation criterion of aggregation models by an additional 29%. Our approach to quality assessment for CBSCD projects could potentially facilitate software managers to assess the overall quality of a crowdsourced project consisting of programming contests.     

A practical view of software measurement and implementationexperiences within Motorola

A practical view of software measurement that formed the basis for a companywide software metrics initiative within Motorola is described. A multidimensional view of measurement is provided by identifying different dimensions (e.g., metric usefulness/utility, metric types or categories, metric audiences, etc.) that were considered in this companywide metrics implementation process. The definitions of the common set of Motorola software metrics, as well as the charts used for presenting these metrics, are included. The metrics were derived using the goal/question metric approach to measurement. A distinction is made between the use of metrics for process improvement over time across projects and the use of metrics for in-process project control. Important experiences in implementing the software metrics initiative within Motorola are also included     

A Methodology for Development of Software Composite-Metrics for Program Control-Flow Structure

is paper reports on a pioneer effort for the establishment of a software compostie-metric with key capability of distinguishing among different structrues.As a part of this effort most of the previously proposed program control-flow complexity metrics are evaluated.It is obseved that most of these metrics are inhrently limited in distinguishing capability.However,the concept of composite metrics in potentially useful for the development of a practical metrics.This paper presents a methology for the development of a practical composite metric using statistical techniques.The proposed metric differs from all previous metrics in 2 ways:(1)It is based on an overall structural analysis of a given program in deeper and broader context.It captures various structural measurements taken from all existing structural levels;(2)It unifies a set of 19 important structural metrics.The compositing model of these metrics in based on statistical techniques rather than on an arbitrary method.Experinces with the proposd metric clearly indicate that it distinguishes different structures better than the previous metrics.     

Source code size estimation approaches for object-oriented systems from UML class diagrams: A comparative study

BackgroundSource code size in terms of SLOC (source lines of code) is the input of many parametric software effort estimation models. However, it is unavailable at the early phase of software development.ObjectiveWe investigate the accuracy of early SLOC estimation approaches for an object-oriented system using the information collected from its UML class diagram available at the early software development phase.MethodWe use different modeling techniques to build the prediction models for investigating the accuracy of six types of metrics to estimate SLOC. The used techniques include linear models, non-linear models, rule/tree-based models, and instance-based models. The investigated metrics are class diagram metrics, predictive object points, object-oriented project size metric, fast&&serious class points, objective class points, and object-oriented function points.ResultsBased on 100 open-source Java systems, we find that the prediction model built using object-oriented project size metric and ordinary least square regression with a logarithmic transformation achieves the highest accuracy (mean MMRE = 0.19 and mean Pred(25) = 0.74).ConclusionWe should use object-oriented project size metric and ordinary least square regression with a logarithmic transformation to build a simple, accurate, and comprehensible SLOC estimation model.     

Communication metrics for software development

Presents empirical evidence that metrics on communication artifacts generated by groupware tools can be used to gain significant insight into the development process that produced them. We describe a test-bed for developing and testing communication metrics, a senior-level software engineering project course at Carnegie Mellon University, in which we conducted several studies and experiments from 1991-1996 with more than 400 participants. Such a test-bed is an ideal environment for empirical software engineering, providing sufficient realism while allowing for controlled observation of important project parameters. We describe three proof-of-concept experiments to illustrate the value of communication metrics in software development projects. Finally, we propose a statistical framework based on structural equations for validating these communication metrics     

Statische Analyse von Java-Anwendungen — Eignen sich Lines-of-Code-Metrik und Halstead-Länge?

Many of the recently developed software systems are implemented in Java. For these systems, activities presently are mainly related to software development tasks rather than to dedicated software maintenance tasks. For these Java systems, therefore, experimental confirmation of established metrics for measuring code quantities that are related to software maintenance is not available. This also includes very basic size measures such as the LOC metric and the Halstead length. In this article, the application of these metrics for Java systems as well as some of the associated difficulties are outlined. The presented results are based on experimental data and include empirical correlations between the basic size metrics as well as newly derived scaling laws which are suitable for maintenance related software measurement.     

An empirical study of object-oriented system evolution

Software metrics have been used to measure software artifacts statically—measurements are taken after the artifacts are created. In this study, three metrics—System Design Instability (SDI), Class Implementation Instability (CII), and System Implementation Instability (SII)—are used for the purpose of measuring object-oriented (OO) software evolution. The metrics are used to track the evolution of an OO system in an empirical study. We found that once an OO project starts, the metrics can give good indications of project progress, e.g. how mature the design and implementation is. This information can be used to adjust the project plan in real time.We also performed a study of design instability that examines how the implementation of a class can affect its design. This study determines that some aspects of OO design are independent of implementation, while other aspects are dependent on implementation.     

Metrics to Assess the Likelihood of Project Success Based on Architecture Reviews

Architecture audits are performed very early in the software development lifecycle, typically before low level design or code implementation has begun. An empirical study was performed to assess metrics developed to predict the likelihood of risk of failure of a project. The study used data collected during 50 architecture audits performed over a period of two years for large industrial telecommunications systems. The purpose of such a predictor was to identify at a very early stage, projects that were likely to be at high risk of failure. This would enable the project to take corrective action before significant resources had been expended using a problematic architecture. Detailed information about seven of the 50 projects is presented, and a discussion of how the proposed metric rated each of these projects is presented., A comparison is made of the metric's evaluation and the assessment of the project made by reviewers during the review process.     

Quality metrics for intranet applications

As the number of intranet application increases, software developers face a new software paradigm and possibly a new set of quality requirement. The work discussed here attempts to identify practical software metrics for intranet applications. The six software quality characteristics and 32 quality sub-characteristics of the Extended ISO model are used as a basis to identify the key quality characteristics of intranet applications. From the results of a user survey, three key quality characteristics are identified; they are reliability, functionality, and efficiency. Five sub-characteristics (availability, accuracy, security, suitability and time behaviour) are found to be the key attributes of intranet applications. Finally, a set of three metrics is developed. In order to verify their validity and applicability to intranet projects, an experiment was performed by computing these metrics in five intranet applications. The metric values were then compared with the results from a user satisfaction survey. The metric values and the survey results were closely correlated; a larger score in software metrics leads to a higher user satisfaction score. It is very likely that these software metrics can effectively reflect the quality of an intranet application.     

基于CMM第五级上的软件开发成本、质量和开发周期的回归分析

软件开发中,处于CMM第五级的开发团队拥有相对较高的过程成熟,而高度成熟的过程对开发成本、质量和开发周期具有重要的影响。使用线性回归模型对4个CMM第五级的团队所开发的37个项目进行数据采集和研究后,发现此时大部分被认为影响软件开发成本、质量和开发周期的因素的作用被削弱了,而其中最重要的因素是软件规模。     

Deriving thresholds of software metrics to predict faults on open source software: Replicated case studies

Object-oriented metrics aim to exhibit the quality of source code and give insight to it quantitatively. Each metric assesses the code from a different aspect. There is a relationship between the quality level and the risk level of source code. The objective of this paper is to empirically examine whether or not there are effective threshold values for source code metrics. It is targeted to derive generalized thresholds that can be used in different software systems. The relationship between metric thresholds and fault-proneness was investigated empirically in this study by using ten open-source software systems. Three types of fault-proneness were defined for the software modules: non-fault-prone, more-than-one-fault-prone, and more-than-three-fault-prone. Two independent case studies were carried out to derive two different threshold values. A single set was created by merging ten datasets and was used as training data by the model. The learner model was created using logistic regression and the Bender method. Results revealed that some metrics have threshold effects. Seven metrics gave satisfactory results in the first case study. In the second case study, eleven metrics gave satisfactory results. This study makes contributions primarily for use by software developers and testers. Software developers can see classes or modules that require revising; this, consequently, contributes to an increment in quality for these modules and a decrement in their risk level. Testers can identify modules that need more testing effort and can prioritize modules according to their risk levels.     

Bayesian statistical effort prediction models for data-centred 4GL software development

Constructing an accurate effort prediction model is a challenge in Software Engineering. This paper presents three Bayesian statistical software effort prediction models for database-oriented software systems, which are developed using a specific 4GL toolsuite. The models consist of specification-based software size metrics and development team's productivity metric. The models are constructed based on the subjective knowledge of human expert and calibrated using empirical data collected from 17 software systems developed in the target environment. The models' predictive accuracy is evaluated using subsets of the same data, which were not used for the models' calibration. The results show that the models have achieved very good predictive accuracy in terms of MMRE and pred measures. Hence, it is confirmed that the Bayesian statistical models can predict effort successfully in the target environment. In comparison with commonly used multiple linear regression models, the Bayesian statistical models'predictive accuracy is equivalent in general. However, when the number of software systems used for the models' calibration becomes smaller than five, the predictive accuracy of the best Bayesian statistical models are significantly better than the multiple linear regression model. This result suggests that the Bayesian statistical models would be a better choice when software organizations/practitioners do not posses sufficient empirical data for the models' calibration. The authors expect these findings to encourage more researchers to investigate the use of Bayesian statistical models for predicting software effort.     

Measuring Developers: Aligning Perspectives and Other Best Practices

The paper discusses the software metrics programs. Software metrics programs might rank among the all-time touchiest subjects in software development. Done well, a measurement program can prove an effective tool for keeping on top of development effort-especially for large, distributed projects. It can help developers feel that they have a fair and objective way of communicating their progress and getting resources allocated where they're needed most. An effective metrics program must address both products and processes. Product-related metrics can be tied to customer satisfaction. They often represent objectively verifiable properties such as the number of bugs or defects found in a product. Process metrics, include measures of effort and effectiveness and can give quick feedback about the status of defect containment, productivity, and other desirable properties at many points during development. However, instituting these metrics poses different challenges that the authors discussed and developed some best practice solutions. These include design measures with multiple stakeholders, build on measures that come naturally out of existing processes, and transparency.