Architecture analysis of enterprise systems modifiability - Models, analysis, and validation

Enterprise architecture (EA) models can be used in order to increase the general understanding of enterprise systems and to perform various kinds of analysis. This paper presents instantiated architectural models based on a metamodel for enterprise systems modifiability analysis, i.e. for assessing the cost of making changes to enterprise-wide systems. The instantiated architectural models detailed are based on 21 software change projects conducted at four large Nordic companies. Probabilistic relational models (PRMs) are used for formalizing the EA analysis approach. PRMs enable the combination of regular entity-relationship modeling aspects with means to perform enterprise architecture analysis under uncertainty. The modifiability metamodel employed in the analysis is validated with survey and workshop data (in total 110 experts were surveyed) and with the data collected in the 21 software change projects. Validation indicates that the modifiability metamodel contains the appropriate set of elements. It also indicates that the metamodel produces estimates within a 75% accuracy in 87% of the time and has a mean accuracy of 88% (when considering projects of 2000 man-hours or more).     

Function point analysis: evaluation of a software cost estimation model

The merits of function point analysis are discussed. Function point analysis was chosen because it is one of the most widely used aids for software cost estimation. In the analysis we use data from a large survey of Dutch organizations, from an experiment on the effectiveness of software cost estimation models and from a field study aimed at the adjustment part of the FPA model. Conclusions show that it is indeed widely used and that it performs reasonably well as a product sizer, but that the adjustment part of the model is less useful.     

Evidence-based cost estimation better-quality for software

Our work on COSEEKMO is hardly enough to change experimental methods in the cost estimation community. So, we're also running a new workshop series called PROMISE devoted to repeatable software engineering experiments. Evidence-based reasoning is becoming common in many fields. Evidence-based approaches demand that, among other things, practitioners systematically track down the best evidence relating to some practice; critically relating to some practice; critically appraise that evidence for validity, impact, and applicability; and carefully document it. The software community can bring to bear many methods that could further improve evidence-based cost estimation.     

An empirical analysis of data preprocessing for machine learning-based software cost estimation

ContextDue to the complex nature of software development process, traditional parametric models and statistical methods often appear to be inadequate to model the increasingly complicated relationship between project development cost and the project features (or cost drivers). Machine learning (ML) methods, with several reported successful applications, have gained popularity for software cost estimation in recent years. Data preprocessing has been claimed by many researchers as a fundamental stage of ML methods; however, very few works have been focused on the effects of data preprocessing techniques.ObjectiveThis study aims for an empirical assessment of the effectiveness of data preprocessing techniques on ML methods in the context of software cost estimation.MethodIn this work, we first conduct a literature survey of the recent publications using data preprocessing techniques, followed by a systematic empirical study to analyze the strengths and weaknesses of individual data preprocessing techniques as well as their combinations.ResultsOur results indicate that data preprocessing techniques may significantly influence the final prediction. They sometimes might have negative impacts on prediction performance of ML methods.ConclusionIn order to reduce prediction errors and improve efficiency, a careful selection is necessary according to the characteristics of machine learning methods, as well as the datasets used for software cost estimation.     

Calibrating a software cost estimation model: why and how

Calibration, has been found to be difficult in practice. Wide experience in using the estimation model is necessary; experience which the beginner naturally lacks. This paper indicates why it is important to calibrate a model and how the inexperienced user can be helped by an expert system. In addition, the development of, and experience with, the prototype of an expert system are described. The system dealt with here is intended for the calibration of the PRICE SP estimation.     

Understanding cost drivers of software evolution: a quantitative and qualitative investigation of change effort in two evolving software systems

Making changes to software systems can prove costly and it remains a challenge to understand the factors that affect the costs of software evolution. This study sought to identify such factors by investigating the effort expended by developers to perform 336 change tasks in two different software organizations. We quantitatively analyzed data from version control systems and change trackers to identify factors that correlated with change effort. In-depth interviews with the developers about a subset of the change tasks further refined the analysis. Two central quantitative results found that dispersion of changed code and volatility of the requirements for the change task correlated with change effort. The analysis of the qualitative interviews pointed to two important, underlying cost drivers: Difficulties in comprehending dispersed code and difficulties in anticipating side effects of changes. This study demonstrates a novel method for combining qualitative and quantitative analysis to assess cost drivers of software evolution. Given our findings, we propose improvements to practices and development tools to manage and reduce the costs.     

A method of activity-based software maintenance cost estimation for package software

The Journal of Supercomputing - This paper defines software maintenance activities and develops a model for maintenance cost estimation of package software. First, we classified software...     

Hybrid morphological methodology for software development cost estimation

In this paper we propose a hybrid methodology to design morphological-rank-linear (MRL) perceptrons in the problem of software development cost estimation (SDCE). In this methodology, we use a modified genetic algorithm (MGA) to optimize the parameters of the MRL perceptron, as well as to select an optimal input feature subset of the used databases, aiming at a higher accuracy level for SDCE problems. Besides, for each individual of MGA, a gradient steepest descent method is used to further improve the MRL perceptron parameters supplied by MGA. Finally, we conduct an experimental analysis with the proposed methodology using six well-known benchmark databases of software projects, where two relevant performance metrics and a fitness function are used to assess the performance of the proposed methodology, which is compared to classical machine learning models presented in the literature.     

Optimal test distributions for software failure cost estimation

We generalize the input domain based software reliability measures by E.C. Nelson (1973) and by S.N. Weiss and E.J. Weyuker (1988), introducing expected failure costs under the operational distribution as a measure for software unreliability. This approach incorporates in the reliability concept a distinction between different degrees of failure severity. It is shown how to estimate the proposed quantity by means of random testing, using the Importance Sampling technique from Rare Event Simulation. A test input distribution that yields an unbiased estimator with minimum variance is determined. The practical application of the presented method is outlined, and a detailed numerical example is given     

LSEbA: least squares regression and estimation by analogy in a semi-parametric model for software cost estimation

The importance of Software Cost Estimation at the early stages of the development life cycle is clearly portrayed by the utilization of several models and methods, appeared so far in the literature. The researchers’ interest has been focused on two well known techniques, namely the parametric Regression Analysis and the non-parametric Estimation by Analogy. Despite the several comparison studies, there seems to be a discrepancy in choosing the best prediction technique between them. In this paper, we introduce a semi-parametric technique, called LSEbA that achieves to combine the aforementioned methods retaining the advantages of both approaches. Furthermore, the proposed method is consistent with the mixed nature of Software Cost Estimation data and takes advantage of the whole pure information of the dataset even if there is a large amount of missing values. The paper analytically illustrates the process of building such a model and presents the experimentation on three representative datasets verifying the benefits of the proposed model in terms of accuracy, bias and spread. Comparisons of LSEbA with linear regression, estimation by analogy and a combination of them, based on the average of their outcomes are made through accuracy metrics, statistical tests and a graphical tool, the Regression Error Characteristic curves.     

Improving analogy-based software cost estimation by a resampling method

Estimation by analogy (EbA) is a well-known technique for software cost estimation. The popularity of the method is due to its straightforwardness and its intuitively appealing interpretation. However, in spite of the simplicity in application, the theoretical study of EbA is quite complicated. In this paper, we exploit the relation of EbA method to the nearest neighbor non-parametric regression in order to suggest a resampling procedure, known as iterated bagging, for reducing the prediction error. The improving effect of iterated bagging on EbA is validated using both artificial and real datasets from the literature, obtaining very promising results.     

CHOOSE: Towards a metamodel for enterprise architecture in small and medium-sized enterprises

Enterprise architecture (EA) is a coherent whole of principles, methods, and models that are used in the design and realization of an enterprise’s organizational structure, business processes, information systems, and IT infrastructure. Recent research indicates the need for EA in small and medium-sized enterprises (SMEs), important drivers of the economy, as they struggle with problems related to a lack of structure and overview of their business. However, existing EA frameworks are perceived as too complex and, to date, none of the EA approaches are sufficiently adapted to the SME context. Therefore, this paper presents the CHOOSE metamodel for EA in SMEs that was developed and evaluated through action research in an SME and further refined and validated through case study research in five other SMEs. This metamodel is based on the essential dimensions of EA frameworks and is kept simple so that it may be applied in an SME context. The final CHOOSE metamodel includes only four essential concepts (i.e. goal, actor, operation, object), one for each most frequently used EA focus. As an example, an extract is included from the specific model that was created for the SME used in our action research. Finally, the CHOOSE metamodel is evaluated according to the dimensions essential in EA and the requirements for EA in an SME context.     

适用于软件成本估计及风险评估的混合方法

论文提出了一种用于软件成本估计以及风险评估的方法。该方法将基于算法模型与基于经验的两种成本估计方法相结合,一方面以软件项目基础数据作为评估基础,另一方面则利用了专家知识。另外,该方法还可应用于软件风险评估。为了说明该方法以及证明其可行性,文中提供了一个案例研究。该案例详细说明如何构建成本费用估计模型以及如何利用该方法进行软件风险评估。     

A new perspective on data homogeneity in software cost estimation: a study in the embedded systems domain

Cost estimation and effort allocation are the key challenges for successful project planning and management in software development. Therefore, both industry and the research community have been working on various models and techniques to accurately predict the cost of projects. Recently, researchers have started debating whether the prediction performance depends on the structure of data rather than the models used. In this article, we focus on a new aspect of data homogeneity, “cross- versus within-application domain”, and investigate what kind of training data should be used for software cost estimation in the embedded systems domain. In addition, we try to find out the effect of training dataset size on the prediction performance. Based on our empirical results, we conclude that it is better to use cross-domain data for embedded software cost estimation and the optimum training data size depends on the method used.     

A model for software development effort and cost estimation

Several algorithmic models have been proposed to estimate software costs and other management parameters. Early prediction of completion time is absolutely essential for proper advance planning and aversion of the possible ruin of a project. L.H. Putnam's (1978) SLIM (Software LIfecycle Management) model offers a fairly reliable method that is used extensively to predict project completion times and manpower requirements as the project evolves. However, the nature of the Norden/Rayleigh curve used by Putnam renders it unreliable during the initial phases of the project, especially in projects involving a fast manpower buildup, as is the case with most software projects. In this paper, we propose the use of a model that improves early prediction considerably over the Putnam model. An analytic proof of the model's improved performance is also demonstrated on simulated data     

Visual comparison of software cost estimation models by regression error characteristic analysis

The well-balanced management of a software project is a critical task accomplished at the early stages of the development process. Due to this requirement, a wide variety of prediction methods has been introduced in order to identify the best strategy for software cost estimation. The selection of the best technique is usually based on measures of error whereas in more recent studies researchers use formal statistical procedures. The former approach can lead to unstable and erroneous results due to the existence of outlying points whereas the latter cannot be easily presented to non-experts and has to be carried out by an expert with statistical background. In this paper, we introduce the regression error characteristic (REC) analysis, a powerful visualization tool with interesting geometrical properties, in order to validate and compare different prediction models easily, by a simple inspection of a graph. Moreover, we propose a formal framework covering different aspects of the estimation process such as the calibration of the prediction methodology, the identification of factors that affect the error, the investigation of errors on certain ranges of the actual cost and the examination of the distribution of the cost for certain errors. Application of REC analysis to the ISBSG10 dataset for comparing estimation by analogy and linear regression illustrates the benefits and the significant information obtained.     

支持CMMI的软件过程元模型的研究与实现

为解决软件组织在软件项目开发过程中难以实施CMMI(capability maturity model integration)过程改进的问题,基于模型驱动架构的思想和技术,提出了一种裁剪+重型扩展+轻型扩展的元模型建模策略.给出了该建模策略下的具体实施步骤和方法,实现了一种支持CMMI过程改进的软件过程元建模,同时给出了基于该元模型的建模实例.实验结果表明,该元模型有助于软件组织建立体现其组织特征并有效支持符合CMMI框架的软件过程用户模型.     

Corrigenda: software size estimation of object-oriented systems

In an interesting paper, L.A. Laranjeira (see ibid., vol.6, no.5, p.510-22, 1990) describes a first attempt to understand cost estimation within an object oriented environment. While the presented approach presents many interesting and useful ideas, it is, unfortunately, marred by several mathematical errors pertaining to statistics, exponential functions, and the nature of discrete vs. continuous data. These are discussed here and more appropriate correct procedures outlined     

Cyber-physical systems challenges: a needs analysis for collaborating embedded software systems

Embedding computing power in a physical environment has provided the functional flexibility and performance necessary in modern products such as automobiles, aircraft, smartphones, and more. As product features came to increasingly rely on software, a network infrastructure helped factor out common hardware and offered sharing functionality for further innovation. A logical consequence was the need for system integration. Even in the case of a single original end manufacturer who is responsible for the final product, system integration is quite a challenge. More recently, there have been systems coming online that must perform system integration even after deployment—that is, during operation. This has given rise to the cyber-physical systems (CPS) paradigm. In this paper, select key enablers for a new type of system integration are discussed. The needs and challenges for designing and operating CPS are identified along with corresponding technologies to address the challenges and their potential impact. The intent is to contribute to a model-based research agenda in terms of design methods, implementation technologies, and organization challenges necessary to bring the next-generation systems online.     

基于度量工具的软件成本估算模型使用方法

输入的主观性以及输入过多是妨碍软件成本估算模型实际应用效果的重要影响因素。针对以上问题,提出了一种基于度量工具的软件成本估算模型使用方法。该方法通过引入统计理论中的工具变量,将度量工具所采集的度量元数据自动转换为软件成本估算模型的输入。这一方面可以避免模型校准和估算过程中输入的主观性与不一致性,提高了估算结果的准确性与可靠性;另一方面能减少估算人员的手工操作,提高工作效率,增加了软件成本估算模型的可用性。结合具体实例说明了所提出方法的可行性与有效性。