Fault prediction and the discriminative powers of connectivity-based object-oriented class cohesion metrics

Context

Several metrics have been proposed to measure the extent to which class members are related. Connectivity-based class cohesion metrics measure the degree of connectivity among the class members.

Objective

We propose a new class cohesion metric that has higher discriminative power than any of the existing cohesion metrics. In addition, we empirically compare the connectivity and non-connectivity-based cohesion metrics.

Method

The proposed class cohesion metric is based on counting the number of possible paths in a graph that represents the connectivity pattern of the class members. We theoretically and empirically validate this path connectivity class cohesion (PCCC) metric. The empirical validation compares seven connectivity-based metrics, including PCCC, and 11 non-connectivity-based metrics in terms of discriminative and fault detection powers. The discriminative-power study explores the probability that a cohesion metric will incorrectly determine classes to be cohesively equal when they have different connectivity patterns. The fault detection study investigates whether connectivity-based metrics, including PCCC, better explain the presence of faults from a statistical standpoint in comparison to other non-connectivity-based cohesion metrics, considered individually or in combination.

Results

The theoretical validation demonstrates that PCCC satisfies the key cohesion properties. The results of the empirical studies indicate that, in contrast to other connectivity-based cohesion metrics, PCCC is much better than any comparable cohesion metric in terms of its discriminative power. In addition, the results also indicate that PCCC measures cohesion aspects that are not captured by other metrics, wherein it is considerably better than other connectivity-based metrics but slightly worse than some other non-connectivity-based cohesion metrics in terms of its ability to predict faulty classes.

Conclusion

PCCC is more useful in practice for the applications in which practitioners need to distinguish between the quality of different classes or the quality of different implementations of the same class.     

Barriers in the selection of offshore software development outsourcing vendors: An exploratory study using a systematic literature review

Context

Software development outsourcing is a contract-based relationship between client and vendor organisations in which a client contracts out all or part of its software development activities to a vendor, who provides agreed services for remuneration.

Objective

The objective is to identify various barriers that have a negative impact on software outsourcing clients in the selection process of offshore software development outsourcing vendors.

Method

We have performed a systematic literature review (SLR) process for the identification of barriers. We have performed all the SLR steps such as the protocol development, initial selection, final selection, quality assessment, data extraction and data synthesis.

Results

We have identified barriers such as ‘language and cultural barriers’, ‘country instability’, ‘lack of project management’, ‘lack of protection for intellectual property rights’ and ‘lack of technical capability’ that generally have a negative impact on outsourcing clients. We have identified only one common frequently cited barrier in three types of organisations (i.e. small, medium and large) which is ‘language and cultural barriers’. We did not identify any common frequently cited barrier in three continents (Asia, North America and Europe) and in two decades (1990-1999 and 2000-mid 2008). The results also reveal the similarities and differences in the barriers identified through different study strategies.

Conclusions

Vendors should address frequently cited barriers such as ‘language and cultural barriers’, ‘country instability’, ‘lack of project management’, ‘lack of protection for intellectual property rights’ and ‘lack of technical capability’ in order to compete in the offshore outsourcing business.     

An automated framework for software test oracle

Context

One of the important issues of software testing is to provide an automated test oracle. Test oracles are reliable sources of how the software under test must operate. In particular, they are used to evaluate the actual results that produced by the software. However, in order to generate an automated test oracle, oracle challenges need to be addressed. These challenges are output-domain generation, input domain to output domain mapping, and a comparator to decide on the accuracy of the actual outputs.

Objective

This paper proposes an automated test oracle framework to address all of these challenges.

Method

I/O Relationship Analysis is used to generate the output domain automatically and Multi-Networks Oracles based on artificial neural networks are introduced to handle the second challenge. The last challenge is addressed using an automated comparator that adjusts the oracle precision by defining the comparison tolerance. The proposed approach was evaluated using an industry strength case study, which was injected with some faults. The quality of the proposed oracle was measured by assessing its accuracy, precision, misclassification error and practicality. Mutation testing was considered to provide the evaluation framework by implementing two different versions of the case study: a Golden Version and a Mutated Version. Furthermore, a comparative study between the existing automated oracles and the proposed one is provided based on which challenges they can automate.

Results

Results indicate that the proposed approach automated the oracle generation process 97% in this experiment. Accuracy of the proposed oracle was up to 98.26%, and the oracle detected up to 97.7% of the injected faults.

Conclusion

Consequently, the results of the study highlight the practicality of the proposed oracle in addition to the automation it offers.     

An experimental assessment of module documentation-based testing

Context

Testing a module that has memory using the black-box approach has been found to be expensive and relatively ineffective. Instead, testing without knowledge of the specifications (white-box approach) may not be effective in showing whether a program has been properly implemented as stated in its specifications. We propose instead a grey-box approach called Module Documentation-based Testing or MD-Test, the heart of which is an automatic generation of the test oracle from the external and internal views of the module.

Objective

This paper presents an empirical analysis and comparison of MD-Test against three existing testing tools.

Method

The experiment was conducted using a mutation-testing approach, in two phases that assess the capability of MD-Test in general and its capability of evaluating test results in particular.

Results

The results of the general assessment indicate that MD-Test is more effective than the other three tools under comparison, where it is able to detect all faults. The second phase of the experiment, which is significant to this study, compares the capabilities of MD-Test and JUnit-black using the test evaluation results. Likewise, an analysis of the test evaluation results shows that MD-Test is more effective and efficient, where MD-Test is able to detect at least the same number of faults as, or is at par with, the black-box approach.

Conclusion

It is concluded that test evaluation using grey-box approach is more effective and efficient that the black-box approach when testing a module that has memory.     

Improving the applicability of object-oriented class cohesion metrics

Context

Class cohesion is an important object-oriented quality attribute. It refers to the degree of relatedness between the methods and attributes of a class. Several metrics have been proposed to measure the extent to which the class members are related. Most of these metrics have undefined values for a relatively high percentage of classes, which limits their applicability. The classes that have undefined values lack methods, attributes, or parameter types, or they include only a single method.

Objective

We improve the applicability of the class cohesion metrics by defining their values for such special classes. In addition, we theoretically and empirically validate the improved metrics.

Method

We theoretically examine whether the defined values satisfy the key cohesion properties. In addition, we empirically validate the metrics before and after the improvements to test whether the defined values improve the ability of the metrics to evaluate class cohesion. We also explore the correlation between the metrics and the presence of faulty classes to indirectly determine the strength or weakness of the metrics in indicating class quality.

Results

The results show that our assigned values for the undefined cases do not violate the key cohesion properties and considerably improve the ability of the metrics to explain the presence of faulty classes and may therefore improve their ability to indicate the quality of the class design.

Conclusions

Having the class cohesion metrics defined for all possible cases improves the applicability of the metrics and potentially increases their precision in indicating class quality.     

Modeling process-related RBAC models with extended UML activity models

Context

Business processes are an important source for the engineering of customized software systems and are constantly gaining attention in the area of software engineering as well as in the area of information and system security. While the need to integrate processes and role-based access control (RBAC) models has been repeatedly identified in research and practice, standard process modeling languages do not provide corresponding language elements.

Objective

In this paper, we are concerned with the definition of an integrated approach for modeling processes and process-related RBAC models - including roles, role hierarchies, statically and dynamically mutual exclusive tasks, as well as binding of duty constraints on tasks.

Method

We specify a formal metamodel for process-related RBAC models. Based on this formal model, we define a domain-specific extension for a standard modeling language.

Results

Our formal metamodel is generic and can be used to extend arbitrary process modeling languages. To demonstrate our approach, we present a corresponding extension for UML2 activity models. The name of our extension is Business Activities. Moreover, we implemented a library and runtime engine that can manage Business Activity runtime models and enforce the different policies and constraints in a software system.

Conclusion

The definition of process-related RBAC models at the modeling-level is an important prerequisite for the thorough implementation and enforcement of corresponding policies and constraints in a software system. We identified the need for modeling support of process-related RBAC models from our experience in real-world role engineering projects and case studies. The Business Activities approach presented in this paper is successfully applied in role engineering projects.     

Formal model for assigning human resources to teams in software projects

Context

Human resources play a critical role in software project success. However, people are still the least formalized factor in today’s process models. Generally, people are assigned to roles and project teams are formed on the basis of project leaders’ experience of people, constraints (e.g. availability) and skill requirements. Yet this process has to take multiple factors into account. Few works in the literature model this process. Most of these are informal proposals focusing on the individual assignment of people to project tasks and do not consider other aspects like team formation as a whole.

Objective

In this paper we formulate a formal model for assigning human resources to software project teams. Additionally, we describe the key results of the knowledge management process enacted to output the elements of the model.

Method

The model elements were identified using the Delphi expert consultation method and applying psychological tests. The proposed model was implemented in a software tool and validated on two software development organization assignment scenarios.

Results

We built a formal model for the process of assigning human resources to software project teams. This model takes into account as many factors as possible and aids the assignment of individuals to project roles, as well as the formation of the team as a whole.We found that the rules that were identified to form software development project teams are useful. From the tests we found that model implementation was feasible (all the executions of the implemented problem-solving algorithms output feasible solutions in response times that can be considered as acceptable).

Conclusion

Using the Delphi method we were able to propose software project roles and competences. Psychological tests and data mining tools identified useful rules for forming software project teams. These were used to build a formal model. This model was built into a tool that returns role assignments in acceptable response times. This decision support tool helps managers assign people to roles and to form teams. Using the tool, project leaders can flexibly evaluate different team make-ups, taking into account several factors, as well as different constraints and objectives.     

Comparing the performance of metaheuristics for the analysis of multi-stakeholder tradeoffs in requirements optimisation

Context

In requirements engineering, there will be many different stake holders. Often the requirements engineer has to find a set of requirements that reflect the needs of several different stake holders, while remaining within budget.

Objective

This paper introduces an optimisation-based approach to the automated analysis of requirements assignments when multiple stake holders are to be satisfied by a single choice of requirements.

Method

The paper reports on experiments using two different multi-objective evolutionary optimisation algorithms with real world data sets as well as synthetic data sets. This empirical validation includes a statistical analysis of the performance of the two algorithms.

Results

The results reveal that the Two-Archive algorithm outperformed the others in convergence as the scale of problems increase. The paper also shows how both traditional and animated Kiviat diagrams can be used to visualise the tensions between the stake holders’ competing requirements in the presence of increasing budgetary pressure.

Conclusion

This paper presented the concept of internal tensioning among multi-stakeholder in requirements analysis and optimisation for the first time. This analysis may be useful in internal negotiations over budgetary allowance for the project.     

A systematic review of research on open source software in commercial software product development

Context

The popularity of the open source software development in the last decade, has brought about an increased interest from the industry on how to use open source components, participate in the open source community, build business models around this type of software development, and learn more about open source development methodologies. There is a need to understand the results of research in this area.

Objective

Since there is a need to understand conducted research, the aim of this study is to summarize the findings of research that has ben carried out on usage of open source components and development methodologies by the industry, as well as companies’ participation in the open source community.

Method

Systematic review through searches in library databases and manual identification of articles from the open source conference. The search was first carried out in May 2009 and then once again in May 2010.

Results

In 2009, 237 articles were first found, from which 19 were selected based on content and quality, and in 2010, 76 new articles were found from which four were selected. Twenty three articles were identified in total.

Conclusions

The articles could be divided into four categories: open source as part of component based software engineering, business models with open source in commercial organization, company participation in open source development communities, and usage of open source processes within a company.     

Contribution-based call stack abstraction for call string based pointer analysis

Context

Different method calls may have different contributions to the precision of the final application when abstracted into the call strings. The existing call string based pointer analysis algorithms do not consider such contribution difference and hence may not achieve best cost-effectiveness.

Objective

To be more cost-effective, we try to leverage the contribution information of each method call in call string based pointer analysis.

Method

The paper firstly proposes a contribution-based call stack abstraction method which abstracts the call stacks into call strings with the contribution information under consideration. Then, we apply the new call stack abstraction method to the pointer analysis of AspectJ programs and propose a concern-sensitive points-to analysis method. Besides, the new abstraction method is also applied to multi-threaded Java programs and results in a thread-sensitive pointer analysis method.

Results

The experimental results show that the two pointer analysis methods with contribution-based call stack abstraction can be more cost-effective than the ordinary call string based approaches for an application that detects harmful advices and an application that detects inter-thread data flow.

Conclusion

These pointer analysis methods more concretely and more clearly show that the contribution-based call stack abstraction can lead to better cost-effectiveness for the given applications.     

A formal approach for the development of reactive systems

Context

This paper deals with the development and verification of liveness properties on reactive systems using the Event-B method. By considering the limitation of the Event-B method to invariance properties, we propose to apply the language TLA⁺ to verify liveness properties on Event-B models.

Objective

This paper deals with the use of two verification approaches: theorem proving and model-checking, in the construction and verification of safe reactive systems. The theorem prover concerned is part of the Click_n_Prove tool associated to the Event-B method and the model checker is TLC for TLA⁺ models.

Method

To verify liveness properties on Event-B systems, we extend first the expressivity and the semantics of a B model (called temporal B model) to deal with the specification of fairness and eventuality properties. Second, we propose semantics of the extension over traces, in the same spirit as TLA⁺ does. Third, we give verification rules in the axiomatic way of the Event-B method. Finally, we give transformation rules from a temporal B model into a TLA⁺ module. We present in particular, our prototype system called B2TLA⁺, that we have developed to support this transformation; then we can verify liveness properties thanks to the model checker TLC on finite state systems. For the verification of infinite-state systems, we propose the use of the predicate diagrams and its associated tool DIXIT. As the B refinement preserves invariance properties through refinement steps, we propose some rules to get the preservation of liveness properties by the B refinement.

Results

The proposed approach is applied for the development of some reactive systems examples and our prototype system B2TLA⁺ is successfully used to transform a temporal B model into a TLA⁺ module.

Conclusion

The paper successfully defines an approach for the specification and verification of safety and liveness properties for the development of reactive systems using the Event-B method, the language TLA⁺ and the predicate diagrams with their associated tools. The approach is illustrated on a case study of a parcel sorting system.     

A systematic approach to integrate common timed security rules within a TEFSM-based system specification

Context

Formal methods are very useful in the software industry and are becoming of paramount importance in practical engineering techniques. They involve the design and modeling of various system aspects expressed usually through different paradigms. These different formalisms make the verification of global developed systems more difficult.

Objective

In this paper, we propose to combine two modeling formalisms, in order to express both functional and security timed requirements of a system to obtain all the requirements expressed in a unique formalism.

Method

First, the system behavior is specified according to its functional requirements using Timed Extended Finite State Machine (TEFSM) formalism. Second, this model is augmented by applying a set of dedicated algorithms to integrate timed security requirements specified in Nomad language. This language is adapted to express security properties such as permissions, prohibitions and obligations with time considerations.

Results

The proposed algorithms produce a global TEFSM specification of the system that includes both its functional and security timed requirements.

Conclusion

It is concluded that it is possible to merge several requirement aspects described with different formalisms into a global specification that can be used for several purposes such as code generation, specification correctness proof, model checking or automatic test generation. In this paper, we applied our approach to a France Telecom Travel service to demonstrate its scalability and feasibility.     

Cross-domain probabilistic inference in a clinical decision support system: examples for dermatology and rheumatology

Introduction

Maintaining a large diagnostic knowledge base (KB) is a demanding task for any person or organization. Future clinical decision support system (CDSS) may rely on multiple, smaller and more focused KBs developed and maintained at different locations that work together seamlessly. A cross-domain inference tool has great clinical import and utility.

Methods

We developed a modified multi-membership Bayes formulation to facilitate the cross-domain probabilistic inferencing among KBs with overlapping diseases. Two KBs developed for evaluation were non-infectious generalized blistering diseases (GBD) and autoimmune diseases (AID). After the KBs were finalized, they were evaluated separately for validity.

Result

Ten cases from medical journal case reports were used to evaluate this “cross-domain” inference across the two KBs. The resultant non-error rate (NER) was 90%, and the average of probabilities assigned to the correct diagnosis (AVP) was 64.8% for cross-domain consultations.

Conclusion

A novel formulation is now available to deal with problems occurring in a clinical diagnostic decision support system with multi-domain KBs. The utilization of this formulation will help in the development of more integrated KBs with greater focused knowledge domains.     

The relationship between organizational culture and the deployment of agile methods

Context

Systems development normally takes place in a specific organizational context, including organizational culture. Previous research has identified organizational culture as a factor that potentially affects the deployment systems development methods.

Objective

The purpose is to analyze the relationship between organizational culture and the post-adoption deployment of agile methods.

Method

This study is a theory development exercise. Based on the Competing Values Model of organizational culture, the paper proposes a number of hypotheses about the relationship between organizational culture and the deployment of agile methods.

Results

Inspired by the agile methods thirteen new hypotheses are introduced and discussed. They have interesting implications, when contrasted with ad hoc development and with traditional systems development methods.

Conclusion

Because of the conceptual richness of organizational culture and the ambiguity of the concept of agility the relationship between organizational culture and the deployment of agile systems development forms a rich and interesting research topic. Recognizing that the Competing Values Model represents just one view of organizational culture, the paper introduces a number of alternative conceptions and identifies several interesting paths for future research into the relationship between organizational culture and agile methods deployment.     

A two-stage framework for UML specification matching

Context

Specification matching techniques are crucial for effective retrieval processes. Despite the prevalence for object-oriented methodologies, little attention has been given to Unified Modeling Language (UML) for matching.

Objective

This paper presents a two-stage framework for matching two UML specifications and quantifying the results based on the systematic integration of their structural and behavioral similarities in order to identify the candidate component set for reuse.

Method

The first stage in the framework is an evaluation of the similarities between UML class diagrams using the Structure-Mapping Engine (SME), a simulation of the analogical reasoning approach known as the structure-mapping theory. The second stage, performed on the components identified in the first stage, is based on a graph-similarity scoring algorithm in which UML class diagrams and sequence diagrams are transformed into an SME representation and a Message-Object-Order Graph (MOOG). The effectiveness of the proposed framework was evaluated using a case study.

Results

The experimental results showed a reduction in potential mismatches and an overall high precision and recall.

Conclusion

It is concluded that the two-stage framework is capable of performing more precise matching compared to those of other single-stage matching frameworks. Moreover, the two-stage framework could be utilized within a reuse process, bypassing the need for extra information for retrieval of the components described by UML.     

Data warehouse testing: A prototype-based methodology

Context

Testing is an essential part of the development life-cycle of any software product. While most phases of data warehouse design have received considerable attention in the literature, not much has been written about data warehouse testing.

Objective

In this paper we propose a comprehensive approach to testing data warehouse systems. Its main features are earliness with respect to the life-cycle, modularity, tight coupling with design, scalability, and measurability through proper metrics.

Method

We introduce a number of specific testing activities, we classify them in terms of what is tested and how it is tested, and we show how they can be framed within a prototype-based methodology. We apply our approach to a real case study for a large retail company.

Results

The case study we faced, based on an iterative prototype-based medium-size project, confirmed the validity of our approach. In particular, the main benefits were obtained in terms of project transparency, coordination of the development team, and organization of design activities.

Conclusion

Though some general-purpose testing techniques can be applied to data warehouse projects, the effectiveness of testing can be largely improved by applying specifically-devised techniques and metrics.