A software system evaluation framework

The objective of a software system evaluation framework is to assess the quality and sophistication of software from different points of view. The framework explicitly links process and product aspects with the ultimate utility of systems and it provides a basic set of attributes to characterize the important dimensions of software systems. We describe such a framework and its levels of categorization, and we analyze examples of project classifications. Then we draw some conclusions and present ideas for further research. This evaluation framework assesses a software system's quality by consolidating the viewpoints of producers, operators, users, managers, and stakeholders     

A framework for software process deployment and evaluation

ContextSoftware Process Engineering promotes the systematic production of software by following a set of well-defined technical and management processes. A comprehensive management of these processes involves the accomplishment of a number of activities such as model design, verification, validation, deployment and evaluation. However, the deployment and evaluation activities need more research efforts in order to achieve greater automation.ObjectiveWith the aim of minimizing the required time to adapt the tools at the beginning of each new project and reducing the complexity of the construction of mechanisms for automated evaluation, the Software Process Deployment & Evaluation Framework (SPDEF) has been elaborated and is described in this paper.MethodThe proposed framework is based on the application of well-known techniques in Software Engineering, such as Model Driven Engineering and Information Integration through Linked Open Data. It comprises a systematic method for the deployment and evaluation, a number of models and relationships between models, and some software tools.ResultsAutomated deployment of the OpenUP methodology is tested through the application of the SPDEF framework and support tools to enable the automated quality assessment of software development or maintenance projects.ConclusionsMaking use of the method and the software components developed in the context of the proposed framework, the alignment between the definition of the processes and the supporting tools is improved, while the existing complexity is reduced when it comes to automating the quality evaluation of software processes.     

ERP software implementation: an integrative framework

ERP implementation is a socio-technical challenge that requires a fundamentally different outlook from technologically-driven innovation, and will depend on a balanced perspective where the organisation as a total system is considered. ERP implementation is considered to rely on behavioural processes and actions. It is a process that involves macro-implementation at the strategic level, and micro-implementation at the operational level. This therefore means that implementation in the context of ERP systems is not possible through an ON/OFF approach whereby deployment of the new systems will necessarily yield the desired and expected results. Understanding the implementation process through a balanced perspective will therefore prevent any unpleasant surprises, and will ensure and guide the change process to be embedded in a painless fashion. The balanced perspective means that socio-technical considerations must be borne in mind; the strategic, tactical and operational steps clearly defined; and the expected benefits evaluated and tracked through creating seamless and solid integration. This paper proposes an integrative framework for ERP implementation based on an extensive review of the factors and the essential elements that contribute to success in the context of ERP implementation.     

An evaluation framework for software crowdsourcing

Recently software crowdsourcing has become an emerging area of software engineering. Few papers have presented a systematic analysis on the practices of software crowdsourcing. This paper first presents an evaluation framework to evaluate software crowdsourcing projects with respect to software quality, costs, diversity of solutions, and competition nature in crowdsourcing. Specifically, competitions are evaluated by the min-max relationship from game theory among participants where one party tries to minimize an objective function while the other party tries to maximize the same objective function. The paper then defines a game theory model to analyze the primary factors in these minmax competition rules that affect the nature of participation as well as the software quality. Finally, using the proposed evaluation framework, this paper illustrates two crowdsourcing processes, Harvard-TopCoder and AppStori. The framework demonstrates the sharp contrasts between both crowdsourcing processes as participants will have drastic behaviors in engaging these two projects.     

开源软件自动化评估证据框架

互联网上已形成了规模巨大、种类丰富的开源软件资源。如何准确、快速地判断一个开源项目的各种可信属性是否满足需求是当前软件工程领域研究的热点。深入分析已有开源软件评估模型,总结互联网上软件质量相关的各种信息,提出了面向开源软件的可信评估证据框架,并基于该框架构建了一种开源软件可信证据查询平台。利用该平台能够极大地提高评估效率,用户可以准确、快速、全面地了解相关软件项目的各种信息。最后,以一个知名开源软件证实了该证据框架及证据查询平台的可行性。     

A fuzzy-filtered neuro-fuzzy framework for software fault prediction for inter-version and inter-project evaluation

Fault Prediction is the most required measure to estimate the software quality and reliability. Several methods, measures, aspects and testing methodologies are available to evaluate the software fault. In this paper, a fuzzy-filtered neuro-fuzzy framework is introduced to predict the software faults for internal and external software projects. The suggested framework is split into three primary phases. At the earlier phase, the effective metrics or measures are identified, which can derive the accurate decision on prediction of software fault. In this phase, the composite analytical observation of each software attribute is calculated using Information Gain and Gain Ratio measures. In the second phase, these fuzzy rules are applied on these measures for selection of effective and high-impact features. In the last phase, the Neuro-fuzzy classifier is applied on fuzzy-filtered training and testing sets. The proposed framework is applied to identify the software faults based on inter-version and inter-project evaluation. In this framework, the earlier projects or project-versions are considered as training sets and the new projects or versions are taken as testing sets. The experimentation is conducted on nine open source projects taken from PROMISE repository as well as on PDE and JDT projects. The approximation is applied on internal version-specific fault prediction and external software projects evaluation. The comparative analysis is performed against Decision Tree, Random Tree, Random Forest, Naive Bayes and Multilevel Perceptron classifiers. This prediction result signifies that the proposed framework has gained the higher accuracy, lesser error rate and significant AUC and GM for inter-project and inter-version evaluations.     

A framework for evaluating software technology

Many software development organizations struggle to make informed decisions when investing in new software technologies. The authors' experimental framework can help companies evaluate a new software technology by examining its features in relation to its peers and competitors through a systematic approach that includes modeling experiments     

A software framework for data analysis

The open-source Java software framework JStatCom is presented which supports the development of rich desktop clients for data analysis in a rather general way. The concept is to solve all recurring tasks with the help of reusable components and to enable rapid application development by adopting a standards based approach which is readily supported by existing programming tools. Furthermore, JStatCom allows to call external procedures from within Java that are written in other languages, for example Gauss, Ox or Matlab. This way it is possible to reuse an already existing code base for numerical routines written in domain-specific programming languages and to link them with the Java world. A reference application for JStatCom is the econometric software package JMulTi, which will shortly be introduced.     

A requirements engineering framework for cross-organizational ERP systems

The development of cross-organizational enterprise resource planning (ERP) solutions is becoming increasingly critical to the business strategy of many networked companies. The major function of cross-organizational ERP solutions is to coordinate work in two or more organizations. However, how to align ERP application components and business requirements for coordination and cooperation is hardly known. This paper reports on the outcomes of applying a coordination theory perspective to an analysis of the ERP misalignment problem. We present a conceptual framework for analyzing coordination and cooperation requirements in inter-organizational ERP projects. The framework makes explicit the undocumented built-in assumptions for coordination and cooperation that may have significant implications for the ERP adopters and incorporates a library of existing coordination mechanisms supported by modern ERP systems. We use it to develop a proposal for how to achieve a better alignment between ERP implementations and supported business coordination processes in inter-organizational settings. We report on some early assessments of the implications of our framework for practicing requirements engineers. Both our framework and library rest on a literature survey and the first author’s experience with ERP implementation. In future empirical research, we will further validate and refine our framework.     

A framework for secure execution of software

The protection of software applications is one of the most important problems to solve in information security because it has a crucial effect on other security issues. We can find in the literature many research initiatives that have tried to solve this problem, many of them based on the use of tamperproof hardware tokens. This type of solution depends on two basic premises: (i) increasing the physical security by using tamperproof devices and (ii) increasing the complexity of the analysis of the software. The first premise is reasonable. The second one is certainly related to the first one. In fact, its main goal is that the pirate user not be able to modify the software to bypass an operation that is crucial: checking the presence of the token. However, experience shows that the second premise is not realistic because analysis of the executable code is always possible. Moreover, the techniques used to obstruct the analysis process are not enough to discourage an attacker with average resources.In this paper, we review the most relevant works related to software protection, present a taxonomy of those works, and, most important, introduce a new and robust software protection scheme. This solution, called SmartProt, is based on the use of smart cards and cryptographic techniques, and its security relies only on the first of the premises given above; that is, SmartProt has been designed to avoid attacks based on code analysis and software modification. The entire system is described following a lifecycle approach, explaining in detail the card setup, production, authorization, and execution phases. We also present some interesting applications of SmartProt as well as the protocols developed to manage licences. Finally, we provide an analysis of its implementation details.     

A testing framework for mobile computing software

We present a framework for testing applications for mobile computing devices. When a device is moved into and attached to a new network, the proper functioning of applications running on the device often depends on the resources and services provided locally in the current network. This framework provides an application-level emulator for mobile computing devices to solve this problem. Since the emulator is constructed as a mobile agent, it can carry applications across networks on behalf of its target device and allow the applications to connect to local servers in its current network in the same way as if they had been moved with and executed on the device itself. This paper also demonstrates the utility of this framework by describing the development of typical network-dependent applications in mobile and ubiquitous computing settings.     

A framework for hardware/software codesign

It is argued that a hardware/software codesign methodology should support the following capabilities: integration of the hardware and software design processes; exploration of hardware/software tradeoffs and evaluation of hardware/software alternatives; and model continuity. A codesign methodology that supports many of these capabilities is outlined. The methodology is iterative in nature and serves to guide codesign exploration with the uninterpreted/interpreted modeling approach. It integrates performance (uninterpreted) models and functional (interpreted) models in a common simulation environment     

A framework for efficient FE vibration software

A new hybrid stress finite plate vibration capability, providing high accuracy for coarse-meshes is presented with a view to enhancing the behavioural characteristics of the standard hybrid FEM. The software meets demands of the real-life user for reliable and cost-objective identification of a wide range of vibration modes. The FE matrices are constructed through the expedient of introducing a system of algorithms which provides an efficient and easily implemented capability that can be translated into any of the existing high level computing languages, viz. FORTRAN.The computational scheme enables the development of a large number of increasingly sophisticated elements from a single element module as easily as possible by providing it with a library of datasets. All the previously recognised advantages of the hybrid FEM are retained, whilst an exact analytical integrator returns the requisite information for the elemental matrices, and thereby obviates an aliasing problem that has plagued the cost-objectiveness of the conventional hybrid stress implementations. Extensive numerical tests manifest the numerical potentials of the present hybrid FE computational strategy.     

A formal framework for software product lines

ContextA Software Product Line is a set of software systems that are built from a common set of features. These systems are developed in a prescribed way and they can be adapted to fit the needs of customers. Feature models specify the properties of the systems that are meaningful to customers. A semantics that models the feature level has the potential to support the automatic analysis of entire software product lines.ObjectiveThe objective of this paper is to define a formal framework for Software Product Lines. This framework needs to be general enough to provide a formal semantics for existing frameworks like FODA (Feature Oriented Domain Analysis), but also to be easily adaptable to new problems.MethodWe define an algebraic language, called SPLA, to describe Software Product Lines. We provide the semantics for the algebra in three different ways. The approach followed to give the semantics is inspired by the semantics of process algebras. First we define an operational semantics, next a denotational semantics, and finally an axiomatic semantics. We also have defined a representation of the algebra into propositional logic.ResultsWe prove that the three semantics are equivalent. We also show how FODA diagrams can be automatically translated into SPLA. Furthermore, we have developed our tool, called AT, that implements the formal framework presented in this paper. This tool uses a SAT-solver to check the satisfiability of an SPL.ConclusionThis paper defines a general formal framework for software product lines. We have defined three different semantics that are equivalent; this means that depending on the context we can choose the most convenient approach: operational, denotational or axiomatic. The framework is flexible enough because it is closely related to process algebras. Process algebras are a well-known paradigm for which many extensions have been defined.     

A knowledge-based framework for automated software synthesis

Automated software synthesis is one of the central techniques used in knowledge-based software engineering to enhance the quality and efficiency of software development. Although many software synthesis systems have been developed, automatic control of these systems remains a difficult problem. Our goal is to reduce user interaction in transformational and schema-based synthesizers by means of significant advances in control mechanisms.This paper describes an approach for synthesis control that integrates a blackboard control architecture with an existing synthesis system. We present a framework language called MetaMorphos that allows explicit representations of control knowledge for use in selecting appropriate synthesis actions. MetaMorphos represents control decisions explicitly in terms of actions, events, and states. It is task-specific and contains knowledge about programming and how to select synthesizing methods based on given features. By employing a blackboard control architecture, our synthesis controller provides adaptability for dynamic control behaviors and flexibility to handle unanticipated situations during software development.Applying MetaMorphos in the domain of software synthesis, we illustrate how we use MetaMorphos to select appropriate transformations, data structure and algorithm schemas during the synthesis. An example shows how MetaMorphos handles the difficult problem of selecting schemas for two very similar problems which, in the best case, require different solutions.     

A hierarchical framework for evaluating simulation software

In simulation software selection problems, packages are evaluated either on their own merits or in comparison with other packages. In either method, a comprehensive list of criteria for evaluation of simulation software is essential for proper selection. Although various simulation software evaluation checklists do exist, there are differences in the lists provided and the terminologies used. This paper presents a hierarchical framework for simulation software evaluation consisting of seven main groups and several subgroups. An explanation for each criterion is provided and an analysis of the usability of the proposed framework is further discussed.     

A cataloging framework for software development methods

A framework providing a basis for comparing and evaluating software development methods (SDMs), which are systems of technical procedures and notational conventions for the organized construction of software-based systems, is presented. Using the framework, practitioners and methodologists can describe and rate an SDM's support for 21 properties. The application of the framework to two examples, OMT and R.J.A. Buhr's (1990) architectural design, is discussed. Several suggested uses of the framework include: comparing a group of SDMs with one another; defining a standard in terms of the framework and then using this as a basis for discussion; examining an SDM to discover its coverage and capabilities; and combining the properties of different SDMs to create a new SDM     

A measurement framework for object-oriented software testability

Testing is an expensive activity in the development process of any software system. Measuring and assessing the testability of software would help in planning testing activities and allocating required resources. More importantly, measuring software testability early in the development process, during analysis or design stages, can yield the highest payoff as design refactoring can be used to improve testability before the implementation starts.

This paper presents a generic and extensible measurement framework for object-oriented software testability, which is based on a theory expressed as a set of operational hypotheses. We identify design attributes that have an impact on testability directly or indirectly, by having an impact on testing activities and sub-activities. We also describe the cause-effect relationships between these attributes and software testability based on thorough review of the literature and our own testing experience. Following the scientific method, we express them as operational hypotheses to be further tested. For each attribute, we provide a set of possible measures whose applicability largely depends on the level of details of the design documents and the testing techniques to be applied. The goal of this framework is twofold: (1) to provide structured guidance for practitioners trying to measure design testability, (2) to provide a theoretical framework for facilitating empirical research on testability.     

A framework for model-based design of agent-oriented software

Agents are becoming one of the most important topics in distributed and autonomous decentralized systems, and there are increasing attempts to use agent technologies to develop large-scale commercial and industrial software systems. The complexity of such systems suggests a pressing need for system modeling techniques to support reliable, maintainable, and extensible design. G-nets are a type of Petri net defined to support system modeling in terms of a set of independent and loosely-coupled modules. In this paper, we customize the basic G-net model to define a so-called "agent-based G-net" that can serve as a generic model for agent design. Then, to progress from an agent-based design model to an agent-oriented model, new mechanisms to support inheritance modeling are introduced. To illustrate our formal modeling technique for multiagent systems, an example of an agent family in electronic commerce is provided. Finally, we demonstrate how we can use model checking to verify some key behavioral properties of our agent model. This is facilitated by the use of an existing Petri net tool.