Evaluating and selecting software packages: A review

Evaluating and selecting software packages that meet an organization’s requirements is a difficult software engineering process. Selection of a wrong software package can turn out to be costly and adversely affect business processes. The aim of this paper is to provide a basis to improve the process of evaluation and selection of the software packages. This paper reports a systematic review of papers published in journals and conference proceedings. The review investigates methodologies for selecting software packages, software evaluation techniques, software evaluation criteria, and systems that support decision makers in evaluating software packages. The key findings of the review are: (1) analytic hierarchy process has been widely used for evaluation of the software packages, (2) there is lack of a common list of generic software evaluation criteria and its meaning, and (3) there is need to develop a framework comprising of software selection methodology, evaluation technique, evaluation criteria, and system to assist decision makers in software selection.     

Package Coupling Measurement in Object-Oriented Software

The grouping of correlated classes into a package helps in better organization of modern object-oriented software. The quality of such packages needs to be measured so as to estimate their utilization. In this paper, new package coupling metrics are proposed, which also take into consideration the hierarchical structure of packages and direction of connections among package elements. The proposed measures have been validated theoretically as well as empirically using 18 packages taken from two open source software systems. The results obtained from this study show strong correlation between package coupling and understandability of the package which suggests that proposed metrics could be further used to represent other external software quality factors.     

Framework for evaluation and selection of the software packages: A hybrid knowledge based system approach

Evaluation and selection of the software packages is complicated and time consuming decision making process. Selection of inappropriate software package can turn out to be costly and adversely affects business processes and functioning of the organization. In this paper we describe (i) generic methodology for software selection, (ii) software evaluation criteria, and (iii) hybrid knowledge based system (HKBS) approach to assist decision makers in evaluation and selection of the software packages. The proposed HKBS approach employs an integrated rule based and case based reasoning techniques. Rule based reasoning is used to capture user needs of the software package and formulate a problem case. Case based reasoning is used to retrieve and compare candidate software packages with the user needs of the package. This paper also evaluates and compares HKBS approach with the widely used existing software evaluation techniques such as analytic hierarchy process (AHP) and weighted scoring method (WSM).     

Recommending packages with validity constraints to groups of users

The success of recommender systems has made them the focus of a massive research effort in both industry and academia. Recent work has generalized recommendations to suggest packages of items to single users, or single items to groups of users. However, to the best of our knowledge, the interesting problem of recommending a package to a group of users (P2G) has not been studied to date. This is a problem with several practical applications, such as recommending vacation packages to tourist groups, entertainment packages to groups of friends or sets of courses to groups of students. In this paper, we formulate the P2G problem, and we propose probabilistic models that capture the preference of a group toward a package, incorporating factors such as user impact and package viability. We also investigate the issue of recommendation fairness. This is a novel consideration that arises in our setting, where we require that no user is consistently slighted by the item selection in the package. In addition, we study a special case of the P2G problem, where the recommended items are places and the recommendation should consider the current locations of the users in the group. We present aggregation algorithms for finding the best packages and compare our suggested models with baseline approaches stemming from previous work. The results show that our models find packages of high quality which consider all special requirements of P2G recommendation.     

An empirical investigation on the reusability of design patterns and software packages

Nowadays open-source software communities are thriving. Successful open-source projects are competitive and the amount of source code that is freely available offers great reuse opportunities to software developers. Thus, it is expected that several requirements can be implemented based on open source software reuse. Additionally, design patterns, i.e. well-known solution to common design problems, are introduced as elements of reuse. This study attempts to empirically investigate the reusability of design patterns, classes and software packages. Thus, the results can help developers to identify the most beneficial starting points for white box reuse, which is quite popular among open source communities. In order to achieve this goal we conducted a case study on one hundred (100) open source projects. More specifically, we identified 27,461 classes that participate in design patterns and compared the reusability of each of these classes with the reusability of the pattern and the package that this class belongs to. In more than 40% of the cases investigated, design pattern based class selection, offers the most reusable starting point for white-box reuse. However there are several cases when package based selection might be preferable. The results suggest that each pattern has different level of reusability.     

The Package Blueprint: Visually analyzing and quantifying packages dependencies

Large object-oriented applications are structured over many packages. Packages are important but complex structural entities that are difficult to understand since they act as containers of classes, which can have many dependencies with other classes spread over multiple packages. However to be able to take decisions (e.g. refactoring and/or assessment decisions), maintainers face the challenges of managing (sorting, grouping) the massive amount of dependencies between classes spread over multiple packages. To help maintainers, there is a need for at the same time understanding, and quantifying, dependencies between classes as well as understanding how packages as containers of such classes depend on each other.In this paper, we present a visualization, named Package Blueprint, that reveals in detail package internal structure, as well as the dependencies between an observed package and its neighbors, at both package and class levels. Package blueprint aims at assisting maintainers in understanding package structure and dependencies, in particular when they focus on few packages and want to take refactoring decisions and/or to assess the structure of those packages. A package blueprint is a space filling matrix-based visualization, using two placement strategies that are enclosure and adjacency. Package blueprint is structured around the notion of surfaces that group classes and their dependencies by their packages (i.e., enclosure placement); whilst surfaces are placed next to their parent node which is the package under-analysis (i.e., adjacency placement). We present two views: one stressing how an observed package depends upon the rest of the system and another stressing how the system depends upon that package.To evaluate the contribution of package blueprint for understanding packages we performed an exploratory user study comparing package blueprint with an advanced IDE. The results show that users of package blueprint are faster in analyzing and assessing package structure. The results are proved statically significant and they show that package blueprint considerably improves the experience of standard browser users.     

A heuristic for software evaluation and selection

The proliferation of software packages has created a difficult, complex problem of evaluation and selection for many users. Traditional approaches to the quantification of package performance have relied on compensatory models, such as the linear weighted attribute model, which sums the weighted ratings of software attributes. These approaches define the dimensions of quality too narrowly and, therefore, omit substantial amounts of information from consideration. This paper presents an alternative methodology, previously used in capital rationing and tournament ranking, that expands the opportunity for objective insight into software quality. In particular, it considers three measures of quality, the frequency with which the attribute ratings of one package exceed those of another, the presence of outliers, where very poor performance may exist on a single attribute and be glossed over by compensatory methods, and the cumulative magnitude of attribute ratings on one package that exceed those on others. The proposed methodology is applied to the evaluation of the following software types: word processing, database management systems, spreadsheet/financial planning, integrated software, graphics, data communications and project management.     

Success of software packages in small businesses: an exploratory study

In this paper we develop and use a framework for evaluating the success of microcomputer based integrated administrative software packages in small businesses. The results of an exploratory survey of 66 small business organizations show that small businesses have often been disappointed with their software packages. The disappointment is frequently a result of the inability of the package to adapt to the needs of the company, especially in small businesses of more than 50 employees. For small businesses of less than 20 employees the packages are too difficult to use. These findings indicate that small businesses should place more emphasis on the acquisition, especially the requirements specification, and developers should improve user-friendliness, the quality of support and documentation to fulfil the needs of the smallest of small businesses. The successful implementers—because of their experience—knew their needs, carefully acquired the package, and implemented various parts of it, benefiting from integration. There seem to be no shortcuts to success, but a determined and eager attitude of the personnel can produce success with any of the software packages in our study.     

AIDA's fourth-generation software functionality

Aspects characterizing typical, office-like environments are described and common procedures are extracted. These are used to derive the requirements of fourth-generation software packages. Typical fourth-generation software packages are discussed, and a presentation is given of a fourth-generation software package called AIDA, including all its functional aspects.     

A computational environment for the management, processing, and analysis of geological data

The complexity and diversity of data and processing now being used in geoscientific data analysis are increasing at such a pace that it is difficult to maintain a single software package capable of performing all required tasks. A typical solution is to use a number of commercial software packages loosely linked via file format conversion routines or other customized communication paths. Creating and maintaining these paths is made difficult by both the number of distinct packages being used and the evolutionary changes taking place within each package. In this paper we attempt to create a generic model for a geological data processing task, identify some of the difficulties involved in performing these tasks using current technology, and outline a computational architecture that may correct many of the deficiencies of existing technology. Our goal is not to provide a detailed design but to present some key issues and possible solutions as a foundation for discussion and investigation.     

Statgraphics : Neil W. Polhemus and the Statistical Graphics Corporation. Rockville, MD: STSC. Release No. 2.0, $795.00

Several statistical software packages are now commercially available for microcomputers. The quality and usability of these software packages vary dramatically. The purpose of this paper is to review the package STATGRAPHICS utilizing the statistical software model developed by Ansorge, Wise, and Plake (1985). The paper will address quality and usability dimensions of STATGRAPHICS from the vantage point and needs of an educational researcher.     

openModeller: a generic approach to species’ potential distribution modelling

Species’ potential distribution modelling is the process of building a representation of the fundamental ecological requirements for a species and extrapolating these requirements into a geographical region. The importance of being able to predict the distribution of species is currently highlighted by issues like global climate change, public health problems caused by disease vectors, anthropogenic impacts that can lead to massive species extinction, among other challenges. There are several computational approaches that can be used to generate potential distribution models, each achieving optimal results under different conditions. However, the existing software packages available for this purpose typically implement a single algorithm, and each software package presents a new learning curve to the user. Whenever new software is developed for species’ potential distribution modelling, significant duplication of effort results because many feature requirements are shared between the different packages. Additionally, data preparation and comparison between algorithms becomes difficult when using separate software applications, since each application has different data input and output capabilities. This paper describes a generic approach for building a single computing framework capable of handling different data formats and multiple algorithms that can be used in potential distribution modelling. The ideas described in this paper have been implemented in a free and open source software package called openModeller. The main concepts of species’ potential distribution modelling are also explained and an example use case illustrates potential distribution maps generated by the framework.     

一种基于Internet的JAR包使用信息收集方法

随着软件复用技术的发展,Internet上出现了大量可以被利用的软件资源,如Web Services, JAR包等。但是,这些软件资源常常缺乏必要的描述信息和使用说明。为此,提出了一种基于Internet的JAR包使用信息收集方法,以帮助用户检索并整理Internet上已经存在的JAR包描述信息和用户使用评论,辅助软件复用的成功进行。基于该方法,设计并实现了北京大学软件资源库JAR包使用信息收集子系统。该系统目前已经为6000余个JAR包资源收集、整理了相关的描述和评论信息。     

An approach to expert control of interactive software systems

Expert problem-solving strategies in many domains require the use of detailed mathematical techniques coupled with experiential knowledge about how and when to use the appropriate techniques. In many of these domains, such techniques are made available to experts in large software packages. In attempting to build expert systems for these domains, we wish to make use of these packages, and are therefore faced with an important problem: how to integrate the existing software, and knowledge about its use, into a practical expert system. The expert knowledge is used, in dynamic selection and interpretation of appropriate programs and parameters, to reach a successful goal in the problem solving. We describe the framework of a hybrid expert system for representing problem-solving knowledge in these domains. This hybrid system may be characterized as consisting of a production system and mathematical methods. The software package is reorganized as necessary to map it into the mathematical-method representation of a hybrid system. This approach has evolved out of an effort to build an expert system for performing well-log analysis, ELAS (expert log analysis system).     

The integration of an intelligent knowledge-based system into engineering software using the blackboard structure

Over recent years there has been an increase in the use of generic Computational Fluid Dynamics (CFD) software packages spread across various application fields. This has created the need for the integration of expertise into CFD software. Expertise can be integrated into CFD software in the form of an Intelligent Knowledge-Based System (IKBS). The advantages of integrating intelligence into generic engineering software are discussed with a special view to software engineering considerations. The software modelling cycle of a typical engineering problem is identified and the respective expertise and user control needed for each modelling phase is shown. The requirements of an IKBS for CFD software are discussed and compared to current practice. The blackboard software architecture is presented. This is shown to be appropriate for the integration of an IKBS into an engineering software package. This is demonstrated through the presentation of the prototype CFD software package FLOWES.     

Evaluation of statistical packages for suitability for use by clinical investigators in medicine

With the increased availability of personal computers and statistical software packages, it is inevitable that there will be increasing attempts by clinical investigators to perform data management and statistical analysis. Reviews of statistical packages are abundant in computer and statistical journals. However the majority of them were not written for clinical investigators in medicine. This paper presents an analytic approach to evaluate the suitability of statistical packages for use by clinical investigators for data-management and preliminary statistical-analysis purposes. The evaluation scheme addresses five areas of concern: availability of data-management features; availability of basic statistical-analysis features; ease of use; documentation; and quality of programs. Among six statistical packages reviewed by this process, CRISP is recommended as the most suitable package for clinical investigators to use for data-management and preliminary statistical-analysis purposes.     

ProcessTalk A natural language interface for industrial applications

Many commercial software packages are available for industrial applications. However, small and medium size enterprises, SME, normally lack the in-house qualified personnel to benefit from such packages. As natural languages are the best means of communication, a natural language interface can therefore be a good means to grant the SME an easy access to the sophisticated computerized software packages, thus alleviating the problem of the lack of qualified personnel and reducing the cost of employees' learning curve. At present, no such interfaces exist.

This paper presents the design of a human-machine interface, ProcessTalk, that would accept natural language commands in a text form, then would invoke and execute the appropriate functional/procedural commands required by the desired commercial software package the same way as qualified analysts and engineers would operate. It also exposes its benefits and some of its industrial applications.     

On recommendation problems beyond points of interest

Recommendation systems aim to recommend items or packages of items that are likely to be of interest to users. Previous work on recommendation systems has mostly focused on recommending points of interest (POI), to identify and suggest top-k items or packages that meet selection criteria and satisfy compatibility constraints on items in a package, where the (packages of) items are ranked by their usefulness to the users. As opposed to prior work, this paper investigates two issues beyond POI recommendation that are also important to recommendation systems. When there exist no sufficiently many POI that can be recommended, we propose (1) query relaxation recommendation to help users revise their selection criteria, or (2) adjustment recommendation to guide recommendation systems to modify their item collections, such that the users׳ requirements can be satisfied.We study two related problems, to decide (1) whether the query expressing the selection criteria can be relaxed to a limited extent, and (2) whether we can update a bounded number of items, such that the users can get desired recommendations. We establish the upper and lower bounds of these problems, all matching, for both combined and data complexity, when selection criteria and compatibility constraints are expressed in a variety of query languages, for both item recommendation and package recommendation. To understand where the complexity comes from, we also study the impact of variable sizes of packages, compatibility constraints and selection criteria on the analyses of these problems. Our results indicate that in most cases the complexity bounds of query relaxation and adjustment recommendation are comparable to their counterparts of the basic recommendation problem for testing whether a given set of (resp. packages of) items makes top-k items (resp. packages). In other words, extending recommendation systems with the query relaxation and adjustment recommendation functionalities typically does not incur extra overhead.     

DISPRO: A discrete programming package

Conclusions The DISPRO package is open for modification and further development. Its modular structure allows expansion of the source language, incorporation of new system and application modules, realization in the current version of an interface with DBMS and other application packages (e.g., report generator, matrix generator), synthesis of the package with PL/1 and FORTRAN IV programming systems.The first version of DISPRO released in 1980 has successfully passed the requisite acceptance tests and is now officially part of the optimization packages supported by ES series computers. As outlined in this article, the package has tremendous possibilities for solving a wide range of discrete optimization problems. So far, certain classes of discrete programming problems could be solved by LP ASU [31], VEKTOR-1, the Z-system [58], and TsP [31] application packages. In addition, some packages have been developed in recent years for mathematical programming and operations research (DISPLAN, an interactive planning system [60]; PTP, optimization of transportation problems [58]; PMP, separable and parametric linear programming; DILOS, interactive system for unconstrained optimization, nonlinear programming, and optimal control; ORBITAL', unconstrained optimization and approximation [61]; NDO, nondifferentiable and stochastic optimization [62]; and other less known packages reviewed, in particular, in [58].New optimization packages are currently being developed in response to increased demands for quality, efficiency, and applicability in various MIS and ADPS. The next version of the DISPRO package is being designed on the basis of the appropriate principles and requirements for modern method-oriented packages, including accumulation and analysis of package performance data (these data may be used to improve the package software and structure), large dimensionality of the relevant problems, operating reliability, commercial development of packages, and their usability in integrated data processing systems.In conclusion, DISPRO was developed by the concerted effort of a large team. In particular, we should mention the contributions of Yu. M. Ermol'ev, V. L. Volkovich, A. I. Kuksa, Yu. Yu. Chervak, N. S. Podgorodetskaya, E. S. Karpinka, V. M. Sementsova, V. I. Artemenko, and other members of the team.Translated from Kibernetika, No. 3, pp. 117–137, May–June, 1981.     

Symbolic modeling in robotics: Genesis,applicaton, and future prospects

This survey article gives an overview of software packages for generating numerically efficient manipulator models in symbolic form, i.e. as computer programs written in a high-level language such as C or FORTRAN. We chronicle the history of computational robot dynamics and, to some extent, multibody systems dynamics. We survey several mechanical computer-aided engineering software packages because they have charted the course for symbolic robot modeling software. The attractive features of various programs regarding areas of application (vehicles, robots, satellites, etc.) and design possibilities (kinematic and dynamic analysis, modal analysis, optimization of mechanical design, numerical efficiency of generated symbolic models, etc.) are emphasized. Finally, as an example, we present the SYM program package in more detail and point out the new strategic area of robotics which has emerged during the last two years: computer-aided generation of control laws.