Robust data envelopment analysis approaches for evaluating algorithmic performance

Recent advances in state-of-the-art meta-heuristics feature the incorporation of probabilistic operators aiming to diversify search directions or to escape from being trapped in local optima. This feature would result in non-deterministic output in solutions that vary from one run to another of a meta-heuristic. Consequently, both the average and variation of outputs over multiple runs have to be considered in evaluating performances of different configurations of a meta-heuristic or distinct meta-heuristics. To this end, this work considers each algorithm as a decision-making unit (DMU) and develops robust data envelopment analysis (DEA) models taking into account not only average but also standard deviation of an algorithm’s output for evaluating relative efficiencies of a set of algorithms. The robust DEA models describe uncertain output using an uncertainty set, and aim to maximize a DMU’s worst-case relative efficiency with respect to that uncertainty set. The proposed models are employed to evaluate a set of distinct configurations of a genetic algorithm and a set of parameter settings of a simulated annealing heuristic. Evaluation results demonstrate that the robust DEA models are able to identify efficient algorithmic configurations. The proposed models contribute not only to the evaluation of meta-heuristics but also to the DEA methodology.     

Data envelopment analysis for a supply chain

Data envelopment analysis (DEA) is a method for evaluating the management efficiency of decision-making units (DMUs). This article proposes a DEA model for supply-chain management. Traditional studies focused on the selection of partners and the construction of the supply chain. Therefore, this study considers how to optimize the supply chain itself in order to maximize the benefit by DEA. In addition, a significant matter is that supply chains have sometimes unbalanced business processes. This means that some particular DMUs on the supply chain have a superiority which maintains efficiency. That is why the other DMUs on the supply chain need to operate in unfavorable conditions. As a result, their operations badly affect the total efficiency of the supply chain. Therefore, the proposed method introduces an adjustment variable to calculate the optimum operation of the supply chain. The utility and effectiveness of the proposed method are shown by numerical experiments.     

Data envelopment analysis classification machine

This paper establishes the equivalent relationship between the data classification machine and the data envelopment analysis (DEA) model, and thus build up a DEA based classification machine. A data is characterized by a set of values. Without loss of the generality, it is assumed that the data with a set of smaller values is preferred. The classification is to label if a particular data belongs to a specified group according to a set of predetermined characteristic or attribute values. We treat such a data as a decision making unit (DMU) with these given attribute values as input and an artificial output of identical value 1. Then classifying a data is equivalent to testing if the DMU is in the production possibility set, called acceptance domain, constructed by a sample training data set. The proposed DEA classification machine consists of an acceptance domain and a classification function. The acceptance domain is given by an explicit system of linear inequalities. This makes the classification process computationally convenient. We then discuss the preference cone restricted classification process. The method can be applied to classifying large amount of data. Furthermore, the research finds that DEA classification machines based on different DEA models have the same format. Input-oriented and output-oriented DEA classification machines have similar properties. The method developed has great potential in practice with its computational efficiency.     

Improved data envelopment analysis models for evaluating interval efficiencies of decision-making units

In a recent article, Wang et al. [Wang, N. S., Yi, R. H., & Wang, W. (2008). Evaluating the performances of decision-making units based on interval efficiencies. Journal of Computational and Applied Mathematics, 216, 328–343] proposed a pair of interval data envelopment analysis (DEA) models for measuring the overall performances of decision-making units (DMUs) with crisp data. In this paper, we demonstrate that interval DEA models face problems in determining the efficiency interval for each DMU when there are zero values for every input. To remedy this drawback, we propose a pair of improved interval DEA models which make it possible to perform a DEA analysis using the concepts of the best and the worst relative efficiencies. Two numerical examples will be examined using the improved interval DEA models. One of the examples is a real-world application about 42 educational departments in one of the branches of the Islamic Azad University in Iran that shows the advantages and applicability of the improved approach in real-life situations.     

Data envelopment analysis of reservoir system performance

In long-term performance analyses of water systems with surface reservoirs for different operating scenarios, the analyst (or decision maker) is faced with two connected problems: (1) how to handle the extensive output of the simulation model and derive information on the scenarios scores for a prescribed set of performance criteria, and (2) how to compare scenarios in a multi-criterial sense while identifying the most desired. The data sets may overburden the analyst, while an evaluating procedure may be subjective due to personal preferences, attitudes, knowledge and miscellaneous factors. The data envelopment analysis (DEA) approach proposed here seems to be reliable in treating these situations, and sufficiently objective in evaluating and ranking the scenarios. Certain performance indices are defined as evaluating criteria in a standard multi-criterial sense, and then virtually divided into scenarios' output and input measures. By considering scenarios as product units, the DEA optimizes the weights of inputs and outputs, computes productivity efficiency for each unit, and rank them appropriately. Omitting the analyst's personal judgment on the technical parameters that describe system's performance restricts, in this way, the influence of the decision maker. A case study application on the reservoir system in Brazil proved that a methodological connection for solving decision problems with discrete alternatives really exists between the DEA and standard multi-criteria methods.     

Data envelopment analysis based on team reasoning

Existing approaches to data envelopment analysis focus mainly on the derivation of the efficiency of the individual decision‐making unit (DMU) or on the calculation of the weights of multiple inputs or outputs, but pay little attention to the team interest of all the DMUs. Motivated by the idea of team reasoning, in which the benefit of the team is of higher importance than that of each individual, this paper considers all the DMUs as a team and introduces the team indexes including the overall efficiencies, variance, boundaries of all the DMUs, and relationships between DMUs. Several models are first developed to estimate values of the team indexes based on which decision makers can provide their preferences regarding them. Then, models are established to obtain the interval efficiencies of individual DMUs under the condition that the team indexes are satisfied. Several examples are given to illustrate the proposed approaches and verify their applicability.     

Data envelopment analysis for efficiency measurement in the stochastic case

The data envelopment analysis for measuring efficiency of decision-making units is generalized here for stochastic variations of input and output data. The case of one output and many inputs is considered for three types of data variations, e.g. in the objective function, in the constraints and the outputs. It is shown that a minimax method of efficiency measurement through chance-constrained programming methods can be suitably applied for the case of chance constraints. Some empirical applications to measurement of efficiency of public schools are also analyzed to test the sensitivity and robustness of the efficiency ranking and measurement.     

Data envelopment analysis for evaluating the efficiency of genetic algorithms on solving the vehicle routing problem with soft time windows

This study proposes an alternative to the conventional empirical analysis approach for evaluating the relative efficiency of distinct combinations of algorithmic operators and/or parameter values of genetic algorithms (GAs) on solving the pickup and delivery vehicle routing problem with soft time windows (PDVRPSTW). Our approach considers each combination as a decision-making unit (DMU) and adopts data envelopment analysis (DEA) to determine the relative and cross efficiencies of each combination of GA operators and parameter values on solving the PDVRPSTW. To demonstrate the applicability and advantage of this approach, we implemented a number of combinations of GA’s three main algorithmic operators, namely selection, crossover and mutation, and employed DEA to evaluate and rank the relative efficiencies of these combinations. The numerical results show that DEA is well suited for determining the efficient combinations of GA operators. Among the combinations under consideration, the combinations using tournament selection and simple crossover are generally more efficient. The proposed approach can be adopted to evaluate the relative efficiency of other meta-heuristics, so it also contributes to the algorithm development and evaluation for solving combinatorial optimization problems from the operational research perspective.     

Data envelopment analysis with neutrosophic inputs and outputs

Since the recent appearance of neutrosophic theory as a generalization of fuzzy and intuitionistic fuzzy theories, many multicriteria decision methods have adopted this theory to deal with incomplete and indeterminate data. However, it has not yet been applied to the data envelopment analysis (DEA) methodology. Therefore, this study presents a DEA model with triangular neutrosophic inputs and outputs that considers the truth, indeterminacy, and falsity degrees of each data value. As an alternative, a parametric approach based on what we term the variation degree of a triangular neutrosophic number is developed. This approach transforms a neutrosophic DEA model into an interval DEA model that can be solved using one of many existing techniques. Interval efficiency scores obtained from our numerical example show the flexibility and authenticity of the proposed approach.     

Data envelopment analysis of integer-valued inputs and outputs

This paper addresses DEA scenarios whose inputs and outputs are naturally restricted to take integer values. Conventional DEA models would project the DMU onto targets that generally do not respect such type of integrality constraints. Although integer-valued inputs and outputs can be considered as a special case of ordinal inputs and outputs, the use of that type of models has many drawbacks. In this paper a MILP DEA model that guarantees the required integrality of the computed targets is proposed.     

Data envelopment analysis: a new tool for improving managerial efficiency

Two new applications of the recently developed tool of ‘data envelopment analysis’ are focused on here. One develops the concept of systematic efficiency, and the other the concept of dynamic efficiency. An empirical application shows the tremendous usefulness of this new managerial technique for measuring and improving industrial productivity     

Data envelopment analysis and robust optimization: A review

This paper reviews the milestone approaches for handling uncertainty in data envelopment analysis (DEA). This paper presents the detailed classifications of robust data envelopment analysis (RDEA). RDEA is appropriate for measuring the efficiencies of decision-making units in the presence of the data and distributional uncertainties. This paper reviews scenario-based and uncertainty set of DEA models. It covers 73 studies from 2008 to 2019. The paper concludes with suggestions about the guidelines for future researches in the field of RDEA.     

Data envelopment analysis based on maximin relative efficiency criterion

While gauging the performances of operating entities using imprecise information on the input and output importance weights, an entity is considered Farrell efficient as long as it outperforms its peers for at least one feasible combination of the weights for inputs and outputs. This paper argues that Farrell efficiency computations are based on an optimistic perspective and a Farrell-efficient entity may perform rather poorly when weights corresponding to realistic considerations are assigned to inputs and outputs. An entity is defined as robust efficient if its relative efficiency score reaches 1 in all feasible combinations of the input and output weights. A linear programming based approach is proposed to perform what is referred to as robust efficiency analysis to identify robust efficient entities. In contrast to Farrell efficiency analysis, robust efficiency analysis involves the computation of the lowest efficiency score that can be assigned to an entity relative to the highest score among all the entities where an identical combination of weights for inputs and outputs is applied. The production possibility set underlying the proposed approach is also defined and interpreted. An experimental study illustrates that when compared with Farrell efficiency analysis robust efficiency analysis has sharper discrimination capability and the entity it identifies as efficient has superior average performance.     

Developing a novel inverse data envelopment analysis (DEA) model for evaluating after-sales units

This paper proposes a novel model of inverse data envelopment analysis (IDEA) based on the slack-based measure (SBM) approach. The developed inverse SBM model can maintain relative efficiency of decision making units (DMUs) with new input and output. This model can also measure the input and output volumes when a decision maker (DM) increases efficiency score. The inverse SBM model is a kind of multi-objective non-linear programming (MONLP) problem, which is not easy to solve. Therefore, we suggest a linear programming model for solving inverse SBM model. In this model efficiency score of DMU under evaluation remains unchanged. Furthermore, we suggest an optimal combination of inputs and outputs in the production possibility set (PPS). A case study is presented to demonstrate the efficacy of our proposed model.     

Models of data envelopment analysis and stochastic frontier analysis in the efficiency assessment of universities

This paper systematizes the empirical results on efficiency concepts applied to higher education institutions, data envelopment analysis (DEA) adjusted to heterogeneous samples, inputs and outputs chosen for these institutions and factors tended to make universities efficient. Special attention is paid to the consistency of results yielded by different models.     

Data envelopment analysis for weight derivation and aggregation in the analytic hierarchy process

Data envelopment analysis (DEA) is proposed in this paper to generate local weights of alternatives from pair-wise comparison judgment matrices used in the analytic hierarchy process (AHP). The underlying assumption behind the approach is explained, and some salient features are explored. It is proved that DEA correctly estimates the true weights when applied to a consistent matrix formed using a known set of weights. DEA is further proposed to aggregate the local weights of alternatives in terms of different criteria to compute final weights. It is proved further that the proposed approach, called DEAHP in this paper, does not suffer from rank reversal when an irrelevant alternative(s) is added or removed.     

Data envelopment with maximum correlation

Maximizing the correlation between inputs and outputs in data envelopment analysis is analysed in three respects: (a) the minimization of the Lp-norm; (h) the standard regression approach; and (c) the case of composite outputs. The implications of canonical correlation are explored and their applications to efficiency studies of public sector enterprises discussed. This shows that the correlation measure may involve in some sense a basic and fundamental extension of the efficiency model analysed by data envelopment analysis.     

Data envelopment analysis in resource allocation: An application to forest management

This paper discusses the resource allocation problem for not-for-profit organizations that have control over several production units of similar functions. A case of budget allocation among the subdistricts of a forest district in Taiwan is exemplified to illustrate the idea. The model proposed is a nonlinear fractional program superimposed upon the data envelopment analysis framework. This nonlinear fractional program can be transformed to a model similar to the generalized linear program and solved by a type of decomposition method. Within prespecified ranges, the district office searches for ways of allocating a fixed amount of budget to its subdistricts to result in a higher aggregate efficiency score. Wider ranges allow for more flexibility in allocating budget; consequently, higher aggregate efficiency scores are experienced. Since this method is more objective, it is more persuasive to the subdistricts in allocating resources.     

Computer science in Japanese universities

It is argued that despite associating Japan with high technology, most Western scientists know little about computer science in Japan. Many factors contribute to this phenomenon, including language and cultural differences, a shortage of readily available information, and a degree of technical chauvinism. On the basis of observations made during sabbaticals in Japan, the authors provide an informal portrait of computer science in Japanese universities in the hope that enhanced awareness and increased interaction will result. They look specifically at departmental structure, faculty career paths, the student population, and research activity     

Data generator for evaluating ETL process quality

Obtaining the right set of data for evaluating the fulfillment of different quality factors in the extract-transform-load (ETL) process design is rather challenging. First, the real data might be out of reach due to different privacy constraints, while manually providing a synthetic set of data is known as a labor-intensive task that needs to take various combinations of process parameters into account. More importantly, having a single dataset usually does not represent the evolution of data throughout the complete process lifespan, hence missing the plethora of possible test cases. To facilitate such demanding task, in this paper we propose an automatic data generator (i.e., Bijoux). Starting from a given ETL process model, Bijoux extracts the semantics of data transformations, analyzes the constraints they imply over input data, and automatically generates testing datasets. Bijoux is highly modular and configurable to enable end-users to generate datasets for a variety of interesting test scenarios (e.g., evaluating specific parts of an input ETL process design, with different input dataset sizes, different distributions of data, and different operation selectivities). We have developed a running prototype that implements the functionality of our data generation framework and here we report our experimental findings showing the effectiveness and scalability of our approach.