Ensemble of surrogates

The custom in surrogate-based modeling of complex engineering problems is to fit one or more surrogate models and select the one surrogate model that performs best. In this paper, we extend the utility of an ensemble of surrogates to (1) identify regions of possible high errors at locations where predictions of surrogates widely differ, and (2) provide a more robust approximation approach. We explore the possibility of using the best surrogate or a weighted average surrogate model instead of individual surrogate models. The weights associated with each surrogate model are determined based on the errors in surrogates. We demonstrate the advantages of an ensemble of surrogates using analytical problems and one engineering problem. We show that for a single problem the choice of test surrogate can depend on the design of experiments.     

Ensemble of surrogates with recursive arithmetic average

Surrogate models are often used to replace expensive simulations of engineering problems. The common approach is to construct a series of metamodels based on a training set, and then, from these surrogates, pick out the best one with the highest accuracy as an approximation of the computationally intensive simulation. However, because the choice of approximate model depends on design of experiments (DOEs), the traditional strategy thus increases the risk of adopting an inappropriate model. Furthermore, in the design of complex product system, because of its feature of one-of-a-kind production, acquiring more samples is very expensive and intensively time-consuming, and sometimes even impossible. Therefore, in order to save sampling cost, it is a reasonable strategy to take full advantage of all the stand-alone surrogates and then combine them into an ensemble model. Ensemble technique is an effective way to make up for the shortfalls of traditional strategy. Motivated by the previous research on ensemble of surrogates, a new technique for constructing of a more accurate ensemble of surrogates is proposed in this paper. The weights are obtained using a recursive process, in which the values of these weights are updated in each iteration until the last ensemble achieves a desirable prediction accuracy. This technique has been evaluated using five benchmark problems and one reality problem. The results show that the proposed ensemble of surrogates with recursive arithmetic average provides more ideal prediction accuracy than the stand-alone surrogates and for most problems even exceeds the previously presented ensemble techniques. Finally, we should point out that the advantages of combination over selection are still difficult to illuminate. We are still using an “insurance policy” mode rather than offering significant improvements.     

Multiple surrogates: how cross-validation errors can help us to obtain the best predictor

Surrogate models are commonly used to replace expensive simulations of engineering problems. Frequently, a single surrogate is chosen based on past experience. This approach has generated a collection of papers comparing the performance of individual surrogates. Previous work has also shown that fitting multiple surrogates and picking one based on cross-validation errors (PRESS in particular) is a good strategy, and that cross-validation errors may also be used to create a weighted surrogate. In this paper, we discussed how PRESS (obtained either from the leave-one-out or from the k-fold strategies) is employed to estimate the RMS error, and whether to use the best PRESS solution or a weighted surrogate when a single surrogate is needed. We also studied the minimization of the integrated square error as a way to compute the weights of the weighted average surrogate. We found that it pays to generate a large set of different surrogates and then use PRESS as a criterion for selection. We found that (1) in general, PRESS is good for filtering out inaccurate surrogates; and (2) with sufficient number of points, PRESS may identify the best surrogate of the set. Hence the use of cross-validation errors for choosing a surrogate and for calculating the weights of weighted surrogates becomes more attractive in high dimensions (when a large number of points is naturally required). However, it appears that the potential gains from using weighted surrogates diminish substantially in high dimensions. We also examined the utility of using all the surrogates for forming the weighted surrogates versus using a subset of the most accurate ones. This decision is shown to depend on the weighting scheme. Finally, we also found that PRESS as obtained through the k-fold strategy successfully estimates the RMSE.     

Ensemble of metamodels with optimized weight factors

Approximate mathematical models (metamodels) are often used as surrogates for more computationally intensive simulations. The common practice is to construct multiple metamodels based on a common training data set, evaluate their accuracy, and then to use only a single model perceived as the best while discarding the rest. This practice has some shortcomings as it does not take full advantage of the resources devoted to constructing different metamodels, and it is based on the assumption that changes in the training data set will not jeopardize the accuracy of the selected model. It is possible to overcome these drawbacks and to improve the prediction accuracy of the surrogate model if the separate stand-alone metamodels are combined to form an ensemble. Motivated by previous research on committee of neural networks and ensemble of surrogate models, a technique for developing a more accurate ensemble of multiple metamodels is presented in this paper. Here, the selection of weight factors in the general weighted-sum formulation of an ensemble is treated as an optimization problem with the desired solution being one that minimizes a selected error metric. The proposed technique is evaluated by considering one industrial and four benchmark problems. The effect of different metrics for estimating the prediction error at either the training data set or a few validation points is also explored. The results show that the optimized ensemble provides more accurate predictions than the stand-alone metamodels and for most problems even surpassing the previously reported ensemble approaches.     

Daily prediction of ICU readmissions using feature engineering and ensemble fuzzy modeling

This research is focused on the prediction of ICU readmissions using fuzzy modeling and feature selection approaches. There are a number of published scores for assessing the risk of readmissions, but their poor predictive performance renders them unsuitable for implementation in the clinical setting. In this work, we propose the use of feature engineering and advanced computational intelligence techniques to improve the performance of current models. In particular, we propose an approach that relies on transforming raw vital signs, laboratory results and demographic information into more informative pieces of data, selecting a subset of relevant and non–redundant variables and applying fuzzy ensemble modeling to the feature–engineered data for deriving important nonlinear relations between variables. Different criteria for selecting the best predictor from the ensemble and novel evaluation measures are explored. In particular, the area under the sensitivity curve and area under the specificity curve are investigated. The ensemble approach combined with feature transformation and feature selection showed increased performance, being able to predict early readmissions with an AUC of 0.77 ± 0.02. To the best of our knowledge, this is the first computational intelligence technique allowing the prediction of readmissions in a daily basis. The high balance between sensitivity and specificity shows its strength and suitability for the management of the patient discharge decision making process.     

Classifier subset selection for biomedical named entity recognition

Classifier ensembling approach is considered for biomedical named entity recognition task. A vote-based classifier selection scheme having an intermediate level of search complexity between static classifier selection and real-valued and class-dependent weighting approaches is developed. Assuming that the reliability of the predictions of each classifier differs among classes, the proposed approach is based on selection of the classifiers by taking into account their individual votes. A wide set of classifiers, each based on a different set of features and modeling parameter setting are generated for this purpose. A genetic algorithm is developed so as to label the predictions of these classifiers as reliable or not. During testing, the votes that are labeled as being reliable are combined using weighted majority voting. The classifier ensemble formed by the proposed scheme surpasses the full object F-score of the best individual classifier by 2.75% and it is the highest score achieved on the data set considered.     

QoS Requirement Generation and Algorithm Selection for Composite Service Based on Reference Vector

Under SOA (Service-Oriented Architecture), composite service is formed by aggregating multiple component services together in a given workflow. One key criterion of this research topic is QoS composition. Most work on service composition mainly focuses on the algorithms about how to compose services according to assumed QoS, without considering where the required QoS comes from and the selection of user preferred composition algorithm among those with different computational cost and di?erent selection resu...     

Ensemble cluster pruning via convex-concave programming

Ensemble learning is the process of aggregating the decisions of different learners/models. Fundamentally, the performance of the ensemble relies on the degree of accuracy in individual learner predictions and the degree of diversity among the learners. The trade-off between accuracy and diversity within the ensemble needs to be optimized to provide the best grouping of learners as it relates to their performance. In this optimization theory article, we propose a novel ensemble selection algorithm which, focusing specifically on clustering problems, selects the optimal subset of the ensemble that has both accurate and diverse models. Those ensemble selection algorithms work for a given number of the best learners within the subset prior to their selection. The cardinality of a subset of the ensemble changes the prediction accuracy. The proposed algorithm in this study determines both the number of best learners and also the best ones. We compared our prediction results to recent ensemble clustering selection algorithms by the number of cardinalities and best predictions, finding better and approximated results to the optimum solutions.     

Defending recommender systems: detection of profile injection attacks

Collaborative recommender systems are known to be highly vulnerable to profile injection attacks, attacks that involve the insertion of biased profiles into the ratings database for the purpose of altering the system’s recommendation behavior. Prior work has shown when profiles are reverse engineered to maximize influence; even a small number of malicious profiles can significantly bias the system. This paper describes a classification approach to the problem of detecting and responding to profile injection attacks. A number of attributes are identified that distinguish characteristics present in attack profiles in general, as well as an attribute generation approach for detecting profiles based on reverse engineered attack models. Three well-known classification algorithms are then used to demonstrate the combined benefit of these attributes and the impact the selection of classifier has with respect to improving the robustness of the recommender system. Our study demonstrates this technique significantly reduces the impact of the most powerful attack models previously studied, particularly when combined with a support vector machine classifier. This research was supported in part by the National Science Foundation Cyber Trust program under Grant IIS-0430303 and the National Science Foundation IGERT program under Grant DGE-0549489.     

An Evolutionary Algorithm Approach to Optimal Ensemble Classifiers for DNA Microarray Data Analysis

In general, the analysis of microarray data requires two steps: feature selection and classification. From a variety of feature selection methods and classifiers, it is difficult to find optimal ensembles composed of any feature-classifier pairs. This paper proposes a novel method based on the evolutionary algorithm (EA) to form sophisticated ensembles of features and classifiers that can be used to obtain high classification performance. In spite of the exponential number of possible ensembles of individual feature-classifier pairs, an EA can produce the best ensemble in a reasonable amount of time. The chromosome is encoded with real values to decide the weight for each feature-classifier pair in an ensemble. Experimental results with two well-known microarray datasets in terms of time and classification rate indicate that the proposed method produces ensembles that are superior to individual classifiers, as well as other ensembles optimized by random and greedy strategies.     

Local tests for consistency of support hyperplane data

Support functions and samples of convex bodies in R ⁿ are studied with regard to conditions for their validity or consistency. Necessary and sufficient conditions for a function to be a support function are reviewed in a general setting. An apparently little known classical such result for the planar case due to Rademacher and based on a determinantal inequality is presented and a generalization to arbitrary dimensions is developed. These conditions are global in the sense that they involve values of the support function at widely separated points. The corresponding discrete problem of determining the validity of a set of samples of a support function is treated. Conditions similar to the continuous inequality results are given for the consistency of a set of discrete support observations. These conditions are in terms of a series of local inequality tests involving only neighboring support samples. Our results serve to generalize existing planar conditions to arbitrary dimensions by providing a generalization of the notion of nearest neighbor for plane vectors which utilizes a simple positive cone condition on the respective support sample normals.This work partially supported by the Center for Intelligent Control Systems under the U.S. Army Research Office Grant DAAL03-92-G-0115, the Office of Naval Research under Grant N00014-91-J-1004, and the National Science Foundation under Grant MIP-9015281.Partially supported by the National Science Foundation under grant IRI-9209577 and by the U.S. Army Research Office under grant DAAL03-92-G-0320     

Various approaches for constructing an ensemble of metamodels using local measures

Metamodels are approximate mathematical models used as surrogates for computationally expensive simulations. Since metamodels are widely used in design space exploration and optimization, there is growing interest in developing techniques to enhance their accuracy. It has been shown that the accuracy of metamodel predictions can be increased by combining individual metamodels in the form of an ensemble. Several efforts were focused on determining the contribution (or weight factor) of a metamodel in the ensemble using global error measures. In addition, prediction variance is also used as a local error measure to determine the weight factors. This paper investigates the efficiency of using local error measures, and also presents the use of the pointwise cross validation error as a local error measure as an alternative to using prediction variance. The effectiveness of ensemble models are tested on several problems with varying dimensionality: five mathematical benchmark problems, two structural mechanics problems and an automobile crash problem. It is found that the spatial ensemble models show better performances than the global ensemble for the low-dimensional problems, while the global ensemble is a more accurate model than the spatial ensembles for the high-dimensional problems. Ensembles based on pointwise cross validation error and prediction variance provide similar accuracy. The ensemble models based on local measures reduce cross validation errors drastically, but their performances are not that impressive in reducing the error evaluated at random test points, because the pointwise cross validation error is not a good surrogate for the error at a point.     

Evaluation for Small Visual Difference Between Conforming Meshes on Strain Field

This paper gives a method of quantifying small visual differences between 3D mesh models with conforming topology, based on the theory of strain fields. Strain field is a geometric quantity in elasticity which is used to describe the deformation of elastomer. In this paper we consider the 3D models as objects with elasticity. The further demonstrations are provided: the first is intended to give the reader a visual impression of how our measure works in practice; and the second is to give readers a visua...     

An adaptive multipath routing scheme for connectionless traffic in an ATM network

This paper addresses the problem of routing connectionless traffic through an ATM network. A solution is proposed based on a per-packet adaptive multipath routing scheme which is added to the routing algorithm implemented at the Inter-Working Units. A scheme is presented that distributes packets among multiple Virtual Paths (VPs) according to the utilization of the links on these VPs. The utilization of the VPs is determined by a periodic feedback mechanism. Simulation studies show the effectiveness of the proposed adaptive multipath routing scheme.The work by J. Sole-Pareta was supported in part by a CIRIT (Generalitat de Catalunya) grant (expedient number EE92/2-338), and in part by the National Science Foundation under Grant No. INT-94033646. The work by I. Akyildiz was supported in part by the National Science Foundation under Grant No. INT-94033646.     

OR parallel execution of Prolog programs with side effects

With the growing availability of multiprocessors, a great deal of attention has been given to executing Prolog in parallel. A question that naturally arises is how to execute standard sequential Prolog programs with side effects in parallel. The problem of performing side effects in AND parallel systems has been considered elsewhere. This paper presents a method that generates sequential semantics of side effect predicates in an OR parallel system. First, a general method is given for performing data side effects such as read and write. This method is then extended to control side effects such as asserta, assertz, and retract. Finally, a constant-time algorithm for performing cut is presented.The work of L. V. Kale was supported by the National Science Foundation under Grant NSF-CCR-8700988. The work of D. A. Padua and D. C. Sehr was supported in part by the National Science Foundation under Grant NSF-MIP-8410110, the Department of Energy under Grant DOE DE-FG02-85ER25001, and a donation from the IBM Corporation to the Center for Supercomputing Research and Development. D. C. Sehr holds a fellowship from the Office of Naval Research.     

A Bitonic Selection Algorithm on Multiprocessor System

The so-called (m, n) selection problem is defined as the selection of them smallest (or largest) numbers fromn given numbers (n>m). Solving this problem in parallel mode has been successful on the networks, but it is seldom studied on the multiprocessor systems. This paper first, based on Batcher’s principle of bitonic merging, proposes the bitonic selection network. Then it repeals the varying rule of the pivots in all successive ranks of the network through our observation to the data transfer property in the network. Finally, according to this rule, the parallel selection algorithm with running timeO (lognlogm)¹⁾ onn processors is presented. This work was supported by the National Science Foundation of China under Grant Tech.-85217.     

Demand-Driven Memory Leak Detection Based on Flow- and Context-Sensitive Pointer Analysis

We present a demand-driven approach to memory leak detection algorithm based on flow- and context-sensitive pointer analysis. The detection algorithm firstly assumes the presence of a memory leak at some program point and then runs a backward analysis to see if this assumption can be disproved. Our algorithm computes the memory abstraction of programs based on points-to graph resulting from flow- and context-sensitive pointer analysis. We have implemented the algorithm in the SUIF2 compiler infrastructure and used the implementation to analyze a set of C benchmark programs. The experimental results show that the approach has better precision with satisfied scalability as expected. This work is supported by the National Natural Science Foundation of China under Grant Nos. 60725206, 60673118, and 90612009, the National High-Tech Research and Development 863 Program of China under Grant No. 2006AA01Z429, the National Basic Research 973 Program of China under Grant No. 2005CB321802, the Program for New Century Excellent Talents in University under Grant No. NCET-04-0996, and the Hunan Natural Science Foundation under Grant No. 07JJ1011.     

Ensembles of label noise filters: a ranking approach

Label noise can be a major problem in classification tasks, since most machine learning algorithms rely on data labels in their inductive process. Thereupon, various techniques for label noise identification have been investigated in the literature. The bias of each technique defines how suitable it is for each dataset. Besides, while some techniques identify a large number of examples as noisy and have a high false positive rate, others are very restrictive and therefore not able to identify all noisy examples. This paper investigates how label noise detection can be improved by using an ensemble of noise filtering techniques. These filters, individual and ensembles, are experimentally compared. Another concern in this paper is the computational cost of ensembles, once, for a particular dataset, an individual technique can have the same predictive performance as an ensemble. In this case the individual technique should be preferred. To deal with this situation, this study also proposes the use of meta-learning to recommend, for a new dataset, the best filter. An extensive experimental evaluation of the use of individual filters, ensemble filters and meta-learning was performed using public datasets with imputed label noise. The results show that ensembles of noise filters can improve noise filtering performance and that a recommendation system based on meta-learning can successfully recommend the best filtering technique for new datasets. A case study using a real dataset from the ecological niche modeling domain is also presented and evaluated, with the results validated by an expert.     

A Global Chance-Constraint for Stochastic Inventory Systems Under Service Level Constraints

We consider a class of production/inventory control problems that has a single product and a single stocking location, for which a stochastic demand with a known non-stationary probability distribution is given. Under the widely-known replenishment cycle policy the problem of computing policy parameters under service level constraints has been modeled using various techniques. Tarim and Kingsman introduced a modeling strategy that constitutes the state-of-the-art approach for solving this problem. In this paper we identify two sources of approximation in Tarim and Kingsman’s model and we propose an exact stochastic constraint programming approach. We build our approach on a novel concept, global chance-constraints, which we introduce in this paper. Solutions provided by our exact approach are employed to analyze the accuracy of the model developed by Tarim and Kingsman. This work was supported by Science Foundation Ireland under Grant No. 03/CE3/I405 as part of the Centre for Telecommunications Value-Chain-Driven Research (CTVR) and Grant No. 00/PI.1/C075.     

The deterministic subspace method for constructing classifier ensembles

Ensemble classification remains one of the most popular techniques in contemporary machine learning, being characterized by both high efficiency and stability. An ideal ensemble comprises mutually complementary individual classifiers which are characterized by the high diversity and accuracy. This may be achieved, e.g., by training individual classification models on feature subspaces. Random Subspace is the most well-known method based on this principle. Its main limitation lies in stochastic nature, as it cannot be considered as a stable and a suitable classifier for real-life applications. In this paper, we propose an alternative approach, Deterministic Subspace method, capable of creating subspaces in guided and repetitive manner. Thus, our method will always converge to the same final ensemble for a given dataset. We describe general algorithm and three dedicated measures used in the feature selection process. Finally, we present the results of the experimental study, which prove the usefulness of the proposed method.