Multivariate decision trees using linear discriminants and tabu search

A new decision tree method for application in data mining, machine learning, pattern recognition, and other areas is proposed in this paper. The new method incorporates a classical multivariate statistical method, linear discriminant function, into decision trees' recursive partitioning process. The proposed method considers not only the linear combination with all variables, but also combinations with fewer variables. It uses a tabu search technique to find appropriate variable combinations within a reasonable length of time. For problems with more than two classes, the tabu search technique is also used to group the data into two superclasses before each split. The results of our experimental study indicate that the proposed algorithm appears to outperform some of the major classification algorithms in terms of classification accuracy, the proposed algorithm generates decision trees with relatively small sizes, and the proposed algorithm runs faster than most multivariate decision trees and its computing time increases linearly with data size, indicating that the algorithm is scalable to large datasets.     

Genetic algorithms combined with discriminant analysis for key variable identification

Many trouble-shooting problems in process industries are related to key variable identification for classifications. The contribution charts, based on principal component analysis (PCA), can be applied for this purpose. Genetic algorithms (GAs) have been proposed recently for many applications including variable selection for multivariate calibration, molecular modeling, regression analysis, model identification, curve fitting, and classification. In this paper, GAs are incorporated with Fisher discriminant analysis (FDA) for key variable identification. GAs are used as an optimization tool to determine variables that maximize the FDA classification success rate for two given data sets. GA/FDA is a proposed solution for the variable selection problem in discriminant analysis. The Tennessee Eastman process (TEP) simulator was used to generate the data sets to evaluate the correctness of the key variable selection using GA/FDA, and the T² and Q statistic contribution charts. GA/FDA correctly identifies the key variables for the TEP case studies that were tested. For one case study where the correlation changes in two data sets, the contribution charts incorrectly suggest that the operating conditions are similar. On the other hand, GA/FDA not only determines that the operating conditions are different, but also identifies the key variables for the change. For another case study where many key variables are responsible for the changes in the two data sets, the contribution charts only identifies a fraction of the key variables, while GA/FDA correctly identifies all of the key variables. GA/FDA is a promising technique for key variable identification, as is evidenced in successful applications at The Dow Chemical Company.     

Some local search algorithms for no-wait flow-shop problem with makespan criterion

This paper develops and compares different local search algorithms for the no-wait flow-shop problem with makespan criterion (C_max). We present several variants of descending search and tabu search algorithms. In the algorithms the multimoves are used that consist in performing several moves simultaneously in a single iteration of algorithm and allow us to accelerate the convergence to good solutions. Besides, in the tabu search algorithms a dynamic tabu list is proposed that assists additionally to avoid trapped at a local optimum. The proposed algorithms are empirically evaluated and found to be relatively more effective in finding better quality solutions than existing algorithms. The presented ideas can be applied in any local search procedures.     

An Efficient Stochastic Search for Bayesian Variable Selection with High-Dimensional Correlated Predictors

We present a Bayesian variable selection method for the setting in which the number of independent variables or predictors in a particular dataset is much larger than the available sample size. While most of the existing methods allow some degree of correlations among predictors but do not consider these correlations for variable selection, our method accounts for correlations among the predictors in variable selection. Our correlation-based stochastic search (CBS) method, the hybrid-CBS algorithm, extends a popular search algorithm for high-dimensional data, the stochastic search variable selection (SSVS) method. Similar to SSVS, we search the space of all possible models using variable addition, deletion or swap moves. However, our moves through the model space are designed to accommodate correlations among the variables. We describe our approach for continuous, binary, ordinal, and count outcome data. The impact of choices of prior distributions and hyperparameters is assessed in simulation studies. We also examined the performance of variable selection and prediction as the correlation structure of the predictors varies. We found that the hybrid-CBS resulted in lower prediction errors and identified better the true outcome associated predictors than SSVS when predictors were moderately to highly correlated. We illustrate the method on data from a proteomic profiling study of melanoma, a type of skin cancer.     

Decomposition based hybrid VNS–TS algorithm for distributed parallel factories scheduling with virtual corporation

The main purpose of this paper is to develop a decomposition based hybrid variable neighborhood search/tabu search (DVT) algorithm for multi-factory production network scheduling problem where a number of different individual factories collaborate despite their different objectives. It is assumed some of the network's factories are interested in total processing cost minimization whereas the remaining factories are interested in the production profits maximization. It is also assumed that jobs can migrate from their original factory to other factories but a transportation time is incurred. Our proposed algorithm comprises of a tabu search and a variable neighborhood search with several local search algorithms. In this hybridization, to improve the search ability of the algorithm, we make use of guiding principles with ordering of neighborhood structures by mixed integer linear programming relaxation. In the proposed algorithm, the parallel search strategy is designed for a scalar bi-objective. Multiple objectives are combined with L₁-metric technique then each sub-search procedure evolves separately until a good approximation of the Pareto-front is obtained. The non-dominated sets obtained from our algorithm and original heuristic (algorithm without ordering concept) are compared using three different indices. Furthermore, the problem is modeled as a mixed integer linear programming and solved by improved ϵ-constraint approach (IEA) with CPLEX solver. The results of comparisons between IEA and DVT algorithm showed the proposed algorithm yielded most of the solutions in the net non-dominated front.     

Fine-tuning a Tabu Search Algorithm with Statistical Tests

Tabu Search is a metaheuristic that has proven to be very effective for solving various types of combinatorial optimization problems. To achieve the best results with a tabu search algorithm, significant benefits can sometimes be gained by determining preferred values for certain search parameters such as tabu tenures, move selection probabilities, the timing and structure of elite solution recovery for intensification, etc. In this paper, we present and implement some new ideas for fine-tuning a tabu search algorithm using statistical tests. Although the focus of this work is to improve a particular tabu search algorithm developed for solving a telecommunications network design problem, the implications are quite general. The same ideas and procedures can easily be adapted and applied to other tabu search algorithms as well.     

逐步贝叶斯判别分析中的变量优化方法研究

判别分析中特征变量是影响判别结果的决定性因素,选取适当的特征变量组合可以提高正判率、减少计算量。介绍了贝叶斯判别和逐步判别法的基本原理,分析了目前出现的一些特征变量优化方法,以油气解释评价中的贝叶斯判别应用为例,对于逐步贝叶斯判别中的变量优化方法进行了研究和总结,提出了变量的多步优化策略和分步多模型优化策略,包含了从变量范围选择、数据预处理、特征变量提取到初步筛选和逐步判别的完整过程,使得正判率不断优化,最终得到了较为满意的判别结果。     

Lateness minimization with Tabu search for job shop scheduling problem with sequence dependent setup times

We tackle the job shop scheduling problem with sequence dependent setup times and maximum lateness minimization by means of a tabu search algorithm. We start by defining a disjunctive model for this problem, which allows us to study some properties of the problem. Using these properties we define a new local search neighborhood structure, which is then incorporated into the proposed tabu search algorithm. To assess the performance of this algorithm, we present the results of an extensive experimental study, including an analysis of the tabu search algorithm under different running conditions and a comparison with the state-of-the-art algorithms. The experiments are performed across two sets of conventional benchmarks with 960 and 17 instances respectively. The results demonstrate that the proposed tabu search algorithm is superior to the state-of-the-art methods both in quality and stability. In particular, our algorithm establishes new best solutions for 817 of the 960 instances of the first set and reaches the best known solutions in 16 of the 17 instances of the second set.     

Self Controlling Tabu Search algorithm for the Quadratic Assignment Problem

This paper addresses the application of the principles of feedback and self-controlling software to the tabu search algorithm. We introduce two new reaction strategies for the tabu search algorithm. The first strategy treats the tabu search algorithm as a target system to be controlled and uses a control-theoretic approach to adjust the algorithm parameters that affect search intensification. The second strategy is a flexible diversification strategy which can adjust the algorithm’s parameters based on the search history. These two strategies, combined with tabu search, form the Self Controlling Tabu Search (SC-Tabu) algorithm. The algorithm is implemented and tested on the Quadratic Assignment Problem (QAP). The results show that the self-controlling features of the algorithm make it possible to achieve good performance on different types of QAP instances.     

Frequency-driven tabu search for the maximum s-plex problem

The maximum s-plex problem is an important model for social network analysis and other studies. In this study, we present an effective frequency-driven multi-neighborhood tabu search algorithm (FD-TS) to solve the problem on very large networks. The proposed FD-TS algorithm relies on two transformation operators (Add and Swap) to locate high-quality solutions, and a frequency-driven perturbation operator (Press) to escape and search beyond the identified local optimum traps. We report computational results for 47 massive real-life (sparse) graphs from the SNAP Collection and the 10th DIMACS Challenge, as well as 52 (dense) graphs from the 2nd DIMACS Challenge (results for 48 more graphs are also provided in the Appendix). We demonstrate the effectiveness of our approach by presenting comparisons with the current best-performing algorithms.     

Discrete-variable optimal structural design using tabu search

Tabu search is a discrete-variable optimization algorithm with the ability to avoid entrapment by local optima and hence continue searching for a global optimum. In this paper tabu search is applied to the optimal structural design, in terms of weight minimization, of two standard (test) structural configurations; a 10-bar planar truss and a 25-bar space truss. The design variables are the cross-sectional areas of the bars, which take discrete values.An implementation of tabu search in a structural design context is presented which features-depth neighbourhoods and a search back-track facility. Investigations show that tabu search may readily cope with problem formulations that include buckling, gravity effects and design variables with realistic values. Furthermore, compared to previous research, superior (i.e. lower) minimum weights may be obtained.It is shown that tabu search is a technically viable technique for use in optimal structural design, although, for practical use, current (significant) execution times may inhibit its utilisation.     

基于单亲遗传禁忌搜索算法的手术排程问题研究

针对规模较大的手术排程问题,分别以所有病人完成手术过程的最长时间和平均时间最小化为目标,构建了手术排程问题的数学模型。在分析解的最优化条件基础上,设计了一种将单亲遗传算法与禁忌搜索算法相结合的混合优化算法。按照个体的优劣及算法迭代情况设计了一种自适应选择机制,使个体自适应地选择执行变异操作或禁忌搜索算法。最后,仿真实验结果表明了所提算法的有效性和自适应选择机制的可行性。     

Tabu搜索在特征选择中的应用

研究利用Tabu搜索从大特征集中选择一组有效特征的问题.分析了Tabu搜索中 表长、邻域大小和候选解数量等参数对Tabu搜索的影响.对两种特征选择的问题,与经典及 最近新提出的一些特征选择方法如SFS,SBS,GSFS,GSBS,PTA,BB,GA和SFFS,SFBS等 算法的实验比较表明,Tabu搜索在求解时间和解的质量上都取得了满意的结果.     

An integer program and a hybrid genetic algorithm for the university timetabling problem

The university timetabling problem (UTP) has been studied by numerous research groups for decades. In addition to addressing hard and soft constraints, we extend the UTP by considering consecutiveness and periodicity constraints of multi-session lectures, which are common in many eastern Asian universities. Because schedulers can decide the consecutiveness and periodicity constraints for the multi-session lectures within a limited ratio, we consider these novel decision variables in our model. We develop a mixed integer linear program for the UTP. For the analysis, we convert the UTP into the three-dimensional container packing problem (3DCPP) and create a hybrid genetic algorithm (HGA), which has been shown to be efficient in solving the 3DCPP. We also develop a tabu search algorithm based on the existing UTP literature and compare the findings with that of our HGA. The results show that our HGA obtains a better solution than the tabu search algorithm in a reasonable amount of time.     

Entropic component analysis and its application in geological data

We present an entropic component analysis for identifying key parameters or variables and the joint effects of various parameters that characterize complex systems. This approach identifies key parameters through solving the variable selection problem. It consists of two steps. First, a Bayesian approach is utilized to convert the variable selection problem into the model selection problem. Second, the model selection is achieved uniquely by evaluating the information difference of models by relative entropies of these models and a reference model. We study a geological sample classification problem, where a brine sample from Texas and Oklahoma oil field is considered, to illustrate and examine the proposed approach. The results are consistent with qualitative analysis of the lithology and quantitative discriminant function analysis. Furthermore, the proposed approach reveals the joint effects of the parameters, while it is unclear from the discriminant function analysis. The proposed approach could be thus promising to various geological data analysis.     

A Tabu search approach for designing a non-hierarchical video-on-demand network architecture

In this paper, we present a tabu search to design a non-hierarchical and decentralized video-on-demand (VOD) network architecture. To optimize the VOD network resource, we consider optimization of both video server locations and storage allocation subject to the tradeoffs among installation cost for video servers, program storage cost, and transmission (or communication) cost. In applying a tabu search technique to the problem, neighborhood structure and search strategy are elaborated to improve solution quality and to reduce computation time. We report the results of the computational experiments to demonstrate the performance of the proposed tabu search. A comparative study shows that our algorithm is promising.     

基于可变禁忌长度的优化停机位分配

针对机场"最大化停机位利用率"以及"最小化旅客行走路程"问题,提出了一种动态、灵活分配停机位的禁忌搜索算法。首先介绍了基本禁忌搜索算法的相关设计,然后引出了改进后的动态禁忌搜索算法（DTS算法）,最后利用实际数据对改进后的禁忌搜索算法进行演算。通过几组数据的对比可看出,突出可变禁忌长度能够缩短全局寻优的循环次数。而与相关文献的演算结果进行对比显示：在资源不受限情况下,旅客行走总时间减少了15.75%;在资源受限情况下,旅客行走总时间减少了22.84%。实验结果表明,采用动态禁忌搜索算法能够得到更小的旅客行走路程的分配方案。     

An efficient variable partitioning approach for functional decomposition of circuits

Functional decomposition is a process of splitting a complex circuit into smaller sub-circuits. There exist two major strategies in decomposition, namely, serial and parallel decomposition. In serial decomposition the problem the complex function represented as a truth table with support set variables and partitioned into free and bout set variables. The minterms corresponding to the bound set variables are represented as an equivalent function called the predecessor function. Equivalent minterms of the bound set variables are assigned an output code. The assigned output codes and the free set variable minterms are represented as the successor function. Serial decomposition is further categorized into disjoint and non-disjoint decomposition, when the free and bound set variables are disjoint and non-disjoint respectively. This paper deals with the problem of determining the set of best free and bound variables (variable partitioning problem) for disjoint serial decomposition. Variable partitioning is the first step in decomposition process. An efficient variable partition algorithm is one that determines the set of all free and bound set variables that satisfy the decomposition theorem in minimal time and by exploring the search space effectively. This will allow the decomposition algorithm to determine the best variable partition of a function that results in smaller decomposed functions and with maximum number of do not cares in these functions. Classical approaches to determine the best free and bound set use exhaustive search methods. The time and memory requirements for such approaches are exponential or super exponential.A novel heuristic search approach is proposed to determine the set of good variable partitions in minimal time by minimally exploring the search space. There are two heuristics employed in the proposed search approach, (1) r-admissibility based heuristic or pruned breadth first search (PBFS) approach and (2) Information relation based heuristic or improved pruned breadth first search (IPBFS) approach. The r-admissibility based heuristic is based on r-partition characteristics of the free and bound set variables. The information relation and measure based heuristic is based on information relationship of free and bound set variables that are expressed as r-partition heuristics. The proposed variable partition search approach has been successfully implemented and test with MCNC and Espresso benchmarks and the results indicate that the time complexity is comparable to r-admissible heuristic algorithm and the quality of solution is comparable to exact variable partitioning algorithm. A comparison of PBFS and IPBFS heuristics for certain benchmarks are also discussed in this paper.     

A tabu search algorithm for order acceptance and scheduling

We consider a make-to-order production system, where limited production capacity and order delivery requirements necessitate selective acceptance of the orders. Since tardiness penalties cause loss of revenue, scheduling and order acceptance decisions must be taken jointly to maximize total revenue. We present a tabu search algorithm that solves the order acceptance and scheduling problem on a single machine with release dates and sequence dependent setup times. We analyze the performance of the tabu search algorithm on an extensive set of test instances with up to 100 orders and compare it with two heuristics from the literature. In the comparison, we report optimality gaps which are calculated with respect to bounds generated from a mixed integer programming formulation. The results show that the tabu search algorithm gives near optimal solutions that are significantly better compared to the solutions given by the two heuristics. Furthermore, the run time of the tabu search algorithm is very small, even for 100 orders. The success of the proposed heuristic largely depends on its capability to incorporate in its search acceptance and scheduling decisions simultaneously.     

A graph coloring heuristic using partial solutions and a reactive tabu scheme

Most of the recent heuristics for the graph coloring problem start from an infeasible k-coloring (adjacent vertices may have the same color) and try to make the solution feasible through a sequence of color exchanges. In contrast, our approach (called FOO-PARTIALCOL), which is based on tabu search, considers feasible but partial solutions and tries to increase the size of the current partial solution. A solution consists of k disjoint stable sets (and, therefore, is a feasible, partial k-coloring) and a set of uncolored vertices. We introduce a reactive tabu tenure which substantially enhances the performance of both our heuristic as well as the classical tabu algorithm (called TABUCOL) proposed by Hertz and de Werra [Using tabu search techniques for graph coloring, Computing 1987;39:345–51]. We will report numerical results on different benchmark graphs and we will observe that FOO-PARTIALCOL, though very simple, outperforms TABUCOL on some instances, provides very competitive results on a set of benchmark graphs which are known to be difficult, and outperforms the best-known methods on the graph flat300_28_0. For this graph, FOO-PARTIALCOL finds an optimal coloring with 28 colors. The best coloring achieved to date uses 31 colors. Algorithms very close to TABUCOL are still used as intensification procedures in the best coloring methods, which are evolutionary heuristics. FOO-PARTIALCOL could then be a powerful alternative. In conclusion FOO-PARTIALCOL is one of the most efficient simple local search coloring methods yet available.