Genetic algorithms in classifier fusion

An intense research around classifier fusion in recent years revealed that combining performance strongly depends on careful selection of classifiers to be combined. Classifier performance depends, in turn, on careful selection of features, which could be further restricted by the subspaces of the data domain. On the other hand, there is already a number of classifier fusion techniques available and the choice of the most suitable method depends back on the selections made within classifier, features and data spaces. In all these multidimensional selection tasks genetic algorithms (GA) appear to be one of the most suitable techniques providing reasonable balance between searching complexity and the performance of the solutions found. In this work, an attempt is made to revise the capability of genetic algorithms to be applied to selection across many dimensions of the classifier fusion process including data, features, classifiers and even classifier combiners. In the first of the discussed models the potential for combined classification improvement by GA-selected weights for the soft combining of classifier outputs has been investigated. The second of the proposed models describes a more general system where the specifically designed GA is applied to selection carried out simultaneously along many dimensions of the classifier fusion process. Both, the weighted soft combiners and the prototype of the three-dimensional fusion–classifier–feature selection model have been developed and tested using typical benchmark datasets and some comparative experimental results are also presented.     

Fuzzy classifier design using genetic algorithms

A new method for design of a fuzzy-rule-based classifier using genetic algorithms (GAs) is discussed. The optimal parameters of the fuzzy classifier including fuzzy membership functions and the size and structure of fuzzy rules are extracted from the training data using GAs. This is done by introducing new representation schemes for fuzzy membership functions and fuzzy rules. An effectiveness measure for fuzzy rules is developed that allows for systematic addition or deletion of rules during the GA optimization process. A clustering method is utilized for generating new rules to be added when additions are required. The performance of the classifier is tested on two real-world databases (Iris and Wine) and a simulated Gaussian database. The results indicate that highly accurate classifiers could be designed with relatively few fuzzy rules. The performance is also compared to other fuzzy classifiers tested on the same databases.     

Designing a classifier by a layered multi-population genetic programming approach

This paper proposes a method called layered genetic programming (LAGEP) to construct a classifier based on multi-population genetic programming (MGP). LAGEP employs layer architecture to arrange multiple populations. A layer is composed of a number of populations. The results of populations are discriminant functions. These functions transform the training set to construct a new training set. The successive layer uses the new training set to obtain better discriminant functions. Moreover, because the functions generated by each layer will be composed to a long discriminant function, which is the result of LAGEP, every layer can evolve with short individuals. For each population, we propose an adaptive mutation rate tuning method to increase the mutation rate based on fitness values and remaining generations. Several experiments are conducted with different settings of LAGEP and several real-world medical problems. Experiment results show that LAGEP achieves comparable accuracy to single population GP in much less time.     

Optimal resampling and classifier prototype selection in classifier ensembles using genetic algorithms

Ensembles of classifiers that are trained on different parts of the input space provide good results in general. As a popular boosting technique, AdaBoost is an iterative and gradient based deterministic method used for this purpose where an exponential loss function is minimized. Bagging is a random search based ensemble creation technique where the training set of each classifier is arbitrarily selected. In this paper, a genetic algorithm based ensemble creation approach is proposed where both resampled training sets and classifier prototypes evolve so as to maximize the combined accuracy. The objective function based random search procedure of the resultant system guided by both ensemble accuracy and diversity can be considered to share the basic properties of bagging and boosting. Experimental results have shown that the proposed approach provides better combined accuracies using a fewer number of classifiers than AdaBoost.     

Designing targeted libraries with genetic algorithms

In combinatorial synthesis, molecules are assembled by linking chemically similar fragments. Because the number of available chemical fragments often greatly exceeds the number that can be used in one synthetic experiment, one needs a rational method for choosing a subset of desirable fragments. If a combinatorial library is to be targeted against a particular biological activity, virtual screening methods can be used to predict which molecules in a virtual library are most likely to be active. When the number of possible molecules in a virtual library is very large, genetic algorithms (GAs) or simulated annealing can be used to quickly find high-scoring molecules by sampling a small subset of the total combinatorial space. We previously demonstrated how a GA can be used to select a subset of fragments for a combinatorial library, and we used topology-based methods of scoring. Here we extend that earlier work in three ways. (1) We demonstrate use of the GA with 3D scoring methods developed in our laboratory. (2) We show that the approach of assembling libraries from fragments in high-scoring molecules is a reasonable one. (3) We compare results from a library-based GA to those from a molecule-based GA.     

Overfitting cautious selection of classifier ensembles with genetic algorithms

Information fusion research has recently focused on the characteristics of the decision profiles of ensemble members in order to optimize performance. These characteristics are particularly important in the selection of ensemble members. However, even though the control of overfitting is a challenge in machine learning problems, much less work has been devoted to the control of overfitting in selection tasks. The objectives of this paper are: (1) to show that overfitting can be detected at the selection stage; and (2) to present strategies to control overfitting. Decision trees and k nearest neighbors classifiers are used to create homogeneous ensembles, while single- and multi-objective genetic algorithms are employed as search algorithms at the selection stage. In this study, we use bagging and random subspace methods for ensemble generation. The classification error rate and a set of diversity measures are applied as search criteria. We show experimentally that the selection of classifier ensembles conducted by genetic algorithms is prone to overfitting, especially in the multi-objective case. In this study, the partial validation, backwarding and global validation strategies are tailored for classifier ensemble selection problem and compared. This comparison allows us to show that a global validation strategy should be applied to control overfitting in pattern recognition systems involving an ensemble member selection task. Furthermore, this study has helped us to establish that the global validation strategy can be used to measure the relationship between diversity and classification performance when diversity measures are employed as single-objective functions.     

用改进的遗传算法训练神经网络构造分类器

针对基本遗传算法存在容易早熟和局部搜索能力弱等缺陷,提出了改进的遗传算法,引入交叉概率和变异概率与个体的适度值相联系,改进了操作算子,而且在交叉操作后又引入模拟退火机制,提高遗传算法的局部搜索能力。同时,用改进的遗传算法和基本的遗传算法训练神经网络构造分类器,实验结果表明,改进的遗传算法在最好个体适度值和最好分类准确性等方面性能更好。     

Sum versus vote fusion in multiple classifier systems

Amidst the conflicting experimental evidence of superiority of one over the other, we investigate the Sum and majority Vote combining rules in a two class case, under the assumption of experts being of equal strength and estimation errors conditionally independent and identically distributed. We show, analytically, that, for Gaussian estimation error distributions, Sum always outperforms Vote. For heavy tail distributions, we demonstrate by simulation that Vote may outperform Sum. Results on synthetic data confirm the theoretical predictions. Experiments on real data support the general findings, but also show the effect of the usual assumptions of conditional independence, identical error distributions, and common target outputs of the experts not being fully satisfied.     

Classifier systems and genetic algorithms

Classifier systems are massively parallel, message-passing, rule-based systems that learn through credit assignment (the bucket brigade algorithm) and rule discovery (the genetic algorithm). They typically operate in environments that exhibit one or more of the following characteristics: (1) perpetually novel events accompanied by large amounts of noisy or irrelevant data; (2) continual, often real-time, requirements for action; (3) implicitly or inexactly defined goals; and (4) sparse payoff or reinforcement obtainable only through long action sequences. Classifier systems are designed to absorb new information continuously from such environments, devising sets of competing hypotheses (expressed as rules) without disturbing significantly capabilities already acquired. This paper reviews the definition, theory, and extant applications of classifier systems, comparing them with other machine learning techniques, and closing with a discussion of advantages, problems, and possible extensions of classifier systems.     

Testing real-time systems using genetic algorithms

The development of real-time systems is an essential industrial activity whose importance is increasing. The most important analytical method to assure the quality of real-time systems is dynamic testing. Testing is the only method which examines the actual run-time behaviour of real-time software, based on an execution in the real application environment. Dynamic aspects like the duration of computations, the memory actually needed, or the synchronization of parallel processes are of major importance for the correct function of real-time systems and have to be tested. A comprehensive investigation of existing software test methods shows that they mostly concentrate on testing for functional correctness. They are not suited for an examination of temporal correctness which is essential to real-time systems. Very small systems show a wide range of different execution times. Therefore, existing test procedures must be supplemented by new methods, which concentrate on determining whether the system violates its specified timing constraints. In general, this means that outputs are produced too early or their computation takes too long. The task of the tester is to find the inputs with the longest or shortest execution times to check whether they produce a temporal error. If the search for such inputs is interpreted as a problem of optimization, genetic algorithms can be used to find the inputs with the longest or shortest execution times automatically. The fitness function is the execution time measured in processor cycles. Experiments using genetic algorithms on a number of programs with up to 1511 LOC and 843 integer input parameters have successfully identified new longer and shorter paths than had been found using random testing or systematic testing. Genetic algorithms are able therefore to check large programs and they show considerable promise in establishing the validity of the temporal behaviour of real-time software.     

Leveraging saving-based algorithms by master-slave genetic algorithms

Saving-based algorithms are commonly used as inner mechanisms of efficient heuristic construction procedures. We present a general mechanism for enhancing the effectiveness of such heuristics based on a two-level genetic algorithm. The higher-level algorithm searches in the space of possible merge lists which are then used by the lower-level saving-based algorithm to build the solution. We describe the general framework and we illustrate its application to three hard combinatorial problems. Experimental results on three hard combinatorial optimization problems show that the approach is very effective and it enables considerable enhancement of the performance of saving-based algorithms.     

System-level simulations of multi-sensor systems and data fusion algorithms

Microsystem Technologies - System-level simulations of sensors are valuables for optimizing device and system parameters and validating data-processing algorithms. Nowadays, the tendency of...     

Designing broadcasting algorithms in the postal model for message-passing systems

In many distributed-memory parallel computers and high-speed communication networks, the exact structure of the underlying communication network may be ignored. These systems assume that the network creates a complete communication graph between the processors, in which passing messages is associated with communication latencies. In this paper we explore the impact of communication latencies on the design of broadcasting algorithms for fully connected message-passing systems. For this purpose, we introduce thepostal model that incorporates a communication latency parameter 1. This parameter measures the inverse of the ratio between the time it takes an originator of a message to send the message and the time that passes until the recipient of the message receives it. We present an optimal algorithm for broadcasting one message in systems withn processors and communication latency , the running time of which is (( logn)/log( + 1)). For broadcastingm 1 messages, we first examine several generalizations of the algorithm for broadcasting one message and then analyze a family of broadcasting algorithms based on degree-d trees. All the algorithms described in this paper are practical event-driven algorithms that preserve the order of messages.     

Controller design for time-delay systems using genetic algorithms

This paper presents a new approach to design controllers for time-delay systems by using genetic algorithms (GAs) together with the solvability of linear matrix inequalities (LMIs). Both of the state-feedback controller and the static output feedback controller can be designed with this approach. It is confirmed by numerical examples that this approach achieves less conservative results than previously existing methods on the given examples.     

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions (i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.     

Fingerprint matching by genetic algorithms

Fingerprint matching is still a challenging problem for reliable person authentication because of the complex distortions involved in two impressions of the same finger. In this paper, we propose a fingerprint-matching approach based on genetic algorithms (GA), which tries to find the optimal transformation between two different fingerprints. In order to deal with low-quality fingerprint images, which introduce significant occlusion and clutter of minutiae features, we design a fitness function based on the local properties of each triplet of minutiae. The experimental results on National Institute of Standards and Technology fingerprint database, NIST-4, not only show that the proposed approach can achieve good performance even when a large portion of fingerprints in the database are of poor quality, but also show that the proposed approach is better than another approach, which is based on mean-squared error estimation.     

基于多时隙融合的LTE-U空闲信道评估算法

LTE-U是近年来3GPP提出的用于缓解授权频段通信压力的新兴技术,让LTE-U与Wi Fi在同信道高效共存是研究热点之一。首先建立了LTE-U与Wi Fi共存场景下的空闲信道评估(CCA)系统模型,随后提出了基于先听后说(LBT)的多时隙CCA方案。在此方案基础上,提出了4种数据融合算法,同时为降低判决复杂度引入了硬判决融合,并得到多时隙最优硬判决融合准则。理论分析和实验结果表明,所提的方案和算法可使CCA变得更加灵活,同时对信道状态的判定更加准确,可提升系统共存的效率。     

Synthesizing structures of real-time computer systems using genetic algorithms

A genetic algorithm is suggested for synthesizing real-time computer systems. The emphasis is on adjusting the algorithm to specific features of the problem and justifying the decisions made. Results of the analysis of the algorithm are also presented. Parameters of the genetic algorithm that provide good solutions are chosen by using a computational experiment.