On output independence and complementariness in rank-based multiple classifier decision systems

This study presents a theoretical analysis of output independence and complementariness between classifiers in a rank-based multiple classifier decision system in the context of the partitioned observation space theory. To enable such an analysis, an information theoretic interpretation of a rank-based multiple classifier system is developed and basic concepts from information theory are applied to develop measures for output independence and complementariness. It is shown that output independence of classifiers is not a requirement for achieving complementariness between these classifiers. Namely, output independence does not imply a performance improvement by combining multiple classifiers. A condition called dominance is shown to be important instead. The information theoretic measures proposed for output independence and complementariness are justified by simulated examples.     

A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems

In this paper, a generalized adaptive ensemble generation and aggregation (GAEGA) method for the design of multiple classifier systems (MCSs) is proposed. GAEGA adopts an “over-generation and selection” strategy to achieve a good bias-variance tradeoff. In the training phase, different ensembles of classifiers are adaptively generated by fitting the validation data globally with different degrees. The test data are then classified by each of the generated ensembles. The final decision is made by taking into consideration both the ability of each ensemble to fit the validation data locally and reducing the risk of overfitting. In this paper, the performance of GAEGA is assessed experimentally in comparison with other multiple classifier aggregation methods on 16 data sets. The experimental results demonstrate that GAEGA significantly outperforms the other methods in terms of average accuracy, ranging from 2.6% to 17.6%.