Effectiveness of error correctingoutput coding methods in ensemble and monolithic learningmachines

Abstract Error Correcting Output Coding (ECOC) methods formulticlass classification present several open problems rangingfrom the trade-off between their error recovering capabilitiesand the learnability of the induced dichotomies to the selectionof proper base learners and to the design of well-separatedcodes for a given multiclass problem. We experimentally analysesome of the main factors affecting the effectiveness of ECOCmethods. We show that the architecture of ECOC learning machinesinfluences the accuracy of the ECOC classifier, highlightingthat ensembles of parallel and independent dichotomicMulti-Layer Perceptrons are well-suited to implement ECOCmethods. We quantitatively evaluate the dependence amongcodeword bit errors using mutual information based measures,experimentally showing that a low dependence enhances thegeneralisation capabilities of ECOC. Moreover we show that theproper selection of the base learner and the decoding functionof the reconstruction stage significantly affects theperformance of the ECOC ensemble. The analysis of therelationships between the error recovering power, the accuracyof the base learners, and the dependence among codeword bitsshow that all these factors concur to the effectiveness of ECOCmethods in a not straightforward way, very likely dependent onthe distribution and complexity of the data.An erratum to this article can be found at