Soft Margins for AdaBoost

Recently ensemble methods like ADABOOST have been applied successfully in many problems, while seemingly defying the problems of overfitting.ADABOOST rarely overfits in the low noise regime, however, we show that it clearly does so for higher noise levels. Central to the understanding of this fact is the margin distribution. ADABOOST can be viewed as a constraint gradient descent in an error function with respect to the margin. We find that ADABOOST asymptotically achieves a hard margin distribution, i.e. the algorithm concentrates its resources on a few hard-to-learn patterns that are interestingly very similar to Support Vectors. A hard margin is clearly a sub-optimal strategy in the noisy case, and regularization, in our case a mistrust in the data, must be introduced in the algorithm to alleviate the distortions that single difficult patterns (e.g. outliers) can cause to the margin distribution. We propose several regularization methods and generalizations of the original ADABOOST algorithm to achieve a soft margin. In particular we suggest (1) regularized ADABOOST_REG where the gradient decent is done directly with respect to the soft margin and (2) regularized linear and quadratic programming (LP/QP-) ADABOOST, where the soft margin is attained by introducing slack variables.Extensive simulations demonstrate that the proposed regularized ADABOOST-type algorithms are useful and yield competitive results for noisy data.     

LBP-自适应增强模型的木材纹理分类

针对传统木材纹理分类的准确率低且难度大的问题,依据LBP(局部二值)算子和ADABOOST(自适应增强)算法理论,提出了LBP-ADABOOST模型对木材纹理进行识别分类.通过均匀旋转不变特性与原始LBP算子相融合,提取纹理的特征值,结合自适应增强算法,从而训练得到每类纹理所对应的分类器模型参数,构造分类器,实现对木材纹理准确高效分类.实验结果表明相比于BP神经网络,SVM支持向量机等分类算法,该模型的实验结果误差率为4%左右,准确率高,实用性强.