The theoretical analysis of FDA and applications

Representation and embedding are usually the two necessary phases in designing a classifier. Fisher discriminant analysis (FDA) is regarded as seeking a direction for which the projected samples are well separated. In this paper, we analyze FDA in terms of representation and embedding. The main contribution is that we prove that the general framework of FDA is based on the simplest and most intuitive FDA with zero within-class variance and therefore the mechanism of FDA is clearly illustrated. Based on our analysis, ε-insensitive SVM regression can be viewed as a soft FDA with ε-insensitive within-class variance and L₁ norm penalty. To verify this viewpoint, several real classification experiments are conducted to demonstrate that the performance of the regression-based classification technique is comparable to regular FDA and SVM.