一种适用于小样本问题的基于边界的特征提取算法

特征提取技术是模式识别领域进行数据降维和强化判别信息的有效方法.线性判别分析是监督特征提取方法的典型代表,获得广泛应用,但受到小样本问题的制约.对此提出一种适用于小样本问题的基于边界的特征提取算法.算法利用高维数据小样本情况下线性可分概率增加以及其低维投影趋于正态分布的特点,定义了新的类别边界,不但考虑了由线性判别分析提出的类内、类间离散度,也兼顾各类别的方差差异性.通过极大化该边界获得最优投影向量,同时避免因类内离散度矩阵奇异导致的小样本问题.进一步将算法推广到多类问题.高光谱数据特征提取与分类实验表明,算法在小样本情况下对于两类和多类问题均具有良好的推广性能,优于多种线性判别分析的改进算法,并且在样本较多时也取得了满意结果.

A Margin Based Feature Extraction Algorithm for the Small Sample Size Problem

Feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information for the classification and recognition tasks.Linear Discriminant Analysis(LDA)is the most popular supervised method for feature extraction,but it often suffers the small sample size problem due to the singularity of the within-class scatter which arises if the number of samples is smaller than the dimensionality of samples.A margin based fea- ture extraction algorithm is proposed for the problem.In view of the facts that for the high-di- mensional data,the probability of linear separability may grow in case of small samples and the low-dimensional projection is approximately normal,the proposed algorithm introduces a new definition of the margin,which involves not only the between-class scatter and within-class scat- ter proposed by LDA criterion,but also the differences of the class variances.Through maximal- izing the margin,we can obtain the optimal projection vector,and avoid the small sample size problem.Through theoretical analysis,the algorithm is further extended to the multi-class case. The experiment results show that the algorithm outperforms several improved versions of LDA in the case of small samples.At the same time,a satisfying performance is also achieved for larger samples.