Forest optimization algorithm-based feature selection using classifier ensemble

Features selection is the process of choosing the relevant subset of features from the high-dimensional dataset to enhance the performance of the classifier. Much research has been carried out in the present world for the process of feature selection. Algorithms such as Naïve Bayes (NB), decision tree, and genetic algorithm are applied to the high-dimensional dataset to select the relevant features and also to increase the computational speed. The proposed model presents a solution for selection of features using ensemble classifier algorithms. The proposed algorithm is the combination of minimum redundancy and maximum relevance (mRMR) and forest optimization algorithm (FOA). Ensemble-based algorithms such as support vector machine (SVM), K-nearest neighbor (KNN), and NB is further used to enhance the performance of the classifier algorithm. The mRMR-FOA is used to select the relevant features from the various datasets and 21% to 24% improvement is recorded in the feature selection. The ensemble classifier algorithms further improves the performance of the algorithm and provides accuracy of 96%.     

基于卷积特征和贝叶斯分类器的人脸识别

为解决传统人脸识别算法特征提取困难的问题,提出了基于卷积特征和贝叶斯分类器的人脸识别方法,利用卷积神经网络提取人脸特征,通过主成分分析法对特征降维,最后利用贝叶斯分类器进行判别分类,在ORL（olivetti research laboratory）人脸库上进行实验,获得了99.00%的识别准确率。实验结果表明,卷积神经网络提取的人脸图像特征具有很强的辨识度,与PCA（principal component analysis）和贝叶斯分类器结合之后可有效提高人脸识别的准确率。     

Discriminative common vector approach based feature selection in face recognition

A novel feature selection algorithm is proposed, which is related to the Discriminative Common Vector Approach (DCVA) utilized as a means to reduce the computational complexity of the facial recognition problem. The recognition performance of the selected features is tested with DCVA and well known subspace methods over AR and YALE face databases. Moreover, the scheme indicates that important facial parts like eyes, eyebrows, noses, and lips must be kept for recognition purposes while eliminating the pixels in cheek, chin, and forehead areas. This additional knowledge comes out in the form of T-shaped and elliptical face masks used to specify the region of interest (ROI). Hence, besides the excellent dimensionality reduction given by the use of the DCVA technique, there is an intelligent use of the original database that provides superior results even in the presence of an occlusion as it is the case when the facial images have scarves.     

基于空间支持向量域分类器的人脸识别

提出了一种基于小波变换、奇异值分解与空间支持向量域分类器相结合的人脸识别方法。在使用空间支持向量分类器对不同人脸图像的奇异特征向量进行分类时,计算所测样本到各个超球球心的距离,并根据其与超球半径的关系来判断其所归属。并在ORL人脸数据库中进行实验。实验表明提出的人脸识别方法识别精度可达97.5%。     

Pattern representation in feature extraction and classifier design: matrix versus vector.

The matrix, as an extended pattern representation to the vector, has proven to be effective in feature extraction. However, the subsequent classifier following the matrix-pattern- oriented feature extraction is generally still based on the vector pattern representation (namely, MatFE + VecCD), where it has been demonstrated that the effectiveness in classification just attributes to the matrix representation in feature extraction. This paper looks at the possibility of applying the matrix pattern representation to both feature extraction and classifier design. To this end, we propose a so-called fully matrixized approach, i.e., the matrix-pattern-oriented feature extraction followed by the matrix-pattern-oriented classifier design (MatFE + MatCD). To more comprehensively validate MatFE + MatCD, we further consider all the possible combinations of feature extraction (FE) and classifier design (CD) on the basis of patterns represented by matrix and vector respectively, i.e., MatFE + MatCD, MatFE + VecCD, just the matrix-pattern-oriented classifier design (MatCD), the vector-pattern-oriented feature extraction followed by the matrix-pattern-oriented classifier design (VecFE + MatCD), the vector-pattern-oriented feature extraction followed by the vector-pattern-oriented classifier design (VecFE + VecCD) and just the vector-pattern-oriented classifier design (VecCD). The experiments on the combinations have shown the following: 1) the designed fully matrixized approach (MatFE + MatCD) has an effective and efficient performance on those patterns with the prior structural knowledge such as images; and 2) the matrix gives us an alternative feasible pattern representation in feature extraction and classifier designs, and meanwhile provides a necessary validation for "ugly duckling" and "no free lunch" theorems.     

An efficient fuzzy classifier with feature selection based on fuzzyentropy

This paper presents an efficient fuzzy classifier with the ability of feature selection based on a fuzzy entropy measure. Fuzzy entropy is employed to evaluate the information of pattern distribution in the pattern space. With this information, we can partition the pattern space into nonoverlapping decision regions for pattern classification. Since the decision regions do not overlap, both the complexity and computational load of the classifier are reduced and thus the training time and classification time are extremely short. Although the decision regions are partitioned into nonoverlapping subspaces, we can achieve good classification performance since the decision regions can be correctly determined via our proposed fuzzy entropy measure. In addition, we also investigate the use of fuzzy entropy to select relevant features. The feature selection procedure not only reduces the dimensionality of a problem but also discards noise-corrupted, redundant and unimportant features. Finally, we apply the proposed classifier to the Iris database and Wisconsin breast cancer database to evaluate the classification performance. Both of the results show that the proposed classifier can work well for the pattern classification application.     

Modified spider monkey optimization algorithm based feature selection and probabilistic neural network classifier in face recognition

This paper proposes a novel and robust predictive method using modified spider monkey optimization (MSMO) and probabilistic neural network (PNN) for face recognition. The limitation of the traditional spider monkey optimization (SMO) approach to obtaining an optimal solution for classification problems is overcome by enhancing the performance of SMO by modifying the perturbation rate with a non-linear function, thereby improving the convergence of SMO. The framework comprises image preprocessing, feature extraction using dual tree complex wavelet transform (DT-CWT), feature selection using the modified spider monkey optimization algorithm (MSMO), and classification using PNN. The proposed method is tested on the Yale and AR Face datasets. Experimental outcomes reveal that the proposed framework attain an accuracy of 99.4% with appreciable sensitivity, specificity, and G-mean. To examine the efficacy of MSMO, parametric studies are conducted, which showed that MSMO converges faster with high fitness when compared to similar evolutionary algorithms like Genetic Algorithm (GA), Grey Wolf Optimization Algorithm (GWO), Particle Swarm Algorithm (PSO), and Cuckoo Search (CS) in selecting the optimal feature set. The MSMO-PNN method outperforms similar state-of-the-art methods, which reveals that the method proposed is competitive. The proposed model is robust to Gaussian and salt–pepper noise, obtaining the highest accuracy of 97.89% for varied noise density and variance.     

基于动态部位特征的步态识别方法

为了在衣着饰物变化条件下进行步态识别,提出了一种基于动态部位特征的步态识别方法。首先,采用泊松方程给步态轮廓内的每个点赋值,并构造合适的阈值函数来提取步态序列的动态部位特征;然后,统计其等角度间隔的扇形区域内的均值和方差,用其构造动态特征向量;最后,利用支持向量机算法在行走人衣着饰物发生变化的条件下进行步态分类。通过在CASIA大规模步态数据库上的实验,验证了该方法的有效性和鲁棒性。     

基于局部特征与核低秩表示的人脸识别算法

针对人脸识别中的遮挡、伪装、光照及表情变化等问题,提出一种基于局部特征与核低秩表示的人脸识别算法。首先,对训练和测试的样本图片进行LBP特征的提取;然后将其通过映射函数投影到高维特征空间中进行后续操作,投影到高维空间中的特征矩阵通过降维处理后采用低秩表示的方法来提取样本之间的共同特征;最后根据低秩表示的结果进行分类识别。实验证明算法在对遮挡、伪装以及光照变化等噪声的影响鲁棒性更强,同时较当前的一些人脸识别算法的识别率也有了显著的提高。     

基于正则互表示的无监督特征选择方法

针对高维数据含有的冗余特征影响机器学习训练效率和泛化能力的问题,为提升模式识别准确率、降低计算复杂度,提出了一种基于正则互表示（RMR）性质的无监督特征选择方法。首先,利用特征之间的相关性,建立由Frobenius范数约束的无监督特征选择数学模型;然后,设计分治-岭回归优化算法对模型进行快速优化;最后,根据模型最优解综合评估每个特征的重要性,选出原始数据中具有代表性的特征子集。在聚类准确率指标上,RMR方法与Laplacian方法相比提升了7个百分点,与非负判别特征选择（NDFS）方法相比提升了7个百分点,与正则自表示（RSR）方法相比提升了6个百分点,与自表示特征选择（SR_FS）方法相比提升了3个百分点;在数据冗余率指标上,RMR方法与Laplacian方法相比降低了10个百分点,与NDFS方法相比降低了7个百分点,与RSR方法相比降低了3个百分点,与SR_FS方法相比降低了2个百分点。实验结果表明,RMR方法能够有效地选出重要特征,降低数据冗余率,提升样本聚类准确率。     

基于核稀疏表示的属性选择算法

为解决高维数据在分类时造成的“维数灾难”问题,提出一种新的将核函数与稀疏学习相结合的属性选择算法。具体地,首先将每一维属性利用核函数映射到核空间,在此高维核空间上执行线性属性选择,从而实现低维空间上的非线性属性选择;其次,对映射到核空间上的属性进行稀疏重构,得到原始数据集的一种稀疏表达方式;接着利用L 1范数构建属性评分选择机制,选出最优属性子集;最后,将属性选择后的数据用于分类实验。在公开数据集上的实验结果表明,该算法能够较好地实现属性选择,与对比算法相比分类准确率提高了约3%。     

基于稀疏图表示的特征选择方法研究

特征选择旨在降低待处理数据的维度,剔除冗余特征,是机器学习领域的关键问题之一。现有的半监督特征选择方法一般借助图模型提取数据集的聚类结构,但其所提取的聚类结构缺乏清晰的边界,影响了特征选择的效果。为此,提出一种基于稀疏图表示的半监督特征选择方法,构建了聚类结构和特征选择的联合学习模型,采用l__1范数约束图模型以得到清晰的聚类结构,并引入l_2,1范数以避免噪声的干扰并提高特征选择的准确度。为了验证本方法的有效性,选择了目前流行的几种特征方法进行对比分析,实验结果表明了本方法的有效性。     

Hand gesture recognition using two-level speed normalization,feature selection and classifier fusion

Hand gesture recognition provides an alternative way to many devices for human computer interaction. In this work, we have developed a classifier fusion based dynamic free-air hand gesture recognition system to identify the isolated gestures. Different users gesticulate at different speed for the same gesture. Hence, when comparing different samples of the same gesture, variations due to difference in gesturing speed should not contribute to the dissimilarity score. Thus, we have introduced a two-level speed normalization procedure using DTW and Euclidean distance-based techniques. Three features such as ‘orientation between consecutive points’, ‘speed’ and ‘orientation between first and every trajectory points’ were used for the speed normalization. Moreover, in feature extraction stage, 44 features were selected from the existing literatures. Use of total feature set could lead to overfitting, information redundancy and may increase the computational complexity due to higher dimension. Thus, we have tried to overcome this difficulty by selecting optimal set of features using analysis of variance and incremental feature selection techniques. The performance of the system was evaluated using this optimal set of features for different individual classifiers such as ANN, SVM, k-NN and Naïve Bayes. Finally, the decisions of the individual classifiers were combined using classifier fusion model. Based on the experimental results it may be concluded that classifier fusion provides satisfactory results compared to other individual classifiers. An accuracy of 94.78 % was achieved using the classifier fusion technique as compared to baseline CRF (85.07 %) and HCRF (89.91 %) models.     

From dynamic classifier selection to dynamic ensemble selection

In handwritten pattern recognition, the multiple classifier system has been shown to be useful for improving recognition rates. One of the most important tasks in optimizing a multiple classifier system is to select a group of adequate classifiers, known as an Ensemble of Classifiers (EoC), from a pool of classifiers. Static selection schemes select an EoC for all test patterns, and dynamic selection schemes select different classifiers for different test patterns. Nevertheless, it has been shown that traditional dynamic selection performs no better than static selection. We propose four new dynamic selection schemes which explore the properties of the oracle concept. Our results suggest that the proposed schemes, using the majority voting rule for combining classifiers, perform better than the static selection method.     

Color laser printer forensic based on noisy feature and support vector machine classifier

Digital forensics in the ubiquitous era can enhance and protect the reliability of multimedia content where this content is accessed, manipulated, and distributed using high quality computer devices. Color laser printer forensics is a kind of digital forensics which identifies the printing source of color printed materials such as fine arts, money, and document and helps to catch a criminal. This paper present a new color laser printer forensic algorithm based on noisy texture analysis and support vector machine classifier that can detect which color laser printer was used to print the unknown images. Since each printer vender uses their own printing process, printed documents from different venders have a little invisible difference looks like noise. In our identification scheme, the invisible noises are estimated with the wiener-filter and the 2D Discrete Wavelet Transform (DWT) filter. Then, a gray level co-occurrence matrix (GLCM) is calculated to analyze the texture of the noise. From the GLCM, 384 statistical features are extracted and applied to train and test the support vector machine classifier for identifying the color laser printers. In the experiment, a total of 4,800 images from 8 color laser printer models were used, where half of the image is for training and the other half is for classification. Results prove that the presented algorithm performs well by achieving 99.3%, 97.4% and 88.7% accuracy for the brand, toner and model identification respectively.     

基于特征向量的SAR图像目标识别方法研究

用于描述区域特征的Hu矩不变量在模式识别中得到广泛使用。然而在噪声影响下,尤其是SAR图像中严重的相干斑噪声,Hu 矩不变量不再保持其完美的性能。以Hu七个矩不变量为基础,结合SAR图像的特点,引入四个仿射矩不变量和SAR图像中目标区域的峰值、均值和方差系数,构成SAR图像中目标识别的特征向量。该特征向量体现了SAR图像区域目标的形状特征和区域的灰度信息。通过对两种不同分辨率下的T72坦克SAR图像的目标识别仿真实验,均获得了较好的目标识别效果,说明所选取的SAR图像目标识别的特征向量是有效的,具有较强的目标识别性能。     

Integrated classifier hyperplane placement and feature selection

The process of placing a separating hyperplane for data classification is normally disconnected from the process of selecting the features to use. An approach for feature selection that is conceptually simple but computationally explosive is to simply apply the hyperplane placement process to all possible subsets of features, selecting the smallest set of features that provides reasonable classification accuracy. Two ways to speed this process are (i) use a faster filtering criterion instead of a complete hyperplane placement, and (ii) use a greedy forward or backwards sequential selection method. This paper introduces a new filtering criterion that is very fast: maximizing the drop in the sum of infeasibilities in a linear-programming transformation of the problem. It also shows how the linear programming transformation can be applied to reduce the number of features after a separating hyperplane has already been placed while maintaining the separation that was originally induced by the hyperplane. Finally, a new and highly effective integrated method that simultaneously selects features while placing the separating hyperplane is introduced.     

Improving dynamic facial expression recognition with feature subset selection

This paper addresses the dynamic recognition of basic facial expressions in videos using feature subset selection. Feature selection has been already used by some static classifiers where the facial expression is recognized from one single image. Past work on dynamic facial expression recognition has emphasized the issues of feature extraction and classification, however, less attention has been given to the critical issue of feature selection in the dynamic scenario. The main contributions of the paper are as follows. First, we show that dynamic facial expression recognition can be casted into a classical classification problem. Second, we combine a facial dynamics extractor algorithm with a feature selection scheme for generic classifiers.We show that the paradigm of feature subset selection with a wrapper technique can improve the dynamic recognition of facial expressions. We provide evaluations of performance on real video sequences using five standard machine learning approaches: Support Vector Machines, K Nearest Neighbor, Naive Bayes, Bayesian Networks, and Classification Trees.