An optimum feature extraction method for texture classification

Texture can be defined as a local statistical pattern of texture primitives in observer’s domain of interest. Texture classification aims to assign texture labels to unknown textures, according to training samples and classification rules. In this paper a novel method, which is an intelligent system for texture classification is introduced. It used a combination of genetic algorithm, discrete wavelet transform and neural network for optimum feature extraction from texture images. An algorithm called the intelligent system, which processes the pattern recognition approximation, is developed. We tested the proposed method with several texture images. The overall success rate is about 95%.     

An MRF-based kernel method for nonlinear feature extraction

Nonlinear kernel-based feature extraction algorithms have recently been proposed to alleviate the loss of class discrimination after feature extraction. When considering image classification, a kernel function may not be sufficiently effective if it depends only on an information resource from the Euclidean distance in the original feature space. This study presents an extended radial basis kernel function that integrates multiple discriminative information resources, including the Euclidean distance, spatial context, and class membership. The concepts related to Markov random fields (MRFs) are exploited to model the spatial context information existing in the image. Mutual closeness in class membership is defined as a similarity measure with respect to classification. Any dissimilarity from the additional information resources will improve the discrimination between two samples that are only a short Euclidean distance apart in the feature space. The proposed kernel function is used for feature extraction through linear discriminant analysis (LDA) and principal component analysis (PCA). Experiments with synthetic and natural images show the effectiveness of the proposed kernel function with application to image classification.     

An entropy fusion method for feature extraction of EEG

Feature extraction is a vital part in EEG classification. Among the various feature extraction methods, entropy reflects the complexity of the signal. Different entropies reflect the characteristics of the signal from different views. In this paper, we propose a feature extraction method using the fusion of different entropies. The fusion can be a more complete expression of the characteristic of EEG. Four entropies, namely a measure for amplitude based on Shannon entropy, a measure for phase synchronization based on Shannon entropy, wavelet entropy and sample entropy, are firstly extracted from the collected EEG signals. Support vector machine and principal component analysis are then used for classification and dimensionality reduction, respectively. We employ BCI competition 2003 dataset III to evaluate the method. The experimental results show that our method based on four entropies fusion can achieve better classification performance, and the accuracy approximately reaches 88.36 %. Finally, it comes to the conclusion that our method has achieved good performance for feature extraction in EEG classification.

     

A new method for unsupervised linear feature extraction, using fourth-order moments

In this paper we investigate an unsupervised linear feature extraction system based on the second- and fourth-order moments of the input covariance function.

First we demonstrate some drawbacks of systems based on second-order moments (including all Karhunen-Loéve systems). Then we introduce a new version of our Learning filter system that is based on fourth-order moments. We show that this system has the same modular structure as our previous (second-order) systems. Then we discuss some advantages of this new system with the help of some examples. In the first example we demonstrate that this new system can detect meaningful structures even in those subspaces that belong to the same eigenvalue of the covariance matrix. Systems based on second-order moments are unable to find such structures. In the second example we train the system with two pattern sequences that have exactly the same second-order moments. We show that the system stabilizes indeed in two different states that reflect relevant properties of the whole trainings sequence. In the last example we demonstrate that the feature vectors produced by the new system are better adapted to a special type nonlinear post-processing that is motivated by the group-theoretically based filter design methodology.     

Adaptive linear discriminant analysis for online feature extraction

In this paper, we present new adaptive algorithms for the computation of the square root of the inverse covariance matrix. In contrast to the current similar methods, these new algorithms are obtained from an explicit cost function that is introduced for the first time. The new adaptive algorithms are used in a cascade form with a well-known adaptive principal component analysis to construct linear discriminant features. The adaptive nature and fast convergence rate of the new adaptive linear discriminant analysis algorithms make them appropriate for online pattern recognition applications. All adaptive algorithms discussed in this paper are trained simultaneously using a sequence of random data. Experimental results using the synthetic and real multiclass, multidimensional input data demonstrate the effectiveness of the new adaptive algorithms to extract the optimal features for the purpose of classification.     

An algorithm for optimal single linear feature extraction from several gaussian pattern classes

A computational algorithm is presented for the extraction of an optimal single linear feature from several Gaussian pattern classes. The algorithm minimizes the increase in the probability of misclassification in the transformed (feature) space. The general approach used in this procedure was developed in a recent paper by R. J. P. de Figueiredo.⁽¹⁾ Numerical results on the application of this procedure to the remotely sensed data from the Purdue C1 flight line as well asLandsat data are presented. It was found that classification using the optimal single linear feature yielded a value for the probability of misclassification on the order of 30% less than that obtained by using the best single untransformed feature. The optimal single linear feature gave performance results comparable to those obtained by using the two features which maximized the average divergence. Also discussed are improvements in classification results using this method when the size of the training set is small.This work was supported by the Air Force Office of Scientific Research under Grant 75-2777 and by the National Aeronautics and Space Administration under contract NAS 9-12776.     

A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm

This paper presents a novel wrapper feature selection algorithm for classification problems, namely hybrid genetic algorithm (GA)- and extreme learning machine (ELM)-based feature selection algorithm (HGEFS). It utilizes GA to wrap ELM to search for the optimum subsets in the huge feature space, and then, a set of subsets are selected to make ensemble to improve the final prediction accuracy. To prevent GA from being trapped in the local optimum, we propose a novel and efficient mechanism specifically designed for feature selection problems to maintain GA’s diversity. To measure each subset’s quality fairly and efficiently, we adopt a modified ELM called error-minimized extreme learning machine (EM-ELM) which automatically determines an appropriate network architecture for each feature subsets. Moreover, EM-ELM has good generalization ability and extreme learning speed which allows us to perform wrapper feature selection processes in an affordable time. In other words, we simultaneously optimize feature subset and classifiers’ parameters. After finishing the search process of GA, to further promote the prediction accuracy and get a stable result, we select a set of EM-ELMs from the obtained population to make the final ensemble according to a specific ranking and selecting strategy. To verify the performance of HGEFS, empirical comparisons are carried out on different feature selection methods and HGEFS with benchmark datasets. The results reveal that HGEFS is a useful method for feature selection problems and always outperforms other algorithms in comparison.     

A linear feature extraction for multiclass classification problems based on class mean and covariance discriminant information

A parametric linear feature extraction method is proposed for multiclass classification. The skeleton of the proposed method consists of two types of schemes that are complementary to each other with regard to the discriminant information used. The approximate pairwise accuracy criterion (aPAC) and the common-mean feature extraction (CMFE) are chosen to exploit the discriminant information about class mean and about class covariance, respectively. Choosing aPAC rather than the linear discriminant analysis (LDA) can also resolve the problem of overemphasized large distances introduced by LDA, while maintaining other decent properties of LDA. To alleviate the suboptimum problem caused by a direct cascading of the two different types of schemes, there should be a mechanism for sorting and merging features based on their effectiveness. Usage of a sample-based classification error estimation for evaluation of effectiveness of features usually costs a lot of computational time. Therefore, we develop a fast spanning-tree-based parametric classification accuracy estimator as an intermediary for the aPAC and CMFE combination. The entire framework is parametric-based. This avoids paying a costly price in computation, which normally happens to the sample-based approach. Our experiments have shown that the proposed method can achieve a satisfactory performance on real data as well as simulated data.     

Fuzzy integral-based perceptron for two-class pattern classification problems

The single-layer perceptron with single output node is a well-known neural network for two-class classification problems. Furthermore, the sigmoid or logistic function is usually used as the activation function in the output neuron. A critical step is to compute the sum of the products of the connection weights with the corresponding inputs, which indicates the assumption of additivity among individual variables. Unfortunately, because the input variables are not always independent of each other, an assumption of additivity may not be reasonable enough. In this paper, the inner product can be replaced with an aggregation value obtained by a useful fuzzy integral by viewing each of the connection weights as a value of a λ-fuzzy measure for the corresponding variable. A genetic algorithm is then employed to obtain connection weights by maximizing the number of correctly classified training patterns and minimizing the errors between the actual and desired outputs of individual training patterns. The experimental results further demonstrate that the proposed method outperforms the traditional single-layer perceptron and performs well in comparison with other fuzzy or non-fuzzy classification methods.     

Generalized mean for feature extraction in one-class classification problems

Biased discriminant analysis (BDA), which extracts discriminative features for one-class classification problems, is sensitive to outliers in negative samples. This study focuses on the drawback of BDA attributed to the objective function based on the arithmetic mean in one-class classification problems, and proposes an objective function based on a generalized mean. A novel method is also presented to effectively maximize the objective function. The experimental results show that the proposed method provides better discriminative features than the BDA and its variants.     

Tensor linear Laplacian discrimination (TLLD) for feature extraction

Discriminant feature extraction plays a central role in pattern recognition and classification. In this paper, we propose the tensor linear Laplacian discrimination (TLLD) algorithm for extracting discriminant features from tensor data. TLLD is an extension of linear discriminant analysis (LDA) and linear Laplacian discrimination (LLD) in directions of both nonlinear subspace learning and tensor representation. Based on the contextual distance, the weights for the within-class scatters and the between-class scatter can be determined to capture the principal structure of data clusters. This makes TLLD free from the metric of the sample space, which may not be known. Moreover, unlike LLD, the parameter tuning of TLLD is very easy. Experimental results on face recognition, texture classification and handwritten digit recognition show that TLLD is effective in extracting discriminative features.     

Maximization of mutual information for supervised linear feature extraction.

In this paper, we present a novel scheme for linear feature extraction in classification. The method is based on the maximization of the mutual information (MI) between the features extracted and the classes. The sum of the MI corresponding to each of the features is taken as an heuristic that approximates the MI of the whole output vector. Then, a component-by-component gradient-ascent method is proposed for the maximization of the MI, similar to the gradient-based entropy optimization used in independent component analysis (ICA). The simulation results show that not only is the method competitive when compared to existing supervised feature extraction methods in all cases studied, but it also remarkably outperform them when the data are characterized by strongly nonlinear boundaries between classes.     

A new feature extraction method for odour classification

A new method for real time classification of volatile chemical substance traces is presented. The method is based on electrochemical signals of an array of semiconductor gas sensors. In these sensor signals characteristic patterns of different substances are hidden. There are non-linear correlative relationships between the measured sensor signals and the chemical substances which are treated using two methods derived from statistical learning theory (Support Vector Machine - SVM, Maximum Likelihood Estimation - MLE) for the detection of the substance characteristics in the sensor signals. A key criterion for the presented pattern recognition is a newly developed type of features, which is specially adapted to the low frequency signals of semiconductor sensors. The presented features are based on the evaluation of the range of the transient response in the sensor signals in the frequency domain.To derive the new features, both real measurement data and synthetic generated signals were used. In the experiments the focus was set on the creation of reproducible sensor signals to get characteristic signal patterns. Synthetic signals were derived from a Gaussian Plume Model. With the new features, training data sets were calculated using the classification methods SVM and MLE. With these training data sets new sensor measurements may be assigned to the substances which are to be sought. The advantage of the presented method is that no feature reduction is needed and no loss of information occurs in the learning process.The classification results based on the new features have been compared with the classification based on a conventional method for feature extraction. It was proved that the recognition rate of the substances used with the new feature type is higher.The substance classification is primarily limited by the sensitivity of the semiconductor sensors, because sufficiently large sensor signals must have been provided to obtain appropriate substance patterns. At the present stage of development the method presented is suitable for the classification of substance groups, such as nitro aromatics or alcohols, but not for specific substances.     

新的非线性鉴别特征抽取方法及人脸识别

在非线性空间中采用新的最大散度差鉴别准则,提出了一种新的核最大散度差鉴别分析方法.该方法不仅有效地抽取了人脸图像的非线性鉴别特征,而且从根本上避免了以往核Fisher鉴别分析中训练样本总数较多时,通常存在的核散布矩阵奇异的问题,计算复杂度大大降低,识别速度有了明显的提高.在ORL人脸数据库上的实验结果验证了该算法的有效性.     

Heteroscedastic linear feature extraction based on sufficiency conditions

Classification of high-dimensional data typically requires extraction of discriminant features. This paper proposes a linear feature extractor, called whitened linear sufficient statistic (WLSS), which is based on the sufficiency conditions for heteroscedastic Gaussian distributions. WLSS approximates, in the least squares sense, an operator providing a sufficient statistic. The proposed method retains covariance discriminance in heteroscedastic data, while it reduces to the commonly used linear discriminant analysis (LDA) in the homoscedastic case. Compared to similar heteroscedastic methods, WLSS imposes a low computational complexity, and is highly generalizable as confirmed by its consistent competence over various data sets.     

Scaling low-resolution bitmap fonts using linear feature extraction

In certain cases a font, as defined by a set of small raster images, can be scaled by first extracting the linear features that define each character. These can be treated as graphical vectors and scaled, translated, and rotated in an arbitrary fashion. The visual precision of this method depends on the size of the original front templates.     

A multi-manifold discriminant analysis method for image feature extraction

In this paper, we propose a Multi-Manifold Discriminant Analysis (MMDA) method for an image feature extraction and pattern recognition based on graph embedded learning and under the Fisher discriminant analysis framework. In an MMDA, the within-class graph and between-class graph are, respectively, designed to characterize the within-class compactness and the between-class separability, seeking for the discriminant matrix to simultaneously maximize the between-class scatter and minimize the within-class scatter. In addition, in an MMDA, the within-class graph can represent the sub-manifold information, while the between-class graph can represent the multi-manifold information. The proposed MMDA is extensively examined by using the FERET, AR and ORL face databases, and the PolyU finger-knuckle-print databases. The experimental results demonstrate that an MMDA is effective in feature extraction, leading to promising image recognition performance.     

New method for feature extraction based on fractal behavior

In this paper, a novel approach to feature extraction based on fractal theory is presented as a powerful technique in pattern recognition. This paper presents a new fractal feature that can be applied to extract the feature of two-dimensional objects. It is constructed by a hybrid feature extraction combining wavelet analysis, central projection transformation and fractal theory. New fractal feature and fractal signatures are reported. A multiresolution family of the wavelets is also used to compute information conserving micro-features. We employed a central projection method to reduce the dimensionality of the original input pattern. A wavelet transformation technique to transform the derived pattern into a set of sub-patterns. Its fractal dimension can readily be computed, and to use the fractal dimension as the feature vectors. Moreover, a modified fractal signature is also used to distinguish the distinct handwritten signatures. We expect that the proposed fractal method can also be used for improving the extraction and classification of features in pattern recognition.