Feature extraction for one-class classification problems: Enhancements to biased discriminant analysis

In many one-class classification problems such as face detection and object verification, the conventional linear discriminant analysis sometimes fails because it makes an inappropriate assumption on negative samples that they are distributed according to a Gaussian distribution. In addition, it sometimes cannot extract sufficient number of features because it merely makes use of the mean value of each class. In order to resolve these problems, in this paper, we extend the biased discriminant analysis (BDA) which was originally developed for one-class classification problems. The BDA makes no assumption on the distribution of negative samples and tries to separate each negative sample as far away from the center of positive samples as possible. The first extension uses a saturation technique to suppress the influence of the samples which are located far away from the decision boundary. The second one utilizes the L1 norm instead of the L2 norm. Also we present a method to extend BDA and its variants to multi-class classification problems. Our approach is considered useful in the sense that without much complexity, it successfully reduces the negative effect of negative samples which are far away from the center of positive samples, resulting in better classification performances. We have applied the proposed methods to several classification problems and compared the performance with conventional methods.     

Noise reduction and feature extraction based on low-rank representation and pairwise constraint preserving for hyperspectral images

ABSTRACT

Feature extraction (FE) methods play a central role in the classification of hyperspectral images (HSIs). However, all traditional FE methods work in original feature space (OFS), OFS may suffer from noise, outliers and poorly discriminative features. This paper presents a feature space enriching technique to address the problems of noise, outliers and poorly discriminative features which may exist in OFS. The proposed method is based on low-rank representation (LRR) with the capability of pairwise constraint preserving (PCP) termed LRR-PCP. LRR-PCP does not change the dimension of OFS and only can be used as an appropriate preprocessing procedure for any classification algorithm or DR methods. The proposed LRR-PCP aims to enrich the OFS and obtain extracted feature space (EFS) which results in features richer than OFS. The problems of noise and outliers can be decreased using LRR. But, LRR cannot preserve the intrinsic local structure of the original data and only capture the global structure of data. Therefore, two additional penalty terms are added into the objective function of LRR to keep the local discriminative ability and also preserve the data diversity. LRR-PCP method not only can be used in supervised learning but also in unsupervised and semi-supervised learning frameworks. The effectiveness of LRR-PCP is investigated on three HSI data sets using some existing DR methods and as a denoising procedure before the classification task. All experimental results and quantitative analysis demonstrate that applying LRR-PCP on OFS improves the performance of the classification and DR methods in supervised, unsupervised, and semi-supervised conditions.     

基于一类分类的时间序列异常值检测

论文结合相空间重构理论与一类分类方法提出一种时间序列中的异常值检测方法。该方法首先将时间序列映射到相空间,然后对相空间中的点实行一类分类,最后,根据KKT条件进行异常值检测。仿真实验结果表明了所给方法的可行性和有效性。     

An efficient orientation distance–based discriminative feature extraction method for multi-classification

Feature extraction is an important step before actual learning. Although many feature extraction methods have been proposed for clustering, classification and regression, very limited work has been done on multi-class classification problems. This paper proposes a novel feature extraction method, called orientation distance–based discriminative (ODD) feature extraction, particularly designed for multi-class classification problems. Our proposed method works in two steps. In the first step, we extend the Fisher Discriminant idea to determine an appropriate kernel function and map the input data with all classes into a feature space where the classes of the data are well separated. In the second step, we put forward two variants of ODD features, i.e., one-vs-all-based ODD and one-vs-one-based ODD features. We first construct hyper-plane (SVM) based on one-vs-all scheme or one-vs-one scheme in the feature space; we then extract one-vs-all-based or one-vs-one-based ODD features between a sample and each hyper-plane. These newly extracted ODD features are treated as the representative features and are thereafter used in the subsequent classification phase. Extensive experiments have been conducted to investigate the performance of one-vs-all-based and one-vs-one-based ODD features for multi-class classification. The statistical results show that the classification accuracy based on ODD features outperforms that of the state-of-the-art feature extraction methods.     

Discriminative Histogram Intersection Metric Learning and Its Applications

In this paper, a novel method called discriminative histogram intersection metric learning (DHIML) is proposed for pair matching and classification. Specifically, we introduce a discrimination term for learning a metric from binary information such as same/not-same or similar/dissimilar, and then combine it with the classification error for the discrimination in classifier construction. Compared with conventional approaches, the proposed method has several advantages. 1) The histogram intersection strategy is adopted into metric learning to deal with the widely used histogram features effectively. 2) By introducing discriminative term and classification error term into metric learning, a more discriminative distance metric and a classifier can be learned together. 3) The objective function is robust to outliers and noises for both features and labels in the training. The performance of the proposed method is tested on four applications: face verification, face-track identification, face-track clustering, and image classification. Evaluations on the challenging restricted protocol of Labeled Faces in the Wild (LFW) benchmark, a dataset with more than 7 000 face-tracks, and Caltech-101 dataset validate the robustness and discriminability of the proposed metric learning, compared with the recent state-of-the-art approaches.     

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

Identifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.     

A multi-one-class dynamic classifier for adaptive digitization of document streams

In this paper, we present a new dynamic classifier design based on a set of one-class independent SVM for image data stream categorization. Dynamic or continuous learning and classification has been recently investigated to deal with different situations, like online learning of fixed concepts, learning in non-stationary environments (concept drift) or learning from imbalanced data. Most of solutions are not able to deal at the same time with many of these specificities. Particularly, adding new concepts, merging or splitting concepts are most of the time considered as less important and are consequently less studied, whereas they present a high interest for stream-based document image classification. To deal with that kind of data, we explore a learning and classification scheme based on one-class SVM classifiers that we call mOC-iSVM (multi-one-class incremental SVM). Even if one-class classifiers are suffering from a lack of discriminative power, they have, as a counterpart, a lot of interesting properties coming from their independent modeling. The experiments presented in the paper show the theoretical feasibility on different benchmarks considering addition of new classes. Experiments also demonstrate that the mOC-iSVM model can be efficiently used for tasks dedicated to documents classification (by image quality and image content) in a context of streams, handling many typical scenarii for concepts extension, drift, split and merge.     

Unsupervised feature selection based on kernel fisher discriminant analysis and regression learning

In this paper, we propose a new feature selection method called kernel fisher discriminant analysis and regression learning based algorithm for unsupervised feature selection. The existing feature selection methods are based on either manifold learning or discriminative techniques, each of which has some shortcomings. Although some studies show the advantages of two-steps method benefiting from both manifold learning and discriminative techniques, a joint formulation has been shown to be more efficient. To do so, we construct a global discriminant objective term of a clustering framework based on the kernel method. We add another term of regression learning into the objective function, which can impose the optimization to select a low-dimensional representation of the original dataset. We use L_2,1-norm of the features to impose a sparse structure upon features, which can result in more discriminative features. We propose an algorithm to solve the optimization problem introduced in this paper. We further discuss convergence, parameter sensitivity, computational complexity, as well as the clustering and classification accuracy of the proposed algorithm. In order to demonstrate the effectiveness of the proposed algorithm, we perform a set of experiments with different available datasets. The results obtained by the proposed algorithm are compared against the state-of-the-art algorithms. These results show that our method outperforms the existing state-of-the-art methods in many cases on different datasets, but the improved performance comes with the cost of increased time complexity.

     

基于小波特征和多类支持向量机的病态语音识别方法

研究一种应用小波特征向量和多类支持向量机进行病态语音识别的方法,该方法基于连续小波变换提取语音特征向量,利用多类支持向量机进行病态语音分类。为了简化二分类支持向量机进行多类分类时所带来的计算复杂性,根据一类支持向量机分类思想提出一种多类分类算法。该算法能够使每一类样本都独立地获得一个决策函数,通过决策函数的最大值来判断样本所属的类。实验表明,在病态语音识别系统中,多类支持向量机与小波特征向量相结合具有良好的识别效果和应用价值。     

Covariance-guided landmine detection and discrimination using ground-penetrating radar data

Ground penetrating Radar (GPR) can detect and deliver the response signal from any buried kind of object like plastic or metallic landmines, stones, and wood sticks. It delivers three kinds of data: Ascan, Bscan, and Cscan. However, it cannot discriminate between landmines and inoffensive objects or ‘clutter.’ One-class classification is an alternative to detect landmines, especially, as landmines features data are unbalanced. In this article, we investigate the effectiveness of the Covariance-guided One-Class Support Vector Machine (COSVM) to detect, discriminate, and locate landmines efficiently. In fact, compared to existing one-class classifiers, the COSVM has the advantage of emphasizing low variance directions. Moreover, we will compare the one-class classification to multiclass classification to tease out the advantage of the former over the latter as data are unbalanced. Our method consists of extracting Ascan GPR data. Extracted features are used as an input for COSVM to discriminate between landmines and clutter. We provide an extensive evaluation of our detection method compared to other methods based on relevant state of the art one-class and multiclass classifiers, on the well-known MACADAM database. Our experimental results show clearly the superiority of using COSVM in landmine detection and localization.     

An evaluation of dimension reduction techniques for one-class classification

Dimension reduction (DR) is important in the processing of data in domains such as multimedia or bioinformatics because such data can be of very high dimension. Dimension reduction in a supervised learning context is a well posed problem in that there is a clear objective of discovering a reduced representation of the data where the classes are well separated. By contrast DR in an unsupervised context is ill posed in that the overall objective is less clear. Nevertheless successful unsupervised DR techniques such as principal component analysis (PCA) exist—PCA has the pragmatic objective of transforming the data into a reduced number of dimensions that still captures most of the variation in the data. While one-class classification falls somewhere between the supervised and unsupervised learning categories, supervised DR techniques appear not to be applicable at all for one-class classification because of the absence of a second class label in the training data. In this paper we evaluate the use of a number of up-to-date unsupervised DR techniques for one-class classification and we show that techniques based on cluster coherence and locality preservation are effective.     

Pulse-coupled neural networks and one-class support vector machines for geometry invariant texture retrieval

The pulse-coupled neural network (PCNN) has been widely used in image processing. The outputs of PCNN represent unique features of original stimulus and are invariant to translation, rotation, scaling and distortion, which is particularly suitable for feature extraction. In this paper, PCNN and intersecting cortical model (ICM), which is a simplified version of PCNN model, are applied to extract geometrical changes of rotation and scale invariant texture features, then an one-class support vector machine based classification method is employed to train and predict the features. The experimental results show that the pulse features outperform of the classic Gabor features in aspects of both feature extraction time and retrieval accuracy, and the proposed one-class support vector machine based retrieval system is more accurate and robust to geometrical changes than the traditional Euclidean distance based system.     

基于一类分类的聚类方法及其应用

在分析一类分类方法最新研究成果的基础上,提出了基于一类分类的模式聚类方法,通过与传统聚类方法的比较,说明该方法对非线性数据处理聚类的优越性,并以某企业对供应商关系调查数据为例,将这种方法应用于企业供应商关系分析中,聚类结果表明了该方法的有效性,为企业与其它组织之间的复杂关系网络的特征数据分析提供了一种可行的方法。     

基于遗传算法和一类SVM的隐秘图像检测方案

针对二类支持向量机分类器在隐秘图像检测中训练步骤复杂与推广性弱的缺点,提出了一种新的基于遗传算法和一类支持向量机的隐秘图像检测方案。采用遗传算法进行图像特征选择,一类支持向量机作为分类器。实验结果表明,与只利用一类支持向量机分类,但未进行特征选择的隐秘检测方法相比,提高了隐秘图像检测的识别率和系统检测效率。     

结合动态自适应调制和结构关系学习的细粒度图像分类

由于细粒度图像类间差异小, 类内差异大的特点, 因此细粒度图像分类任务关键在于寻找类别间细微差异. 最近, 基于Vision Transformer的网络大多侧重挖掘图像最显著判别区域特征. 这存在两个问题: 首先, 网络忽略从其他判别区域挖掘分类线索, 容易混淆相似类别; 其次, 忽略了图像的结构关系, 导致提取的类别特征不准确. 为解决上述问题, 本文提出动态自适应调制和结构关系学习两个模块, 通过动态自适应调制模块迫使网络寻找多个判别区域, 再利用结构关系学习模块构建判别区域间结构关系; 最后利用图卷积网络融合语义信息和结构信息得出预测分类结果. 所提出的方法在CUB-200-2011数据集和NA-Birds数据集上测试准确率分别达到92.9%和93.0%, 优于现有最先进网络.     

Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database

Despite extensive studies for the industrial applications of deep learning, its actual usage in manufacturing sites has been extremely restrained by the difficulty in obtaining sufficient industrial data, especially for production failure cases. In this study, we introduced a fault-detection module based on one-class deep learning for imbalanced industrial time-series data, which consists of three submodules, namely, time-series prediction based on deep learning, residual calculation, and one-class classification using one-class support vector machine and isolation forest. Four different networks were used for the time-series prediction: multilayer perception (MLP), residual network (ResNet), long–short-term memory (LSTM), and ResNet–LSTM, each trained with the one-class data having only the production success cases. We adopted the residuals of the deep-learning prediction as an elaborated feature for the construction of the one-class classification. We also tested the fault-detection module with the actual mass production data of a die-casting process. By adopting the features elaborated by the deep-learning time-series prediction, we showed that the total accuracy of the one-class classification significantly improved from 90.0% to 96.0%. Especially for its capability to detect production failures, the accuracy improved from 84.0% to 96.0%. The area under the receiver operating characteristics (AUROC) also improved from 87.56% to 98.96%. ResNet showed the best performance for detecting production failures, whereas ResNet–LSTM produced better results for ensuring the production success. Our results suggest that the one-class deep learning is a promising approach for extracting important features from time-series data to realize a one-class fault-detection module.     

基于监督判别局部保持投影的表情识别算法

LPP算法是无监督算法,并没有考虑到不同类别的样本对分类效果的影响,结果会造成不同类数据点的重叠,故所获得的子空间对于分类问题来说未必是最优的。提出一种新的基于监督判别局部保持投影（SDLPP）的表情识别算法。利用样本的类别信息重新构造LPP算法中的相似矩阵,然后在目标函数中增加类间散度约束,这样就会在保持样本点局部结构的同时,使不同类的样本点相互远离,从而得到更具有判别性的表情特征。该算法在识别率上比其他方法都有较大提高,通过在JAFFE表情库上的实验验证了其有效性。