Locally nearest neighbor classifiers for pattern classification

In this paper, two novel classifiers based on locally nearest neighborhood rule, called nearest neighbor line and nearest neighbor plane, are presented for pattern classification. Comparison to nearest feature line and nearest feature plane, the proposed methods take much lower computation cost and achieve competitive performance.     

Center-based nearest neighbor classifier

In this paper, a novel center-based nearest neighbor (CNN) classifier is proposed to deal with the pattern classification problems. Unlike nearest feature line (NFL) method, CNN considers the line passing through a sample point with known label and the center of the sample class. This line is called the center-based line (CL). These lines seem to have more capacity of representation for sample classes than the original samples and thus can capture more information. Similar to NFL, CNN is based on the nearest distance from an unknown sample point to a certain CL for classification. As a result, the computation time of CNN can be shortened dramatically with less accuracy decrease when compared with NFL. The performance of CNN is demonstrated in one simulation experiment from computational biology and high classification accuracy has been achieved in the leave-one-out test. The comparisons with nearest neighbor (NN) classifier and NFL classifier indicate that this novel classifier achieves competitive performance.     

A novel classifier based on shortest feature line segment

A new approach called shortest feature line segment (SFLS) is proposed to implement pattern classification in this paper, which can retain the ideas and advantages of nearest feature line (NFL) and at the same time can counteract the drawbacks of NFL. The proposed SFLS uses the length of the feature line segment satisfying given geometric relation with query point instead of the perpendicular distance defined in NFL. SFLS has clear geometric-theoretic foundation and is relatively simple. Experimental results on some artificial datasets and real-world datasets are provided, together with the comparisons between SFLS and other neighborhood-based classification methods, including nearest neighbor (NN), k-NN, NFL and some refined NFL methods, etc. It can be concluded that SFLS is a simple yet effective classification approach.     

On the use of nearest feature line for speaker identification

As a new pattern classification method, nearest feature line (NFL) provides an effective way to tackle the sort of pattern recognition problems where only limited data are available for training. In this paper, we explore the use of NFL for speaker identification in terms of limited data and examine how the NFL performs in such a vexing problem of various mismatches between training and test. In order to speed up NFL in decision-making, we propose an alternative method for similarity measure. We have applied the improved NFL to speaker identification of different operating modes. Its text-dependent performance is better than the dynamic time warping (DTW) on the Ti46 corpus, while its computational load is much lower than that of DTW. Moreover, we propose an utterance partitioning strategy used in the NFL for better performance. For the text-independent mode, we employ the NFL to be a new similarity measure in vector quantization (VQ), which causes the VQ to perform better on the KING corpus. Some computational issues on the NFL are also discussed in this paper.     

RBF-based neurodynamic nearest neighbor classification in real pattern space

Superposition of radial basis functions centered at given prototype patterns constitutes one of the most suitable energy forms for gradient systems that perform nearest neighbor classification with real-valued static prototypes. It is shown in this paper that a continuous-time dynamical neural network model, employing a radial basis function and a sigmoid multi-layer perceptron sub-networks, is capable of maximizing such an energy form locally, thus performing almost perfectly nearest neighbor classification, when initiated by a distorted pattern. The proposed design scheme allows for explicit representation of prototype patterns as network parameters, as well as augmenting additional or forgetting existing memory patterns. The dynamical classification scheme implemented by the network eliminates all comparisons, which are the vital steps of the conventional nearest neighbor classification process. The performance of the proposed network model is demonstrated on binary and gray-scale image reconstruction applications.     

Comparison of the nearest feature classifiers for face recognition

This paper presents an experimental comparison of the nearest feature classifiers, using an approach based on binomial tests in order to evaluate their strengths and weaknesses. In addition, classification accuracies and the accuracy-dimensionality tradeoff have been considered as comparison criteria. We extend two of the nearest feature classifiers to label the query point by a majority vote of the samples. Comparisons were carried out for face recognition using ORL database. We apply the eigenface representation for feature extraction. Experimental results showed that even though the classification accuracy of k-NFP outperforms k-NFL in some dimensions, these rate differences do not have statistical significance.     

融合PLS监督特征提取和虚假最近邻点的数据分类特征选择

在高维数据分类中,针对多重共线性、冗余特征及噪声易导致分类器识别精度低和时空开销大的问题,提出融合偏最小二乘(Partial Least Squares,PLS)有监督特征提取和虚假最近邻点(False Nearest Neighbors,FNN)的特征选择方法:首先利用偏最小二乘对高维数据提取主元,消除特征之间的多重共线性,得到携带监督信息的独立主元空间;然后通过计算各特征选择前后在此空间的相关性,建立基于虚假最近邻点的特征相似性测度,得到原始特征对类别变量解释能力强弱排序;最后,依次剔除解释能力弱的特征,构造出各种分类模型,并以支持向量机(Support Vector Machine,SVM)分类识别率为模型评估准则,搜索出识别率最高但含特征数最少的分类模型,此模型所含的特征即为最佳特征子集。3个数据集模型仿真结果:均表明,由此法选择出的最佳特征子集与各数据集的本质分类特征吻合,说明该方法:有良好的特征选择能力,为数据分类特征选择提供了一条新途径。     

Partitioning of feature space for pattern classification

The article proposes a simple approach for finding a fuzzy partitioning of a feature space for pattern classification problems. A feature space is initially decomposed into some overlapping hyperboxes depending on the relative positions of the pattern classes found in the training samples. A few fuzzy if-then rules reflecting the pattern classes by the generated hyperboxes are then obtained in terms of a relational matrix. The relational matrix is utilized in the modified compositional rule of inference in order to recognize an unknown pattern. The proposed system is capable of handling imprecise information both in the learning and the processing phases. The imprecise information is considered to be either incomplete or mixed or interval or linguistic in form. Ways of handling such imprecise information are also discussed. The effectiveness of the system is demonstrated on some synthetic data sets in two-dimensional feature space. The practical applicability of the system is verified on four real data such as the Iris data set, an appendicitis data set, a speech data set and a hepatic disease data set.     

Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition

The polynomial classifier (PC) that takes the binomial terms of reduced subspace features as inputs has shown superior performance to multilayer neural networks in pattern classification. In this paper, we propose a class-specific feature polynomial classifier (CFPC) that extracts class-specific features from class-specific subspaces, unlike the ordinary PC that uses a class-independent subspace. The CFPC can be viewed as a hybrid of ordinary PC and projection distance method. The class-specific features better separate one class from the others, and the incorporation of class-specific projection distance further improves the separability. The connecting weights of CFPC are efficiently learned class-by-class to minimize the mean square error on training samples. To justify the promise of CFPC, we have conducted experiments of handwritten digit recognition and numeral string recognition on the NIST Special Database 19 (SD19). The digit recognition task was also benchmarked on two standard databases USPS and MNIST. The results show that the performance of CFPC is superior to that of ordinary PC, and is competitive with support vector classifiers (SVCs).     

Optimal reference subset selection for nearest neighbor classification by tabu search

This paper presents an approach to select the optimal reference subset (ORS) for nearest neighbor classifier. The optimal reference subset, which has minimum sample size and satisfies a certain resubstitution error rate threshold, is obtained through a tabu search (TS) algorithm. When the error rate threshold is set to zero, the algorithm obtains a near minimal consistent subset of a given training set. While the threshold is set to a small appropriate value, the obtained reference subset may have reasonably good generalization capacity. A neighborhood exploration method and an aspiration criterion are proposed to improve the efficiency of TS. Experimental results based on a number of typical data sets are presented and analyzed to illustrate the benefits of the proposed method. The performances of the result consistent and non-consistent reference subsets are evaluated.     

Wavelet-based feature extraction using probabilistic finite state automata for pattern classification

Real-time data-driven pattern classification requires extraction of relevant features from the observed time series as low-dimensional and yet information-rich representations of the underlying dynamics. These low-dimensional features facilitate in situ decision-making in diverse applications, such as computer vision, structural health monitoring, and robotics. Wavelet transforms of time series have been widely used for feature extraction owing to their time-frequency localization properties. In this regard, this paper presents a symbolic dynamics-based method to model surface images, generated by wavelet coefficients in the scale-shift space. These symbolic dynamics-based models (e.g., probabilistic finite state automata (PFSA)) capture the relevant information, embedded in the sensor data, from the associated Perron-Frobenius operators (i.e., the state-transition probability matrices). The proposed method of pattern classification has been experimentally validated on laboratory apparatuses for two different applications: (i) early detection of evolving damage in polycrystalline alloy structures, and (ii) classification of mobile robots and their motion profiles.     

An error-counting network for pattern classification

This paper presents a novel quadratic error-counting network for pattern classification. Two computational issues namely, the network learning issue and the classification error-counting issue have been addressed. Essentially, a linear series functional approximation to network structure and a smooth quadratic error-counting cost function were proposed to resolve these two computational issues within a single framework. Our analysis shows that the quadratic error-counting objective can be related to the least-squares-error objective by adjusting the class-specific normalization factors. The binary classification network is subsequently extended to cater for multicategory problems. An extensive empirical evaluation validates the usefulness of proposed method.     

使用概念基元特征进行自动文本分类

自动文本分类技术是大规模文档数据处理的关键技术,在文本分类过程中通常先进行文本表示,即把文本转化为特征向量,这其中常用的特征有特征词、词频、N-gram等等。论文研究了一种新的特征,即词语的HNC概念符号。词语的HNC概念符号来自于HNC(概念层次网络,HierarchicalNetworkofConcepts)建立的语义网络,以符号表达式的方式表示了词语的语义信息。因此使用HNC概念符号作为特征实际上是以文本中蕴含的语义信息作为特征,和词频等使用文本表层信息的特征有本质的不同。采用最大熵模型的方法建立分类器,以分词和HNC概念符号作为特征进行了研究,并对分类结果进行了比较。结果表明,HNC特征优于分词特征。     

One-against-all ensemble for multiclass pattern classification

This paper presents a new method called one-against-all ensemble for solving multiclass pattern classification problems. The proposed method incorporates a neural network ensemble into the one-against-all method to improve the generalization performance of the classifier. The experimental results show that the proposed method can reduce the uncertainty of the decision and it is comparable to the other widely used methods.     

Gaussian kernel optimization for pattern classification

This paper presents a novel algorithm to optimize the Gaussian kernel for pattern classification tasks, where it is desirable to have well-separated samples in the kernel feature space. We propose to optimize the Gaussian kernel parameters by maximizing a classical class separability criterion, and the problem is solved through a quasi-Newton algorithm by making use of a recently proposed decomposition of the objective criterion. The proposed method is evaluated on five data sets with two kernel-based learning algorithms. The experimental results indicate that it achieves the best overall classification performance, compared with three competing solutions. In particular, the proposed method provides a valuable kernel optimization solution in the severe small sample size scenario.     

线特征提取的多尺度分析

线特征提取是计算机视觉中重要的低级处理过程,而多尺度分析是采用微分几何方法进行线特征提取时一个重要内容。研究了在对不同宽度线特征进行检测时,尺度因子的选择问题,分析了变化的线宽与特定尺度因子间的关系,得到新的尺度因子确定方法。实验表明该方法简单、省时、有效。     

Combined pattern search optimization of feature extraction and classification parameters in facial recognition

Constantly, the assumption is made that there is an independent contribution of the individual feature extraction and classifier parameters to the recognition performance. In our approach, the problems of feature extraction and classifier design are viewed together as a single matter of estimating the optimal parameters from limited data. We propose, for the problem of facial recognition, a combination between an Interest Operator based feature extraction technique and a k-NN statistical classifier having the parameters determined using a pattern search based optimization technique. This approach enables us to achieve both higher classification accuracy and faster processing time.     

Filter-based models for pattern classification

In this paper we consider a technique for pattern classification based upon the development of prototypes which capture the distinguishing features (“disjunctive prototypes”) of each pattern class and, via cross-correlation with incoming test images, enable efficient pattern classification. We evaluate such a classification procedure with prototypes based on the images per se (direct code), Gabor scheme (multiple fixed filter representation) and an edge (scale space-based) coding scheme. Our analyses, and comparisons with human pattern classification performance, indicate that the edge-only disjunctive prototypes provide the most discriminating classification performance and are the more representative of human behaviour.     

EMG pattern classification using SOFMs for hand signal recognition

 We propose a method of pattern classification of electromyographic (EMG) signals using a set of self- organizing feature maps (SOFMs). The proposed method is simple to apply in that the EMG signals are directly input to the SOFMs without preprocessing. Experimental results are presented that show the effectiveness of the SOFM based classifier for the recognition of the hand signal version of the Korean alphabet from EMG signal patterns.