Center-based nearest neighbor classifier

In this paper, a novel center-based nearest neighbor (CNN) classifier is proposed to deal with the pattern classification problems. Unlike nearest feature line (NFL) method, CNN considers the line passing through a sample point with known label and the center of the sample class. This line is called the center-based line (CL). These lines seem to have more capacity of representation for sample classes than the original samples and thus can capture more information. Similar to NFL, CNN is based on the nearest distance from an unknown sample point to a certain CL for classification. As a result, the computation time of CNN can be shortened dramatically with less accuracy decrease when compared with NFL. The performance of CNN is demonstrated in one simulation experiment from computational biology and high classification accuracy has been achieved in the leave-one-out test. The comparisons with nearest neighbor (NN) classifier and NFL classifier indicate that this novel classifier achieves competitive performance.     

The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier

We present the Nearest Subclass Classifier (NSC), which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier. The algorithm is based on the Maximum Variance Cluster algorithm and, as such, it belongs to the class of prototype-based classifiers. The variance constraint parameter of the cluster algorithm serves to regularize the classifier, that is, to prevent overfitting. With a low variance constraint value, the classifier turns into the nearest neighbor classifier and, with a high variance parameter, it becomes the nearest mean classifier with the respective properties. In other words, the number of prototypes ranges from the whole training set to only one per class. In the experiments, we compared the NSC with regard to its performance and data set compression ratio to several other prototype-based methods. On several data sets, the NSC performed similarly to the k-nearest neighbor classifier, which is a well-established classifier in many domains. Also concerning storage requirements and classification speed, the NSC has favorable properties, so it gives a good compromise between classification performance and efficiency.     

A bootstrap technique for nearest neighbor classifier design

A bootstrap technique for nearest neighbor classifier design is proposed. Our primary interest in designing a classifier is in small training sample size situations. Conventional bootstrapping techniques sample the training samples with replacement. On the other hand, our technique generates bootstrap samples by locally combining original training samples. The nearest neighbor classifier is designed on the bootstrap samples and is tested on the test samples independent of training samples. The performance of the proposed classifier is demonstrated on three artificial data sets and one real data set. Experimental results show that the nearest neighbor classifier designed on the bootstrap samples outperforms the conventional k-NN classifiers as well as the edited 1-NN classifiers, particularly in high dimensions     

Overlap pattern synthesis with an efficient nearest neighbor classifier

Nearest neighbor (NN) classifier is the most popular non-parametric classifier. It is a simple classifier with no design phase and shows good performance. Important factors affecting the efficiency and performance of NN classifier are (i) memory required to store the training set, (ii) classification time required to search the nearest neighbor of a given test pattern, and (iii) due to the curse of dimensionality the number of training patterns needed by it to achieve a given classification accuracy becomes prohibitively large when the dimensionality of the data is high. In this paper, we propose novel techniques to improve the performance of NN classifier and at the same time to reduce its computational burden. These techniques are broadly based on: (i) overlap based pattern synthesis which can generate a larger number of artificial patterns than the number of input patterns and thus can reduce the curse of dimensionality effect, (ii) a compact representation of the given set of training patterns called overlap pattern graph (OLP-graph) which can be incrementally built by scanning the training set only once and (iii) an efficient NN classifier called OLP-NNC which directly works with OLP-graph and does implicit overlap based pattern synthesis. A comparison based on experimental results is given between some of the relevant classifiers. The proposed schemes are suitable for applications dealing with large and high dimensional datasets like those in data mining.     

A new nearest neighbor classification method based on fuzzy set theory and aggregation operators

The Fuzzy Nearest Neighbor Classification (FuzzyNNC) has been successfully used, as a tool to deal with supervised classification problems. It has significantly increased the classification accuracy by considering the uncertainty associated with the class labels of the training patterns. Nevertheless, FuzzyNNC's limited methods fail to efficiently handle the imprecision in features measurement and the uncertainty induced by the choice of the distance measure and the number of neighbors in the decision rule. In this paper, we propose a new method called Fuzzy Analogy-based Classification (FABC) to tackle the FuzzyNNC limitations. In this work, we exploit the fuzzy linguistic modeling and approximate reasoning materials in order to endow FABC with intelligent capabilities, like imprecision tolerance, optimization, adaptability and trade-off. Hence, our approach is composed of two main steps. Firstly, we describe the domain features using fuzzy linguistic variables. Secondly, we define the classification process using two intelligent aggregation operators. The first one allows the optimization of the similarity evaluation, by defining the adequate features to be considered. The second one integrates a trade-off strategy within the decision rule, by using a global voting approach with compensation property. The integration of such mechanisms will increase the classification accuracy and make the FuzzyNNC approach more useful for classification problems where imprecision and uncertainty are unavoidable. The proposed FABC is validated on the most known datasets, representing various classification difficulties and compared to the many extensions of the FuzzyNNC approach. The results obtained show that our proposed FABC method can be adapted to different classification problems and improve the classification accuracy. Thus, the FABC has the best rank value against the comparison methods with high significant level. Moreover, we conclude that our optimized similarity and global voting rule are more robust to handle the uncertainty in the classification process than those used by the comparison methods.     

基于小波和最近邻凸包分类器的人脸识别

本文提出一种新型的人脸识别方法.该方法首先通过二维小波变换提取人脸图像的低频特征,然后采用最近邻凸包分类器对该特征进行分类.二维小波变换是提取图像特征的有效方法之一,在保留原始图像的主要特征的同时,还能够有效降低图像维数;最近邻凸包分类器是一种以测试样本点到各类别训练样本凸包的距离作为相似性度量的分类算法.本文将这两项技术相结合在ORL人脸识别数据库上取得了良好的实验效果.     

基于自然最近邻居的社团检测算法

针对传统社团检测算法无法判断网络中特殊节点和SCAN算法对于参数依赖性太大的缺点,提出了一种基于自然最近邻居概念的社团检测算法CD3N.算法利用自然最近邻居无参的特性,首先以结构相似度为基准,计算出网络节点的自然最近邻居,并依此构造小值最近邻域图;然后取邻域图中邻居数最多的节点为核心节点,根据可达关系,构造关于核心节点的社团;重复选取核心节点并构造社团的过程,直到没有可归入社团的节点.将算法应用到空手道俱乐部网络和海豚网络中,并与SCAN算法进行对比.实验结果表明,CD3N算法有效解决了参数敏感性问题,能够很好地进行社团检测.     

A new approach for maximizing bichromatic reverse nearest neighbor search

Maximizing bichromatic reverse nearest neighbor (MaxBRNN) is a variant of bichromatic reverse nearest neighbor (BRNN). The purpose of the MaxBRNN problem is to find an optimal region that maximizes the size of BRNNs. This problem has lots of real applications such as location planning and profile-based marketing. The best-known algorithm for the MaxBRNN problem is called MaxOverlap. In this paper, we study the MaxBRNN problem and propose a new approach called MaxSegment for a two-dimensional space when the $L_2$ -norm is used. Then, we extend our algorithm to other variations of the MaxBRNN problem such as the MaxBRNN problem with other metric spaces, and a three-dimensional space. Finally, we conducted experiments on real and synthetic datasets to compare our proposed algorithm with existing algorithms. The experimental results verify the efficiency of our proposed approach.     

Feature selection based on loss-margin of nearest neighbor classification

The problem of selecting a subset of relevant features is classic and found in many branches of science including—examples in pattern recognition. In this paper, we propose a new feature selection criterion based on low-loss nearest neighbor classification and a novel feature selection algorithm that optimizes the margin of nearest neighbor classification through minimizing its loss function. At the same time, theoretical analysis based on energy-based model is presented, and some experiments are also conducted on several benchmark real-world data sets and facial data sets for gender classification to show that the proposed feature selection method outperforms other classic ones.     

基于近似最近邻搜索的改进PRM算法

针对移动机器人工作环境范围复杂时,使用传统概率路线图(PRM)算法非常耗时的问题,提出一种改进的PRM算法.PRM算法最耗时的部分是构建无向路径图,构建无向路径图的关键是近邻搜索.通过使用近似最近邻搜索中的局部敏感哈希算法代替原先最近邻搜索算法,在不降低生成路线图质量的前提下,加快无向路线图的构建速度,减少PRM算法的运行时间.仿真结果表明,改进的PRM算法相较于传统的PRM算法在无向路径图建立时间上减少27.36％ ～33.27％,使PRM算法效率大大提高.     

A parametrically-defined nearest neighbor distance measure

A parametrically-directed nearest neighbor procedure is developed to reduce the error between asymptotic and finite sample risk. Two examples are given.     

Manifold-ranking based retrieval using k-regular nearest neighbor graph

Manifold-ranking is a powerful method in semi-supervised learning, and its performance heavily depends on the quality of the constructed graph. In this paper, we propose a novel graph structure named k-regular nearest neighbor (k-RNN) graph as well as its constructing algorithm, and apply the new graph structure in the framework of manifold-ranking based retrieval. We show that the manifold-ranking algorithm based on our proposed graph structure performs better than that of the existing graph structures such as k-nearest neighbor (k-NN) graph and connected graph in image retrieval, 2D data clustering as well as 3D model retrieval. In addition, the automatic sample reweighting and graph updating algorithms are presented for the relevance feedback of our algorithm. Experiments demonstrate that the proposed algorithm outperforms the state-of-the-art algorithms.     

Boosting k-nearest neighbor classifier by means of input space projection

The k-nearest neighbors classifier is one of the most widely used methods of classification due to several interesting features, such as good generalization and easy implementation. Although simple, it is usually able to match, and even beat, more sophisticated and complex methods. However, no successful method has been reported so far to apply boosting to k-NN. As boosting methods have proved very effective in improving the generalization capabilities of many classification algorithms, proposing an appropriate application of boosting to k-nearest neighbors is of great interest.Ensemble methods rely on the instability of the classifiers to improve their performance, as k-NN is fairly stable with respect to resampling, these methods fail in their attempt to improve the performance of k-NN classifier. On the other hand, k-NN is very sensitive to input selection. In this way, ensembles based on subspace methods are able to improve the performance of single k-NN classifiers. In this paper we make use of the sensitivity of k-NN to input space for developing two methods for boosting k-NN. The two approaches modify the view of the data that each classifier receives so that the accurate classification of difficult instances is favored.The two approaches are compared with the classifier alone and bagging and random subspace methods with a marked and significant improvement of the generalization error. The comparison is performed using a large test set of 45 problems from the UCI Machine Learning Repository. A further study on noise tolerance shows that the proposed methods are less affected by class label noise than the standard methods.     

预测外膜蛋白的核最近邻算法

外膜蛋白由于其位于细菌的表面,从而对于抗生素和疫苗开发具有重要的研究价值.如何准确地将外膜蛋白从球蛋白和内膜蛋白等中识别出来对于从基因组序列中确认外膜蛋白以及预测其二级、三级结构都是一项重要的研究任务.近年来人们已经提出了若干从蛋白质序列出发预测外膜蛋白的方法.本文利用1种新的核方法,即核最近邻算法,结合蛋白质序列的子序列分布预测外膜蛋白,并和支持向量机方法、传统的最近邻算法进行了比较.结果表明本文算法不亚于已有的预测方法,而且新算法更为简洁、容易实现.同时我们发现残基顺序在外膜蛋白预测中具有重要作用.     

基于PCA与改进的最近邻法则的异常检测

提出一种新颖的基于特征抽取的异常检测方法,先对预处理数据进行标准化变换,然后应用主成份分析(PCA)抽取入侵特征,最后应用一种改进的最近邻分类方法--基于中心的最近邻分类法(CNN)检测入侵.利用KDD Cup'99数据集,将PCA删与PCA NN、PCA SVM、标准SVM进行比较,结果显示,在不降低分类器性能的情况下,特征抽取方法能对输入数据有效降维,且在各种方法中,PCA与CNN的结合能得到最优的入侵检测性能.     

基于最近邻域法的气体传感器模式识别技术研究

提出将气体传感器阵列检测与最近邻域法相结合的方法实现气体的模式识别。设计了用该方法进行气体识别的实验系统。该方法具有实验次数少,且识别准确度高的优点。实验以3只金属氧化物半导体气体传感器组成的阵列为例,详细讨论了该方法的实验过程与识别结果。通过对CH4,H,CO 3种气体进行识别实验,结果表明:该方法的正确识别率达到100%,具有很高的实用价值。     

基于Relief的特征加权壳近邻分类算法

壳近邻分类算法克服了k近邻分类在近邻选择上可能存在偏好的问题,使得在大数据集上的分类效果优于k近邻分类,为了进一步提高壳近邻算法的分类性能,提出了基于Relief特征加权的壳近邻分类算法.该算法在Relief算法的基础上求解训练集的特征权值,并利用特征权值来改进算法的距离度量方法和投票机制.实验结果表明,该算法在小数据和大数据上的分类性能都优于k近邻和壳近邻分类算法.     

Effective naive Bayes nearest neighbor based image classification on GPU

Non-parametric classifier, Naive Bayes nearest neighbor, is designed with no training phase, and its performance outperforms many well-trained learning-based image classifiers. Unfortunately, despite its high accuracy, it suffers from great computational pressure from distance computations in space of local feature. This paper explores accelerating strategies from perspectives of both algorithm design and software development. Our approach integrates space decomposition capability of Product quantization (PQ) and parallel accelerating capability of underlying computational platform, Graphics processing unit (GPU). PQ is exploited to compress the indexed local features and prune the search space. GPU is used to ease most of computational pressure by processing the tasks in parallel. To achieve good parallel efficiency, a new sequential classification process is first designed and decomposed into independent components with high parallelism. Effective parallelization techniques are then presented to make use of computational resources. Parallel heap array is built to accelerate the process of feature quantization. Distance table lookup is built to speed up the process of feature search. Comparative experiments on UIUC-Sport dataset demonstrate that our integrated solution outperforms other implementations significantly on Core-quad Intel Core i7 950 CPU and GPU of NVIDIA Geforce GTX460. Scalability experiment on 80 million tiny images database shows that our approach still performs well when large-scale image database is explored.     

A nearest neighbor architecture to overcome dephasing

We design a nearest-neighbor architectural layout that uses fixed positive and negative couplings between qubits, to overcome the effects of relative phases due to qubit precessions, both during idle times and gate operations. The scheme uses decoherence-free subspaces, and we show how to realize gate operations on these encoded qubits. The main advantage of our scheme is that most gate operations are realized by only varying a single control parameter, which greatly reduces the circuit complexity. Moreover, the scheme is robust against phase errors occurring as a result of finite rise and fall times due to non-ideal pulses.