Maximum likelihood classification,optimal or problematic? A comparison with the nearest neighbour classification

Abstract

The maximum likelihood and the nearest neighbour classification algorithms are reviewed, particularly from the point of view of user/analyst requirements. The two algorithms were put to use for the classification or Landsat TM data of agricultural scenes and accuracy with respect to ‘ground truth’ was evaluated using different parametric settings. Results show that within the maximum likelihood classification, accuracies and errors can vary to a considerable degree depending on the formation of the statistical classes from the training data. More interestingly, it was found that the nearest neighbour algorithm produced higher accuracies and was judged to be more robust, but it has computer implementation problems with high data dimensionality.     

Some classification algorithms integrating Dempster-Shafer theoryof evidence with the rank nearest neighbor rules

We propose five different ways of integrating Dempster-Shafer theory of evidence and the rank nearest neighbor classification rules with a view to exploiting the benefits of both. These algorithms have been tested on both real and synthetic data sets and compared with the k-nearest neighbour rule (k-NN), m-multivariate rank nearest neighbour rule (m-MRNN), and k-nearest neighbour Dempster-Shafer theory rule (k-NNDST), which is an algorithm that also combines Dempster-Shafer theory with the k-NN rule. If different features have widely different variances then the distance-based classifier algorithms like k-NN and k-NNDST may not perform well, but in this case the proposed algorithms are expected to perform better. Our simulation results indeed reveal this. Moreover, the proposed algorithms are found to exhibit significant improvement over the m-MRNN rule     

Localized algorithms for vlsi processor arrays

In this paper we analyze the algorithms expressed as a system of recurrence equations. The algorithms are called 2^?1 output algorithms if two output values of one function (variable identification) are specified by the system of recurrence equations for each index point in the algorithm. The algorithm is in free form if the indexes of these two values are not dependent. Two standard classes are determined by this criteria: the nearest neighbour and the all pair form. For example the sorting algorithm can be expressed in the all pair form i.e., the linear insertion algorithm or in the nearest neighbour form i.e., the bubble sort algorithm. However these algorithms are different in their nature. A procedure to eliminate the computational broadcast for the all pair 2^?1output algorithm has been proposed by the authors in [1]. The result obtained by implementing this procedure was a localized form of the algorithm and a system of uniform recurrence equations by eliminating the computational and data broadcast. So he data dependence method can be efficiently used for parallel implementations. The proposed procedure cannot be implemented directly on the nearest neighbour form algorithms. Here we show how the algorithm can be restructured into a form where the computational and data broadcast can be eliminated. These transformations result in localized algorithms. A few examples show how these algorithms can be implemented on processor arrays. For example, the Gentleman Kung triangular array [2] can be used for solving the QR decomposition algorithm for both forms of the algorithm. The implementations differ in the order of the data flow and the processor operation. We show that the implementation of the nearest neighbour algorithm is even better than the standard one.     

An incremental prototype set building technique

This paper deals with the task of finding a set of prototypes from the training set. A reduced set is obtained which is used instead of the training set when nearest neighbour classification is used. Prototypes are added in an incremental fashion, where at each step of the algorithm, the number of prototypes selected keeps on increasing. The number of patterns in the training data classified correctly also keeps on increasing till all patterns are classified properly. After this, a deletion operator is used where some prototypes which are not so useful are removed. This method has been used to obtain the prototypes for a variety of benchmark data sets and results have been presented.     

A Review of: “The GIS survival Guide. By M. J. CLARK. (Southampton: GeoData Institute, 1992.) [Pp.41.) Price λ5, λ'1·20 (academic).

The classification of the off-diagonal points within a typical grey level co-occurrence matrix (GLCM) is discussed through the application of an intuitive nearest peak and a boundary rule method. Both approaches are applied to a synthetic image consisting of five regions with varying amounts of added random noise and also to an image containing three Brodatz textures of different standard deviation. The two approaches correctly identify the majority of the internal region pixels. However, the nearest peak method is shown to produce serious misclassifications at the region boundaries in the form of bands of additional regions. The boundary rule method does not show this characteristic. The overall classification accuracy and the k hat statistic were used to test the performance of each technique.     

Fusion of point-based postal data with IKONOS imagery

Research in urban remote sensing has been recently reinvigorated by both the continuing fusion with GIS and the advent of high spatial resolution satellite sensor data. Both will be examined by this paper in terms of how GIS data at the point level can assist the identification and interpretation of urban land use patterns from classified land cover. Specifically, how spatial statistics can be used to summarise the two-dimensional patterns of point data representing residential and commercial buildings. In this paper point data refer to the location of postal addresses known as ADDRESS-POINT^TM and collected by the Ordnance Survey of Great Britain and COMPAS^TM in Northern Ireland. Groups of these postal points are characterised using standard nearest-neighbour and linear nearest-neighbour indices in terms of the spacing and arrangement of residential and commercial buildings. The indices then form the basis for the interpretation of urban pixels classified from IKONOS imagery at the 4 m spatial resolution. In addition, the paper will outline an agenda for constructing an automated pattern recognition system that would ultimately identify and characterise the physical arrangement of buildings in terms of density (compactness versus sparseness) and linearity. Preliminary results so far are most encouraging. Using ground truth from aerial photographs at 15 cm spatial resolution, classified IKONOS imagery representing two cities in the United Kingdom, Bristol and Belfast, have been investigated. In both, spatial patterns have demonstrated the ability to identify misclassified urban pixels and characterise a variety of building arrangements. Also, using the software e-Cognition, a spatial classification based on nearest neighbour contextual rules produced accuracies of 95.4% compared to 90.7% from a multispectral-only classification. Further, more extensive testing is continuing.     

Using Hellinger distance in a nearest neighbour classifier for relational databases

Nearest neighbour algorithms classify a previously unseen input case by finding similar cases to make predictions about the unknown features of the input case. The usefulness of the nearest neighbour algorithms has been demonstrated in many real-world domains. Unfortunately, most of the similarity measures discussed in the current nearest neighbour learning literature handle only limited data types, thus limiting their applicability to relational database applications.In this paper, we propose an enhanced nearest neighbour learning algorithm that is applicable to relational databases. The proposed method allows one to define similarity on a wide spectrum of attribute types. It automatically assigns to each attribute a weight of its importance with respect to the target attribute. The method has been implemented as a computer program and its effectiveness has been tested on four publicly available machine learning databases. Its performance is compared to another well-known machine learning method, C4.5. Our experimentation with the system demonstrates that the classification accuracy of the proposed system was superior to that of C4.5 in most cases.     

The <Emphasis Type="Italic">k</Emphasis>-Nearest Neighbour Join: Turbo Charging the KDD Process

The similarity join has become an important database primitive for supporting similarity searches and data mining. A similarity join combines two sets of complex objects such that the result contains all pairs of similar objects. Two types of the similarity join are well-known, the distance range join, in which the user defines a distance threshold for the join, and the closest pair query or k-distance join, which retrieves the k most similar pairs. In this paper, we propose an important, third similarity join operation called the k-nearest neighbour join, which combines each point of one point set with its k nearest neighbours in the other set. We discover that many standard algorithms of Knowledge Discovery in Databases (KDD) such as k-means and k-medoid clustering, nearest neighbour classification, data cleansing, postprocessing of sampling-based data mining, etc. can be implemented on top of the k-nn join operation to achieve performance improvements without affecting the quality of the result of these algorithms. We propose a new algorithm to compute the k-nearest neighbour join using the multipage index (MuX), a specialised index structure for the similarity join. To reduce both CPU and I/O costs, we develop optimal loading and processing strategies.     

加权局部二值模式的人脸特征提取

目的 为了能够得到图像更加丰富的纹理特征,提出一种新的自适应加权局部二值模式算法。方法 首先,将图像进行分块,利用新算法提取每个子块的局部二值模式的纹理直方图;然后,将各子图像的信息熵作为直方图的加权依据,对每个子块对应的直方图进行自适应加权,并将所有子块的直方图连接成最终的纹理特征。提取每个子块的局部纹理特征时的方法为:以某一像素点为中心取相邻的8个像素组成一个局部邻域,在该邻域内依据自适应设定的阈值分别比较3对水平方向和3对竖直方向像素值的大小,以此获得6位二进制码并将每位二进制码乘以相应的权重后相加,累加和即为该邻域新的局部二值模式纹理特征。结果 在两大人脸数据库上进行的实验结果表明,利用本文提出的方法提取纹理特征,并结合最近邻分类法可以得到85.29%和96.50%的正确识别率。结论 文中提出的自适应加权局部二值模式特征能够获取图像中更加丰富的纹理信息,因而具有较高的正确识别率,并且对于其他的物体识别也具有一定的参考价值。     

The effects of band-to-band misregistration on multi-spectral classification

Abstract

Landsat-MSS data covering 636 ? 457 pixels with band-to-band rnisregistration was digitally analysed for assessing the effect of misregistration on classification accuracy of land use/landcover mapping. Raw MSS data with band-to-band misregistration was geometricaly corrected using GCPs and a polynominal transformation of the first order. Subsequently, both raw and geometrically corrected data were analysed on a DIPIX system using a Gaussian maximum likelihood algorithm. Results indicate a substantial degradation in the classification accuracy due to mixed pixels which have remained unclassified owing to misregistration between bandpairs.     

一种新的基于统计的航迹关联算法

多源航迹融合一直是信息融合领域的研究热点。在对最近邻数据关联算法的研究基础上,提出一种新的"筒状"最近邻数据关联算法,对该算法进行仿真分析,与最近邻数据关联算法相比,新算法在密集环境下关联性能明显更优,易于工程实现。     

A Kernel-based sparse representation method for face recognition

Sparse Representation Method has been proved to outperform conventional face recognition (FR) methods and is widely applied in recent years. A novel Kernel-based Sparse Representation Method (KBSRM) is proposed in this paper. In order to cope with the possible complex variation of the face images caused by varying facial expression and pose, the KBSRM first uses a kernel-induced distance to determine N nearest neighbors of the testing sample from all the training samples. Then, in the second step, the KBSRM represents the testing sample as a linear combination of the determinate N nearest neighbors and performs the classification by the representation result. It can be inferred that the N nearest training samples selected are closer to the test sample than the rest, so using the N nearest neighbors to represent the testing sample can make the ultimate classification more accurate. A number of FR experiments show that the KBSRM can achieve a better classification result than the algorithm mentioned in Xu et al. (Neural Comput Appl doi:10.1007/s00521-012-0833-5).     

Vector choice in the texture spectrum approach

Recently the texture spectrum approach has been proposed as a statistical method for texture analysis, and applied to remotely sensed images. In the present study, this method is generalized to a space of M vectors and N grey level intervals for the elements of texture units, instead of M = 8 and N = 3 in the earlier studies of the texture spectrum. In this way, the texture unit set can be defined in a neighbourhood of 3 pixels by 3 pixels, 5 pixels by 5 pixels, 7 pixels by 7 pixels or other forms and sizes, and the co-occurrence matrix approach is unified to the texture spectrum method with the extreme case of M = 1. Several combinations of M and N have been evaluated to classify an imagery composed of six natural textures. The results show maximum discrimination for M = 5, followed by M = 4. In this way we minimize the calculation lime needed to maximize the accuracy of the classification.     

Object-based classification using SPOT-5 imagery for Moso bamboo forest mapping

This study proposed a multi-scale, object-based classification analysis of SPOT-5 imagery to map Moso bamboo forest. A three-level hierarchical network of image objects was developed through multi-scale segmentation. By combining spectral and textural properties, both the classification tree and nearest neighbour classifiers were used to classify the image objects at Level 2 in the three-level object hierarchy. The feature selection results showed that most of the object features were related to the spectral properties for both the classification tree and nearest neighbour classifiers. Contextual information characterized by the composition of classified image objects using the class-related features assisted the detection of shadow areas at Levels 1 and 3. Better classification results were achieved using the nearest neighbour algorithm, with both the producer’s and user’s accuracy higher than 90% for Moso bamboo and an overall accuracy of over 85%. The object-based approach toward incorporating textural and contextual information in classification sequence at various scales shows promise in the analysis of forest ecosystems of a complex nature.     

Spatial-spectral neighbour graph for dimensionality reduction of hyperspectral image classification

Recently, graph embedding-based methods have drawn increasing attention for dimensionality reduction (DR) of hyperspectral image (HSI) classification. Graph construction is a critical step for those DR methods. Pairwise similarity graph is generally employed to reflect the geometric structure in the original data. However, it ignores the similarity of neighbouring pixels. In order to further improve the classification performance, both spectral and spatial-contextual information should be taken into account in HSI classification. In this paper, a novel spatial-spectral neighbour graph (SSNG) is proposed for DR of HSI classification, which consists of the following four steps. First, a superpixel-based segmentation algorithm is adopted to divide HSI into many superpixels. Second, a novel distance metric is utilized to reflect the similarity of two spectral pixels in each superpixel. In the third step, a spatial-spectral neighbour graph is constructed according to the above distance metric. At last, support vector machine with a composite kernel (SVM-CK) is adopted to classify the dimensionality-reduced HSI. Experimental results on three real hyperspectral datasets demonstrate that our method can achieve higher classification accuracy with relatively less consumed time than other graph embedding-based methods.     

Extraction and analysis of multiple time window features associated with muscle fatigue conditions using sEMG signals

In this work, an attempt has been made to differentiate surface electromyography (sEMG) signals under muscle fatigue and non-fatigue conditions with multiple time window (MTW) features. sEMG signals are recorded from biceps brachii muscles of 50 volunteers. Eleven MTW features are extracted from the acquired signals using four window functions, namely rectangular windows, Hamming windows, trapezoidal windows, and Slepian windows. Prominent features are selected using genetic algorithm and information gain based ranking. Four different classification algorithms, namely naïve Bayes, support vector machines, k-nearest neighbour, and linear discriminant analysis, are used for the study. Classifier performances with the MTW features are compared with the currently used time- and frequency-domain features. The results show a reduction in mean and median frequencies of the signals under fatigue. Mean and variance of the features differ by an order of magnitude between the two cases considered. The number of features is reduced by 45% with the genetic algorithm and 36% with information gain based ranking. The k-nearest neighbour algorithm is found to be the most accurate in classifying the features, with a maximum accuracy of 93% with the features selected using information gain ranking.     

A resource limited artificial immune system algorithm for supervised classification of multi/hyper‐spectral remote sensing imagery

The resource limited artificial immune system (RLAIS), a new computational intelligence approach, is being increasingly recognized as one of the most competitive methods for data clustering and analysis. Nevertheless, owing to the inherent complexity of the conventional RLAIS algorithm, its application to multi/hyper‐class remote sensing image classification has been considerably limited. This paper explores a novel artificial immune algorithm based on the resource limited principles for supervised multi/hyper‐spectral image classification. Three experiments with different types of images were performed to evaluate the performance of the proposed algorithm in comparison with other traditional image classification algorithms: parallelepiped, minimum distance, maximum likelihood, K‐nearest neighbour and back‐propagation neural network. The results show that the proposed algorithm consistently outperforms the traditional algorithms in all the experiments and hence provides an effective new option for processing multi/hyper spectral remote sensing images.     

结合混沌鸟群算法的阴极铜板表面缺陷检测

目的 铜电解过程中常因电解液溶解气体过饱阻止铜离子析出而在铜板表面形成凸起,常由操作员目视对铜板表面质量进行鉴别以决定归类,针对人工判别电解阴极铜板表面质量准确度和效率都较低的问题,提出一种结合混沌鸟群算法的铜板表面凸起智能识别方法。方法 为增强算法的全局搜索能力,引入鸟群算法;选取鸟群劣质个体交替进行混和动态步长位置更新增加种群多样性以免陷入局部最优;对铜板表面缺陷进行分析,提出基点生长法并结合形态学开操作消除铜板图像纹理以提高算法对凸起面积计算的准确性。将最佳熵阈值确定法（Kapur-Sahoo-Wong,KSW）作为鸟群算法的适应度函数对铜板图像进行阈值分割,通过统计分割图像凸起像素点个数,得到实际凸起面积占比以决定铜板是否合格。结果 将本文算法与遗传算法（genetic algorithm,GA）、鸡群算法（chicken swarm optimization,CSO）、萤火虫算法（glowworm swarm optimization,GSO）及鸟群算法（bird swarm algorithm,BSA）4种算法分别在时间、适应度值和结构相似度（structural similarity index measurement,SSIM）3个指标下分析对比,实验结果表明,本文算法适应度值可提高0.0030.701,SSIM值可提高0.0750.169。结论 本文方法能有效检测铜板表面凸起面积占比并对其进行合格品、次品分类。     

谐波分析光谱角制图高光谱影像分类

目的 针对光谱角制图(SAM)分类算法对高光谱像元光谱曲线的局部特征和其辐射强度不敏感,而且易受噪声和维数灾难影响,致使分类效率低和精度较差等缺陷,将谐波分析(HA)技术引入到SAM高光谱影像分类中,提出一种基于谐波分析的光谱角制图(HA-SAM)高光谱影像分类算法.方法 利用HA技术将高光谱影像从光谱维变换到能量谱特征维空间,并提取低次谐波分量及特征系数(谐波余项、相位和振幅),用特征系数组成的向量代替光谱向量,对高光谱影像进行SAM分类.结果 将SAM和HA-SAM同时应用于EO-1卫星的Hyperion高光谱影像分类,通过对比和分析,验证了HA-SAM的优越性,再选择AVIRIS(airborne visible infrared imaging spectrometer)高光谱影像对HA-SAM进行验证,结果表明该算法具有较强的普适性.结论 HA-SAM提高了传统SAM高光谱影像分类的效率和精度,而且适用性较强具有良好的应用前景.     

Compressive sensing based electronic nose platform

Electronic nose (EN) systems play a significant role for gas monitoring and identification in gas plants. Using an EN system which consists of an array of sensors provides a high performance. Nevertheless, this performance is bottlenecked by the high system complexity incorporated with the high number of sensors. In this paper a new EN system is proposed using data sets collected from an in-house fabricated $4\times 4$ tin-oxide gas array sensor. The system exploits the theory of compressive sensing (CS) and distributed compressive sensing (DCS) to reduce the storage capacity and power consumption. The obtained results have shown that compressing the transmitted data to 20% of its original size will preserve the information by achieving a high reconstruction quality. Moreover, exploiting DCS will maintain the same reconstruction quality for just 15% of the original size. This high quality of reconstruction is explored for classification using several classifiers such as decision tree (DT), K-nearest neighbour (KNN) and extended nearest neighbour (ENN) along with linear discrimination analysis (LDA) as feature reduction technique. CS-based reconstructed data has achieved a 95% classification accuracy. Furthermore, DCS-based reconstructed data achieved a 98.33% classification accuracy which is the same as using original data without compression.