Image mining by spectral features: A case study of scenery image classification

Spectral features of images, such as Gabor filters and wavelet transform can be used for texture image classification. That is, a classifier is trained based on some labeled texture features as the training set to classify unlabeled texture features of images into some pre-defined classes. The aim of this paper is twofold. First, it investigates the classification performance of using Gabor filters, wavelet transform, and their combination respectively, as the texture feature representation of scenery images (such as mountain, castle, etc.). A k-nearest neighbor (k-NN) classifier and support vector machine (SVM) are also compared. Second, three k-NN classifiers and three SVMs are combined respectively, in which each of the combined three classifiers uses one of the above three texture feature representations respectively, to see whether combining multiple classifiers can outperform the single classifier in terms of scenery image classification. The result shows that a single SVM using Gabor filters provides the highest classification accuracy than the other two spectral features and the combined three k-NN classifiers and three SVMs.     

A multiple-point spatially weighted k-NN classifier for remote sensing

A novel classification method based on multiple-point statistics (MPS) is proposed in this article. The method is a modified version of the spatially weighted k-nearest neighbour (k-NN) classifier, which accounts for spatial correlation through weights applied to neighbouring pixels. The MPS characterizes the spatial correlation between multiple points of land-cover classes by learning local patterns in a training image. This rich spatial information is then converted to multiple-point probabilities and incorporated into the k-NN classifier. Experiments were conducted in two study areas, in which the proposed method for classification was tested on a WorldView-2 sub-scene of the Sichuan mountainous area and an IKONOS image of the Beijing urban area. The multiple-point weighted k-NN method (MPk-NN) was compared to several alternatives; including the traditional k-NN and two previously published spatially weighted k-NN schemes; the inverse distance weighted k-NN, and the geostatistically weighted k-NN. The classifiers using the Bayesian and Support Vector Machine (SVM) methods, and these classifiers weighted with spatial context using the Markov random field (MRF) model, were also introduced to provide a benchmark comparison with the MPk-NN method. The proposed approach increased classification accuracy significantly relative to the alternatives, and it is, thus, recommended for the identification of land-cover types with complex and diverse spatial distributions.     

Robust visual tracking via bag of superpixels

The Bag of Words (BoW) model is one of the most popular and effective image representation methods and has been drawn increasing interest in computer vision filed. However, little attention is paid on it in visual tracking. In this paper, a visual tracking method based on Bag of Superpixels (BoS) is proposed. In BoS, the training samples are oversegmented to generate enough superpixel patches. Then K-means algorithm is performed on the collected patches to form visual words of the target and a superpixel codebook is constructed. Finally the tracking is accomplished via searching for the highest likelihood between candidates and codebooks within Bayesian inference framework. In this process, an effective updating scheme is adopted to help our tracker resist occlusions and deformations. Experimental results demonstrate that the proposed method outperforms several state-of-the-art trackers.     

VMVC: Verifiable multi-tone visual cryptography

Traditional k out of n threshold visual cryptography scheme is proposed to hide a secret image into n shares, where only k or more shares can visually reveal the secret image. Most of the previous state of art approaches on visual cryptography are almost restricted in processing of binary images as secret, which are inadequate for many applications like securely transmission of medical images(Store and Forward Telemedicine), forensic images etc. In this paper, a new Verifiable Multi-toned Visual Cryptography (VMVC) scheme is proposed to securely transmit the confidential images on web. Proposed approach also provides cheating prevention, since each pixel of shares contains a self embedding verifiable bit for integrity test of that pixel. Many existing approaches are suffering from many unnecessary encryption constraints like random shares, codebook requirement, contrast loss etc, which all are successfully addressed in proposed approach. Some comparisons with previously proposed methods are also made. Experimental results and analysis are used to prove the efficiency of proposed approach.     

An enhanced threshold visual secret sharing based on random grids

Random grids (RG)-based visual secret sharing (VSS) scheme can easily avoid the pixel expansion problem as well as requires no codebook design. However, previous scheme still suffers from low visual quality. In this paper, a new threshold RG-based VSS scheme aiming at improving the visual quality of the previewed image is presented. Compared with previous schemes, our scheme can gain better visual quality in the reconstructed images as well as (k, n) threshold. In addition, the factor affecting the visual quality is analyzed and the differences between related approaches are discussed.     

Liver fibrosis staging using CT image texture analysis and soft computing

Liver biopsy is considered to be the gold standard for analyzing chronic hepatitis and fibrosis; however, it is an invasive and expensive approach, which is also difficult to standardize. Medical imaging techniques such as ultrasonography, computed tomography (CT), and magnetic resonance imaging are non-invasive and helpful methods to interpret liver texture, and may be good alternatives to needle biopsy. Recently, instead of visual inspection of these images, computer-aided image analysis based approaches have become more popular. In this study, a non-invasive, low-cost and relatively accurate method was developed to determine liver fibrosis stage by analyzing some texture features of liver CT images. In this approach, some suitable regions of interests were selected on CT images and a comprehensive set of texture features were obtained from these regions using different methods, such as Gray Level Co-occurrence matrix (GLCM), Laws’ method, Discrete Wavelet Transform (DWT), and Gabor filters. Afterwards, sequential floating forward selection and exhaustive search methods were used in various combinations for the selection of most discriminating features. Finally, those selected texture features were classified using two methods, namely, Support Vector Machines (SVM) and k-nearest neighbors (k-NN). The mean classification accuracy in pairwise group comparisons was approximately 95% for both classification methods using only 5 features. Also, performance of our approach in classifying liver fibrosis stage of subjects in the test set into 7 possible stages was investigated. In this case, both SVM and k-NN methods have returned relatively low classification accuracies. Our pairwise group classification results showed that DWT, Gabor, GLCM, and Laws’ texture features were more successful than the others; as such features extracted from these methods were used in the feature fusion process. Fusing features from these better performing families further improved the classification performance. The results show that our approach can be used as a decision support system in especially pairwise fibrosis stage comparisons.     

A triangle area based nearest neighbors approach to intrusion detection

Intrusion detection is a necessary step to identify unusual access or attacks to secure internal networks. In general, intrusion detection can be approached by machine learning techniques. In literature, advanced techniques by hybrid learning or ensemble methods have been considered, and related work has shown that they are superior to the models using single machine learning techniques. This paper proposes a hybrid learning model based on the triangle area based nearest neighbors (TANN) in order to detect attacks more effectively. In TANN, the k-means clustering is firstly used to obtain cluster centers corresponding to the attack classes, respectively. Then, the triangle area by two cluster centers with one data from the given dataset is calculated and formed a new feature signature of the data. Finally, the k-NN classifier is used to classify similar attacks based on the new feature represented by triangle areas. By using KDD-Cup ’99 as the simulation dataset, the experimental results show that TANN can effectively detect intrusion attacks and provide higher accuracy and detection rates, and the lower false alarm rate than three baseline models based on support vector machines, k-NN, and the hybrid centroid-based classification model by combining k-means and k-NN.     

Land-cover classification of partly missing data using support vector machines

Land-cover classification based on multi-temporal satellite images for scenarios where parts of the data are missing due to, for example, clouds, snow or sensor failure has received little attention in the remote-sensing literature. The goal of this article is to introduce support vector machine (SVM) methods capable of handling missing data in land-cover classification. The novelty of this article consists of combining the powerful SVM regularization framework with a recent statistical theory of missing data, resulting in a new method where an SVM is trained for each missing data pattern, and a given incomplete test vector is classified by selecting the corresponding SVM model. The SVM classifiers are evaluated on Landsat Enhanced Thematic Mapper Plus (ETM?+?) images covering a scene of Norwegian mountain vegetation. The results show that the proposed SVM-based classifier improves the classification accuracy by 5–10% compared with single image classification. The proposed SVM classifier also outperforms recent non-parametric k-nearest neighbours (k-NN) and Parzen window density-based classifiers for incomplete data by about 3%. Moreover, since the resulting SVM classifier may easily be implemented using existing SVM libraries, we consider the new method to be an attractive choice for classification of incomplete data in remote sensing.     

Concept learning by fuzzy k-NN classification and relevance feedback for efficient image retrieval

A new method for combining visual and semantic features in image retrieval is presented. A fuzzy k-NN classifier assigns initial semantic labels to database images. These labels are gradually modified by relevance feedbacks from the users. Experimental results on a database of 1000 images from 10 semantic groups are reported.     

Improved k-nearest neighbor classification

k-nearest neighbor (k-NN) classification is a well-known decision rule that is widely used in pattern classification. However, the traditional implementation of this method is computationally expensive. In this paper we develop two effective techniques, namely, template condensing and preprocessing, to significantly speed up k-NN classification while maintaining the level of accuracy. Our template condensing technique aims at “sparsifying” dense homogeneous clusters of prototypes of any single class. This is implemented by iteratively eliminating patterns which exhibit high attractive capacities. Our preprocessing technique filters a large portion of prototypes which are unlikely to match against the unknown pattern. This again accelerates the classification procedure considerably, especially in cases where the dimensionality of the feature space is high. One of our case studies shows that the incorporation of these two techniques to k-NN rule achieves a seven-fold speed-up without sacrificing accuracy.     

A Dynamic k-Nearest Neighbor Method for WLAN-Based Positioning Systems

The static k-Nearest Neighbor (k-NN) method for localization has limitations in accuracy due to the fixed k value in the algorithm. To address this problem, and achieve better accuracy, we propose a new dynamic k-Nearest Neighbor (Dk-NN) method in which the optimal k value changes based on the topologies and distances of its nearest neighbors. The proposed method has been validated using the WLAN-fingerprint data sets collected at COEX, one of the largest convention centers in Seoul, Korea. The proposed method significantly reduced both the mean error distances and the standard deviations of location estimations, leading to a significant improvement in accuracy by ~ 23% compared to the cluster filtered k-NN (CFK) method, and ~ 17% compared to the k-NN (k = 1) method.     

Manifold-ranking based retrieval using k-regular nearest neighbor graph

Manifold-ranking is a powerful method in semi-supervised learning, and its performance heavily depends on the quality of the constructed graph. In this paper, we propose a novel graph structure named k-regular nearest neighbor (k-RNN) graph as well as its constructing algorithm, and apply the new graph structure in the framework of manifold-ranking based retrieval. We show that the manifold-ranking algorithm based on our proposed graph structure performs better than that of the existing graph structures such as k-nearest neighbor (k-NN) graph and connected graph in image retrieval, 2D data clustering as well as 3D model retrieval. In addition, the automatic sample reweighting and graph updating algorithms are presented for the relevance feedback of our algorithm. Experiments demonstrate that the proposed algorithm outperforms the state-of-the-art algorithms.     

Slicing the metric space to provide quick indexing of complex data in the main memory

Searching in a dataset for elements that are similar to a given query element is a core problem in applications that manage complex data, and has been aided by metric access methods (MAMs). A growing number of applications require indices that must be built faster and repeatedly, also providing faster response for similarity queries. The increase in the main memory capacity and its lowering costs also motivate using memory-based MAMs. In this paper, we propose the Onion-tree, a new and robust dynamic memory-based MAM that slices the metric space into disjoint subspaces to provide quick indexing of complex data. It introduces three major characteristics: (i) a partitioning method that controls the number of disjoint subspaces generated at each node; (ii) a replacement technique that can change the leaf node pivots in insertion operations; and (iii) range and k-NN extended query algorithms to support the new partitioning method, including a new visit order of the subspaces in k-NN queries. Performance tests with both real-world and synthetic datasets showed that the Onion-tree is very compact. Comparisons of the Onion-tree with the MM-tree and a memory-based version of the Slim-tree showed that the Onion-tree was always faster to build the index. The experiments also showed that the Onion-tree significantly improved range and k-NN query processing performance and was the most efficient MAM, followed by the MM-tree, which in turn outperformed the Slim-tree in almost all the tests.     

A numeral character recognition using the PCA mixture model

This paper proposes a method for recognizing the numeral characters based on the PCA (Principal Component Analysis) mixture model. The proposed method is motivated by the idea that the classification accuracy is improved by modeling each class into a mixture of several components and by performing the classification in the compact and decorrelated feature space. For realizing the idea, each numeral class is partitioned into several clusters and each cluster's density is estimated by a Gaussian distribution function in the PCA transformed space. The parameter estimation is performed by an iterative EM (Expectation Maximization) algorithm, and model order is selected by a fast sub-optimal validation scheme. The proposed method is also computation-effective because the optimal feature components for a cluster are determined by a sequential elimination of insignificant feature due to the ordering property of the significance among the feature components in the PCA transformed space. Simulation results shows that the proposed recognition method outperforms other methods such as the k-NN (Nearest Neighbor) method, a single PCA model, or the ICA (Independent Component Analysis) mixture model in terms of recognition accuracy.     

A “soft” K-nearest neighbor voting scheme

The K-Nearest Neighbor (K-NN) voting scheme is widely used in problems requiring pattern recognition or classification. In this voting scheme an unknown pattern is classified according to the classifications of its K nearest neighbors. If a majority of the K nearest neighbors have a given classification C*, then the unknown pattern is also given the classification C*. Although the scheme works well it is sensitive to the number of nearest neighbors, K, which is used. In this paper we describe a fuzzy K-NN voting scheme in which effectively the value of K varies automatically according to the local density of known patterns. We find that the new scheme consistently outperforms the traditional K-NN algorithm. © 2001 John Wiley & Sons, Inc.     

Quadratic-radial-basis-function-kernel for classifying multi-class agricultural datasets with continuous attributes

Classification of agricultural data such as soil data and crop data is significant as it allows the stakeholders to make meaningful decisions for farming. Soil classification aids farmers in deciding the type of crop to be sown for a particular type of soil. Similarly, wheat variety classification assists in selecting the right type of wheat for a particular product. Current methods used for classifying agricultural data are mostly manual. These methods involve agriculture field visits and surveys and are labor-intensive, expensive, and prone to human error. Recently, data mining techniques such as decision trees, k-nearest neighbors (k-NN), support vector machine (SVM), and Naive Bayes (NB) have been used in classification of agricultural data such as soil, crops, and land cover. The resulting classification aid the decision making process of government organizations and agro-industries in the field of agriculture. SVM is a popular approach for data classification. A recent study on SVM highlighted the fact that using multiple kernels instead of a single kernel would lead to better performance because of the greater learning and generalization power. In this work, a hybrid kernel based support vector machine (H-SVM) is proposed for classifying multi-class agricultural datasets having continuous attributes. Genetic algorithm (GA) or gradient descent (GD) methods are utilized to select the SVM parameters C and γ. The proposed kernel is called the quadratic-radial-basis-function kernel (QRK) and it combines both quadratic and radial basis function (RBF) kernels. The proposed classifier has the ability to classify all kinds of multi-class agricultural datasets with continuous features. Rigorous experiments using the proposed method are performed on standard benchmark and real world agriculture datasets. The results reveal a significant performance improvement over state of the art methods such as NB, k-NN, and SVM in terms of performance metrics such as accuracy, sensitivity, specificity, precision, and F-score.     

Meaningful visual secret sharing based on error diffusion and random grids

Random grid (RG) is an efficient method of eliminating the drawback of pixel expansion problem in visual secret sharing (VSS). Error diffusion (ED) technique is a brilliant method that improves the diffusion performance in an image by reducing the pattern noise and removing boundary and ’blackhole’ effects. In this paper, a novel meaningful RG-ED-based VSS, which encodes the (k, n) threshold into meaningful shadow images, is proposed at the price of not-clear recovered images. In addition, the novel scheme realizes the (k, n) threshold, avoids the design of complex codebook and averts the pixel expansion problem. Furthermore, the proposed RG-ED-based VSS inherits conventional benefits of VSS without the need of cryptographic efforts to decode the secret. Compared with other schemes reported in the literature, the present scheme has the benefits mentioned above, at the price of possible degrading of recovered images’ quality.     

Improving nearest neighbor classification with cam weighted distance

Nearest neighbor (NN) classification assumes locally constant class conditional probabilities, and suffers from bias in high dimensions with a small sample set. In this paper, we propose a novel cam weighted distance to ameliorate the curse of dimensionality. Different from the existing neighborhood-based methods which only analyze a small space emanating from the query sample, the proposed nearest neighbor classification using the cam weighted distance (CamNN) optimizes the distance measure based on the analysis of inter-prototype relationship. Our motivation comes from the observation that the prototypes are not isolated. Prototypes with different surroundings should have different effects in the classification. The proposed cam weighted distance is orientation and scale adaptive to take advantage of the relevant information of inter-prototype relationship, so that a better classification performance can be achieved. Experiments show that CamNN significantly outperforms one nearest neighbor classification (1-NN) and k-nearest neighbor classification (k-NN) in most benchmarks, while its computational complexity is comparable with that of 1-NN classification.