A flexible multi-layer self-organizing map for generic processing of tree-structured data

A new multi-layer self-organizing map (MLSOM) is proposed for unsupervised processing tree-structured data. The MLSOM is an improved self-organizing map for handling structured data. By introducing multiple SOM layers, the MLSOM can overcome the computational speed and visualization problems of SOM for structured data (SOM-SD). Node data in different levels of a tree are processed in different layers of the MLSOM. Root nodes are dedicatedly processed on the top SOM layer enabling the MLSOM a better utilization of SOM map compared with the SOM-SD. Thus, the MLSOM exhibits better data organization, clustering, visualization, and classification results of tree-structured data. Experimental results on three different data sets demonstrate that the proposed MLSOM approach can be more efficient and effective than the SOM-SD.     

Content-based image retrieval using growing hierarchical self-organizing quadtree map

In this paper, a growing hierarchical self-organizing quadtree map (GHSOQM) is proposed and used for a content-based image retrieval (CBIR) system. The incorporation of GHSOQM in a CBIR system organizes images in a hierarchical structure. The retrieval time by GHSOQM is less than that by using direct image comparison using a flat structure. Furthermore, the ability of incremental learning enables GHSOQM to be a prospective neural-network-based approach for CBIR systems. We also propose feature matrices, image distance and relevance feedback for region-based images in the GHSOQM-based CBIR system. Experimental results strongly demonstrate the effectiveness of the proposed system.     

Content-based image collection summarization and comparison using self-organizing maps

Progresses made on content-based image retrieval have reactivated the research on image analysis and a number of similarity-based methods have been established to assess the similarity between images. In this paper, the content-based approach is extended towards the problem of image collection summarization and comparison. For these purposes we propose to carry out clustering analysis on visual features using self-organizing maps, and then evaluate their similarity using a few dissimilarity measures implemented on the feature maps. The effectiveness of these dissimilarity measures is then examined with an empirical study.     

PicSOM – content-based image retrieval with self-organizing maps

We have developed a novel system for content-based image retrieval in large, unannotated databases. The system is called PicSOM, and it is based on tree structured self-organizing maps (TS-SOMs). Given a set of reference images, PicSOM is able to retrieve another set of images which are similar to the given ones. Each TS-SOM is formed with a different image feature representation like color, texture, or shape. A new technique introduced in PicSOM facilitates automatic combination of responses from multiple TS-SOMs and their hierarchical levels. This mechanism adapts to the user's preferences in selecting which images resemble each other. Thus, the mechanism implements a relevance feedback technique on content-based image retrieval. The image queries are performed through the World Wide Web and the queries are iteratively refined as the system exposes more images to the user.     

Content-based image retrieval using joint correlograms

The comparison of digital images to determine their degree of similarity is one of the fundamental problems of computer vision. Many techniques exist which accomplish this with a certain level of success, most of which involve either the analysis of pixel-level features or the segmentation of images into sub-objects that can be geometrically compared. In this paper we develop and evaluate a new variation of the pixel feature and analysis technique known as the color correlogram in the context of a content-based image retrieval system. Our approach is to extend the autocorrelogram by adding multiple image features in addition to color. We compare the performance of each index scheme with our method for image retrieval on a large database of images. The experiment shows that our proposed method gives a significant improvement over histogram or color correlogram indexing, and it is also memory-efficient.

Content-based image retrieval with relevance feedback using random walks

In this paper, we propose a novel approach to content-based image retrieval with relevance feedback, which is based on the random walker algorithm introduced in the context of interactive image segmentation. The idea is to treat the relevant and non-relevant images labeled by the user at every feedback round as “seed” nodes for the random walker problem. The ranking score for each unlabeled image is computed as the probability that a random walker starting from that image will reach a relevant seed before encountering a non-relevant one. Our method is easy to implement, parameter-free and scales well to large datasets. Extensive experiments on different real datasets with several image similarity measures show the superiority of our method over different recent approaches.     

Content-based image retrieval using fuzzy perceptual feedback

In this paper, a new framework called fuzzy relevance feedback in interactive content-based image retrieval (CBIR) systems is introduced. Conventional binary labeling scheme in relevance feedback requires a crisp decision to be made on the relevance of the retrieved images. However, it is inflexible as user interpretation of visual content varies with respect to different information needs and perceptual subjectivity. In addition, users tend to learn from the retrieval results to further refine their information requests. It is, therefore, inadequate to describe the user’s fuzzy perception of image similarity with crisp logic. In view of this, we propose a fuzzy relevance feedback approach which enables the user to make a fuzzy judgement. It integrates the user’s fuzzy interpretation of visual content into the notion of relevance feedback. An efficient learning approach is proposed using a fuzzy radial basis function (FRBF) network. The network is constructed based on the user’s feedbacks. The underlying network parameters are optimized by adopting a gradient-descent training strategy due to its computational efficiency. Experimental results using a database of 10,000 images demonstrate the effectiveness of the proposed method.

Similarity-based image organization and browsing using multi-resolution self-organizing map

An algorithm is presented in this paper to facilitate the exploration of large image collections based on visual similarities. Starting with an unordered and unannotated set of images, the algorithm first extracts the salient details into feature vectors using both color and gradient information. The feature vectors are then used to train a self-organizing map which maps high-dimensional feature vectors onto a 2D canvas so that images with similar feature vectors are grouped together. When users browse the image collection, an image collage is generated that selects and displays the most pertinent set of images based on which portion of the 2D canvas is currently in view. Flowing from an overview to details is a seamless operation controlled simply by pan and zoom, with representative images selected in a consistent and predictable way. To make organizing larger image collections practical in interactive time, the organization algorithm is designed to run in parallel on graphics processing units. Overall this paper presents an end-to-end solution that facilitates the surfing of image collections in a fresh way.     

Estimating impervious surfaces from medium spatial resolution imagery using the self-organizing map and multi-layer perceptron neural networks

The studies of impervious surfaces are important because they are related to many environmental problems, such as water quality, stream health, and the urban heat island effect. Previous studies have discussed that the self-organizing map (SOM) can provide a promising alternative to the multi-layer perceptron (MLP) neural networks for image classification at both per-pixel and sub-pixel level. However, the performances of SOM and MLP have not been compared in the estimation and mapping of urban impervious surfaces. In mid-latitude areas, plant phenology has a significant influence on remote sensing of the environment. When the neural networks approaches are applied, how satellite images acquired in different seasons impact impervious surface estimation of various urban surfaces (such as commercial, residential, and suburban/rural areas) remains to be answered. In this paper, an SOM and an MLP neural network were applied to three ASTER images acquired on April 5, 2004, June 16, 2001, and October 3, 2000, respectively, which covered Marion County, Indiana, United States. Six impervious surface maps were yielded, and an accuracy assessment was performed. The root mean square error (RMSE), the mean average error (MAE), and the coefficient of determination (R²) were calculated to indicate the accuracy of impervious surface maps. The results indicated that the SOM can generate a slightly better estimation of impervious surfaces than the MLP. Moreover, the results from three test areas showed that, in the residential areas, more accurate results were yielded by the SOM, which indicates that the SOM was more effective in coping with the mixed pixels than the MLP, because the residential area prevailed with mixed pixels. Results obtained from the commercial area possessed very high RMSE values due to the prevalence of shade, which indicates that both algorithms cannot handle the shade problem well. The lowest RMSE value was obtained from the rural area due to containing of less mixed pixels and shade. This research supports previous observations that the SOM can provide a promising alternative to the MLP neural network. This study also found that the impact of different map sizes on the impervious surface estimation is significant.     

Web image retrieval using majority-based ranking approach

Web image retrieval has characteristics different from typical content-based image retrieval; web images have associated textual cues. However, a web image retrieval system often yields undesirable results, because it uses limited text information such as surrounding text, URLs, and image filenames. In this paper, we propose a new approach to retrieval, which uses the image content of retrieved results without relying on assistance from the user. Our basic hypothesis is that more popular images have a higher probability of being the ones that the user wishes to retrieve. According to this hypothesis, we propose a retrieval approach that is based on a majority of the images under consideration. We define four methods for finding the visual features of majority of images; (1) majority-first method, (2) centroid-of-all method, (3) centroid-of-top K method, and (4) centroid-of-largest-cluster method. In addition, we implement a graph/picture classifier for improving the effectiveness of web image retrieval. We evaluate the retrieval effectiveness of both our methods and conventional ones by using precision and recall graphs. Experimental results show that the proposed methods are more effective than conventional keyword-based retrieval methods.     

Extraction of major object features using VQ clustering for content-based image retrieval

An image representation method using vector quantization (VQ) on color and texture is proposed in this paper. The proposed method is also used to retrieve similar images from database systems. The basic idea is a transformation from the raw pixel data to a small set of image regions, which are coherent in color and texture space. A scheme is provided for object-based image retrieval. Features for image retrieval are the three color features (hue, saturation, and value) from the HSV color model and five textural features (ASM, contrast, correlation, variance, and entropy) from the gray-level co-occurrence matrices. Once the features are extracted from an image, eight-dimensional feature vectors represent each pixel in the image. The VQ algorithm is used to rapidly cluster those feature vectors into groups. A representative feature table based on the dominant groups is obtained and used to retrieve similar images according to the object within the image. This method can retrieve similar images even in cases where objects are translated, scaled, and rotated.     

Content-based image retrieval with the normalized information distance

The main idea of content-based image retrieval (CBIR) is to search on an image’s visual content directly. Typically, features (e.g., color, shape, texture) are extracted from each image and organized into a feature vector. Retrieval is performed by image example where a query image is given as input by the user and an appropriate metric is used to find the best matches in the corresponding feature space. We attempt to bypass the feature selection step (and the metric in the corresponding feature space) by following what we believe is the logical continuation of the CBIR idea of searching visual content directly. It is based on the observation that, since ultimately, the entire visual content of an image is encoded into its raw data (i.e., the raw pixel values), in theory, it should be possible to determine image similarity based on the raw data alone. The main advantage of this approach is its simplicity in that explicit selection, extraction, and weighting of features is not needed. This work is an investigation into an image dissimilarity measure following from the theoretical foundation of the recently proposed normalized information distance (NID) [M. Li, X. Chen, X. Li, B. Ma, P. Vitányi, The similarity metric, in: Proceedings of the 14th ACM-SIAM Symposium on Discrete Algorithms, 2003, pp. 863–872]. Approximations of the Kolmogorov complexity of an image are created by using different compression methods. Using those approximations, the NID between images is calculated and used as a metric for CBIR. The compression-based approximations to Kolmogorov complexity are shown to be valid by proving that they create statistically significant dissimilarity measures by testing them against a null hypothesis of random retrieval. Furthermore, when compared against several feature-based methods, the NID approach performed surprisingly well.     

Exploring statistical correlations for image retrieval

Bridging the cognitive gap in image retrieval has been an active research direction in recent years, of which a key challenge is to get enough training data to learn the mapping functions from low-level feature spaces to high-level semantics. In this paper, image regions are classified into two types: key regions representing the main semantic contents and environmental regions representing the contexts. We attempt to leverage the correlations between types of regions to improve the performance of image retrieval. A Context Expansion approach is explored to take advantages of such correlations by expanding the key regions of the queries using highly correlated environmental regions according to an image thesaurus. The thesaurus serves as both a mapping function between image low-level features and concepts and a store of the statistical correlations between different concepts. It is constructed through a data-driven approach which uses Web data (images, their surrounding textual annotations) as training data source to learn the region concepts and to explore the statistical correlations. Experimental results on a database of 10,000 general-purpose images show the effectiveness of our proposed approach in both improving search precision (i.e. filter irrelevant images) and recall (i.e. retrieval relevant images whose context may be varied). Several major factors which have impact on the performance of our approach are also studied.     

Emotion recognition from geometric facial features using self-organizing map

This paper presents a novel emotion recognition model using the system identification approach. A comprehensive data driven model using an extended Kohonen self-organizing map (KSOM) has been developed whose input is a 26 dimensional facial geometric feature vector comprising eye, lip and eyebrow feature points. The analytical face model using this 26 dimensional geometric feature vector has been effectively used to describe the facial changes due to different expressions. This paper thus includes an automated generation scheme of this geometric facial feature vector. The proposed non-heuristic model has been developed using training data from MMI facial expression database. The emotion recognition accuracy of the proposed scheme has been compared with radial basis function network, multi-layered perceptron model and support vector machine based recognition schemes. The experimental results show that the proposed model is very efficient in recognizing six basic emotions while ensuring significant increase in average classification accuracy over radial basis function and multi-layered perceptron. It also shows that the average recognition rate of the proposed method is comparatively better than multi-class support vector machine.     

Hidden annotation for image retrieval with long-term relevance feedback learning

Hidden annotation (HA) is an important research issue in content-based image retrieval (CBIR). We propose to incorporate long-term relevance feedback (LRF) with HA to increase both efficiency and retrieval accuracy of CBIR systems. The work contains two parts. (1) Through LRF, a multi-layer semantic representation is built to automatically extract hidden semantic concepts underlying images. HA with these concepts alleviates the burden of manual annotation and avoids the ambiguity problem of keyword-based annotation. (2) For each learned concept, semi-supervised learning is incorporated to automatically select a small number of candidate images for annotators to annotate, which improves efficiency of HA.     

Adaptive tree similarity learning for image retrieval

Learning-enhanced relevance feedback is one of the most promising and active research directions in content-based image retrieval in recent years. However, the existing approaches either require prior knowledge of the data or converge slowly and are thus not coneffective. Motivated by the successful history of optimal adaptive filters, we present a new approach to interactive image retrieval based on an adaptive tree similarity model to solve these difficulties. The proposed tree model is a hierarchical nonlinear Boolean representation of a user query concept. Each path of the tree is a clustering pattern of the feedback samples, which is small enough and local in the feature space that it can be approximated by a linear model nicely. Because of the linearity, the parameters of the similartiy model are better learned by the optimal adaptive filter, which does not require any prior knowledge of the data and supports incremental learning with a fast convergence rate. The proposed approach is simple to implement and achieves better performance than most approaches. To illustrate the performance of the proposed approach, extensive experiments have been carried out on a large heterogeneous image collection with 17,000 images, which render promising results on a wide variety of queries.An early version of part of the system was reported in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition 2001.     

基于内容的图像检索技术与医学图像检索

在分析基于内容的图像检索技术特点的基础上,提出了4种基于内容的图像检索方法,并对每种方法的实现特别是特征抽取进行了一定的研究。根据医学图像的使用特点,对基于内容的医学图像检索技术进行了初步的研究;对医学图像特征的抽取,应将重点放在形状特征和纹理特征的抽取上;同时,对医学图像进行检索,还可以使用颜色空间分布特征,来进一步进行相似匹配。