Image database design based on 9D-SPA representation for spatial relations

Spatial relationships between objects are important features for designing a content-based image retrieval system. We propose a new scheme, called 9D-SPA representation, for encoding the spatial relations in an image. With this representation, important functions of intelligent image database systems such as visualization, browsing, spatial reasoning, iconic indexing, and similarity retrieval can be easily achieved. The capability of discriminating images based on 9D-SPA representation is much more powerful than any spatial representation method based on minimum bounding rectangles or centroids of objects. The similarity measures using 9D-SPA representation provide a wide range of fuzzy matching capability in similarity retrieval to meet different user's requirements. Experimental results showed that our system is very effective in terms of recall and precision. In addition, the 9D-SPA representation can be incorporated into a two-level index structure to help reduce the search space of each query processing. The experimental results also demonstrated that, on average, only 0.1254 percent /spl sim/ 1.6829 percent of symbolic pictures (depending on various degrees of similarity) were accessed per query in an image database containing 50,000 symbolic pictures.     

Spatial reasoning and similarity retrieval for image database systems based on RS-strings

In this paper, we propose a rotation-invariant spatial knowledge representation called RS-string. Then we present the string generation algorithm to automatically generate RS-strings for segmented pictures. We also propose the spatial reasoning and similarity retrieval algorithms based on RS-strings. The similarity retrieval algorithm is much more flexible than all previous 2D string representations because our approach can consider every possible view of a query picture. Thus the system does not require the user to provide a query picture which must have the same orientation as that of a database picture. Finally, we provide several examples to demonstrate the capabilities of spatial reasoning and similarity retrieval based on the RS-string representation.     

Spatial arrangement of color in retrieval by visual similarity

In image search based on chromatic similarity, the effectiveness of retrieval can be improved by taking into account the spatial arrangement of colors. This can serve both to distinguish images with the same colors in different arrangement, and to capture the similarity between images with different colors but similar arrangements. We propose a model of representation and comparison which attains this goal by partitioning the image in separate entities and by associating them with individual chromatic attributes and with mutual spatial relationships. The effectiveness of the proposed model is assessed in a user-based evaluation. Experimental results show the capability of the model to join and balance chromatic and spatial similarity, thus improving the effectiveness of retrieval with respect to representations based on a global histogram.     

基于空间上下文加权词汇树的图像检索方法

基于词汇树的词袋模型(Bag-of-Words)表示算法是目前图像检索领域中的主流算法.针对传统词汇树方法中空间上下文信息缺失的问题,提出一种基于空间上下文加权词汇树的图像检索方法.该方法在词汇树框架下,首先生成SIFT点的空间上下文信息描述.然后利用SIFT点间的空间上下文相似度对SIFT间的匹配得分进行加权,得到图像间的相似度.最后,通过相似度排序完成图像检索.实验结果表明,该方法能够大幅度提高图像检索的性能,同时,对大规模图像库有较好的适用性.     

2D C-Tree Spatial Representation for Iconic Image

The 2D string approaches provide a natural way of constructing iconic indexing for images. The 2D C-string representation with an efficient cutting mechanism is more characteristic of spatial knowledge and efficient in the representation of iconic images. However, the computation of object ranks in a 2D C-string might make the inference of spatial reasoning somewhat complicated. This shortcoming is overcome by the 2D C-tree representation. The 2D C-tree not only keeps the comprehensive spatial knowledge in the original 2D C-string, but also the ordered labeled tree is more suitable for spatial reasoning and image retrieval. The spatial knowledge can be derived directly from the inference rules embedded in the characteristic structure of the 2D C-tree representation.     

空间对象的几何表示

1 引言空间推理是指利用空间理论和人工智能技术对空间对象建模、描述和表示,并据此对空间对象间的空间关系进行的定性或定量的分析和处理过程。目前,空间推理被广泛应用于地理信息系统、机器人导航、高级视觉、自然语言理解、工程设计和物理位置的常识推理等,是人工智能领域的一个研究热点。     

Spatial inference and similarity retrieval of an intelligent image database system based on object's spanning representation

In this paper, we presented a novel image representation method to capture the information about spatial relationships between objects in a picture. Our method is more powerful than all other previous methods in terms of accuracy, flexibility, and capability of discriminating pictures. In addition, our method also provides different degrees of granularity for reasoning about directional relations in both 8- and 16-direction reference frames. In similarity retrieval, our system provides twelve types of similarity measures to support flexible matching between the query picture and the database pictures. By exercising a database containing 3600 pictures, we successfully demonstrated the effectiveness of our image retrieval system. Experiment result showed that 97.8% precision rate can be achieved while maintaining 62.5% recall rate; and 97.9% recall rate can be achieved while maintaining 51.7% precision rate. On an average, 86.1% precision rate and 81.2% recall rate can be achieved simultaneously if the threshold is set to 0.5 or 0.6. This performance is considered to be very good as an information retrieval system.     

Similarity retrieval of iconic image database

The perception of spatial relationships among objects in a picture is one of the important selection criteria to discriminate and retrieve the images in an iconic image database system. The data structure called 2D string, proposed by Chang et al., is adopted to represent symbolic pictures. The 2D string preserves the objects' spatial knowledge embedded in images. Since spatial relationship is a fuzzy concept, the capability of similarity retrieval for the retrieval by subpicture is essential. In this paper, similarity measure based on 2D string longest common subsequence is defined. The algorithm for similarity retrieval is also proposed. Similarity retrieval provides the iconic image database with the distinguishing function different from a conventional database.     

Picture similarity retrieval using the 2D projection intervalrepresentation

Spatial relationships are important ingredients for expressing constraints in retrieval systems for pictorial or multimedia databases. We have proposed a unified representation for spatial relationships, 2D Projection Interval Relationships (2D-PIR), that integrates both directional and topological relationships. We develop techniques for similarity retrieval based on the 2D-PIR representation, including a method for dealing with rotated and reflected images     

基于增强空谱特征网络的空间感知协同表示算法

高光谱图像具有高维度、带间相关性较高、样本数量较少等诸多问题,直接利用表示学习算法对高光谱图像进行分类会导致严重的维数灾难.对于高光谱图像,不是所有的光谱带都可用于特定的分类任务.因此,文中提出基于增强空谱特征网络的空间感知协同表示算法.依据高光谱图像内在的低维流形构建基于空谱特征的分层网络.利用训练的网络对高维数据进行特征提取,并利用空间感知协同表示算法进行分类.在两个高光谱数据集Indian Pines和Pavia University上的实验表明文中算法的有效性.     

一个结合多方面定性空间信息的新方法

定性空间推理是人工智能领域中非常重要的研究内容.空间信息包含拓扑关系、大小关系、形状、距离等很多方面.以往多侧重于单一方面的研究,如何将孤立的各方面信息进行统一表示和推理是当前定性空间推理中的一个重要问题.提出利用结合操作来融合不同空间信息表示的新方法.利用结合操作,可以由原先完备互斥关系集合得到新关系,同时利用原有的复合表自动生成新关系的粗复合表.基于结合操作,给出2个理论模型:结合拓扑关系与大小关系模型、结合拓扑关系与远近关系模型.并提出了邻域划分图的概念,说明了邻域划分图与概念邻域图的关系.利用邻域划分图回答了Galton提出的问题:为什么LOS(视觉光线演算)的概念邻域图不同于标准的空间或时间关系的概念邻域图,这些关系的复合表中关系总是来自于概念邻域图.     

基于粒度层次的方位关系推理

方位关系推理是空间推理的一个重要方面。本文主要研究的是方位关系推理的问题之一——基于层次的方位关系表示与推理。文中首先从区域间的覆盖关系出发，提出了基于空间粒度的层次划分方法；然后，从“点对象”和“矩形对象”两个方面研究了基于粒度层次的方位关系表示与推理方法，并给出了相应的推理算法。文中的推理实例验证了粒度划分与推理方法的可行性。     

Compact and adaptive spatial pyramids for scene recognition

Most successful approaches on scene recognition tend to efficiently combine global image features with spatial local appearance and shape cues. On the other hand, less attention has been devoted for studying spatial texture features within scenes. Our method is based on the insight that scenes can be seen as a composition of micro-texture patterns. This paper analyzes the role of texture along with its spatial layout for scene recognition. However, one main drawback of the resulting spatial representation is its huge dimensionality. Hence, we propose a technique that addresses this problem by presenting a compact Spatial Pyramid (SP) representation. The basis of our compact representation, namely, Compact Adaptive Spatial Pyramid (CASP) consists of a two-stages compression strategy. This strategy is based on the Agglomerative Information Bottleneck (AIB) theory for (i) compressing the least informative SP features, and, (ii) automatically learning the most appropriate shape for each category. Our method exceeds the state-of-the-art results on several challenging scene recognition data sets.     

综合颜色特征的彩色图像检索方法

基于内容的图像检索技术已成为当前的研究热点,文章提出了一种综合利用两种颜色特征进行图像检索的新方法。首先,在变换空间建立色度直方图表示图像的颜色分布特征。为进行图像间的相似性度量,对Swain定义的直方图相似性度量作了改进,为弥补全局直方图不包含颜色空间分布关系的缺点,文章提取了另一种颜色特征,即分块的颜色矩,其距离度量为特征矢量的比值相似度。最后,综合利用上述两个特征对图像进行共同检索。通过对真实图像数据的检索实验表明:综合两种特征检索图像比单一特征检索效果更好。     

图像纹理特征及其在CBIR中的应用

基于内容的多媒体信息检索是当前世界的研究热点，然而在图像内容表示及其相似性度量这两个关键问题上取得的进展还不能令人满意。通过深入研究纹理的本质和人类视觉系统的特性，结合MPEG－7标准提出的图像内容描述了概念，提出一种新的谱方法来描述图像的纹理特征，并将其应用于图像内容的表示和检索，在此基础上，讨论了图像的颜色描述子和相似性度量策略，并给出了实验结果和性能评价。     

图像中物体空间关系的表示及在检索中的应用

基于图像中物体之间的空间关系的图像检索往往受困于待处理的图像中物体种类和空间位置难以自动准确地获取。文中基于物体识别算法的输出,提出一种对物体空间关系的三元组表示法,给出基于这种表示方法对图像索引、相似度计算和检索排序的方法及允许用户使用查询词和空间关系表达查询需求的二维输入界面,并实现原型系统。这种表示法具有良好的鲁棒性,可容忍物体识别算法一定程度的误差,将物体识别得到的置信度加入三元组表示法置信度计算和排序算法中,减少物体识别结果误差对检索性能的影响。在原型系统上的实验表明,该系统在实验中对包含物体位置关系的检索给出更准确的结果,在NDCG@m、MAP、F@m上均优于现有系统。     

Cross-modal alignment with graph reasoning for image-text retrieval

Image-text retrieval task has received a lot of attention in the modern research field of artificial intelligence. It still remains challenging since image and text are heterogeneous cross-modal data. The key issue of image-text retrieval is how to learn a common feature space while semantic correspondence between image and text remains. Existing works cannot gain fine cross-modal feature representation because the semantic relation between local features is not effectively utilized and the noise information is not suppressed. In order to address these issues, we propose a Cross-modal Alignment with Graph Reasoning (CAGR) model, in which the refined cross-modal features in the common feature space are learned and then a fine-grained cross-modal alignment method is implemented. Specifically, we introduce a graph reasoning module to explore semantic connection for local elements in each modality and measure their importance by self-attention mechanism. In a multi-step reasoning manner, the visual semantic graph and textual semantic graph can be effectively learned and the refined visual and textual features can be obtained. Finally, to measure the similarity between image and text, a novel alignment approach named cross-modal attentional fine-grained alignment is used to compute similarity score between two sets of features. Our model achieves the competitive performance compared with the state-of-the-art methods on Flickr30K dataset and MS-COCO dataset. Extensive experiments demonstrate the effectiveness of our model.

     

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization.     

基于图论的无监督区域遥感图像检索算法研究

为了改善基于内容的遥感图像检索技术,以遥感图像区域检索为框架,提出了一种新的基于图论的无监督学习遥感图像检索算法。首先,提出的方法 用图表为每一幅图像建模,同时结合局部信息和相关的空间结构,提供基于区域的图像代表。将每一幅图像初步划分为不同的区域,再通过属性关系图建模,节点和边界分别代表区域特征和它们之间存在的空间关系。然后,通过评估基于图像的相似点实现最相似于查询图像的图像检索。为匹配相应的图像以及按照图像相似点实现图像检索,采用了结合子图同构算法和光谱图嵌入技术的新型非精确图像匹配策略。实验结果显示,与其他两种无监督遥感图像检索方法相比,所提方法的检索性能明显改善。