Effective image annotation for searches using multilevel semantics

There is an increasing need for automatic image annotation tools to enable effective image searching in digital libraries. In this paper, we present a novel probabilistic model for image annotation based on content-based image retrieval techniques and statistical analysis. One key difficulty in applying statistical methods to the annotation of images is that the number of manually labeled images used to train the methods is normally insufficient. Numerous keywords cannot be correctly assigned to appropriate images due to lacking or missing information in the labeled image databases. To deal with this challenging problem, we also propose an enhanced model in which the annotated keywords of a new image are defined in terms of their similarity at different semantic levels, including the image level, keyword level, and concept level. To avoid missing some relevant keywords, the model labels the keywords with the same concepts as the new image. Our experimental results show that the proposed models are effective for annotating images that have different qualities of training data.     

基于本体的图像自动标注

提出一种新的图像本体标注的框架,结合领域本体中概念的关系,通过层次概率标注来获得图像高层语义概念的标注,实现待标注图像语义的自动标注。我们将图像的语义可以定义为属性概念和高层抽象概念,采用二次标注方法实现对于图像语义的自动标注。实验证明,本文的方法可以使图像获得丰富的高层抽象语义概念标注,从而缩小"语义鸿沟",有效提高了检索的效率和精确度。     

图像语义自动标注及其粒度分析方法

缩小图像低层视觉特征与高层语义之间的鸿沟, 以提高图像语义自动标注的精度, 进而快速满足用户检索图像的需求,一直是图像语义自动标注研究的关键. 粒度分析方法是一种层次的、重要的数据分析方法, 为复杂问题的求解提供了新的思路. 图像理解与分析的粒度不同, 图像语义标注的精度则不同, 检索的效率及准确度也就不同. 本文对目前图像语义自动标注模型的方法进行综述和分析, 阐述了粒度分析方法的思想、模型及其在图像语义标注过程中的应用, 探索了以粒度分析为基础的图像语义自动标注方法并给出进一步的研究方向.     

弱标签环境下基于语义邻域学习的图像标注

图像语义自动标注是实现图像语义检索与管理的关键,是具有挑战性的研究课题.传统的图像标注方法需要具有完整、准确标签的数据集才能取得较好的标注性能.然而,在现实应用中获得数据的标签往往是不准确、不完整的,并且标签分布不均衡.对于Web图像和社会化图像尤其如此.为了更好地利用这些弱标签样本,提出了一种基于语义邻域学习的图像自动标注方法(semantic neighborhood learning from weakly labeled image, SNLWL).首先在邻域标签损失误差最小化意义下,填充训练集样本标签.通过递进式的邻域选择过程,保证建立的语义一致邻域内样本具有全局相似性、部分相关性和语义一致性,并且语义标签分布平衡.在邻域标签重构误差最小化意义下进行标签预测,降低噪声标签对性能的影响.多个数据集上的实验结果表明,与已知的具有较好标注效果的方法相比,此方法更适用于处理弱标签数据集,标准评测集上的测试也表明了此方法的有效性.     

基于词频同现与WordNet的图像自动标注改善算法研究

图像自动标注是模式识别与计算机视觉等领域中的重要问题。针对现有图像自动标注模型普遍受到语义鸿沟问题的影响,提出了基于关键词同现的图像自动标注改善方法,该方法利用数据集中标注词间的关联性来改善图像自动标注的结果。此外,针对上述方法不能反映更广义的人的知识以及易受数据库规模影响等问题,提出了基于语义相似的图像自动标注改善方法,通过引入具有大量词汇、包含了人知识的结构化电子词典WordNet来计算词汇间的关系并改善图像自动标注结果。实验结果表明,提出的两个图像自动标注改善方法在各项评价指标上相比以往模型均有所提高。     

融合多特征的深度学习标注方法

缩小图像低层视觉特征与高层语义之间的鸿沟,以提高图像语义自动标注的精度,是研究大规模图像数据管理的关键。提出一种融合多特征的深度学习图像自动标注方法,将图像视觉特征以不同权重组合成词包,根据输入输出变量优化深度信念网络,完成大规模图像数据语义自动标注。在通用Corel图像数据集上的实验表明,融合多特征的深度学习图像自动标注方法,考虑图像不同特征的影响,提高了图像自动标注的精度。     

一种新的基于语义聚类和图算法的自动图像标注方法

针对图像检索中的语义鸿沟问题,提出了一种新颖的自动图像标注方法。该方法首先采用了一种基于软约束的半监督图像聚类算法(SHMRF-Kmeans)对已标注图像的区域进行语义聚类,这种聚类方法可以同时考虑图像的视觉信息和语义信息。并利用图算法——Manifold排序学习算法充分发掘语义概念与区域聚类中心的关系,得到两者的联合概率关系表。然后利用此概率关系表标注未知标注的图像。该方法与以前的方法相比可以更加充分地结合图像的视觉特征和高层语义。通过在通用图像集上的实验结果表明,本文提出的自动图像标注方法是有效的。     

Learning semantic representations of objects and their parts

Recently, large scale image annotation datasets have been collected with millions of images and thousands of possible annotations. Latent variable models, or embedding methods, that simultaneously learn semantic representations of object labels and image representations can provide tractable solutions on such tasks. In this work, we are interested in jointly learning representations both for the objects in an image, and the parts of those objects, because such deeper semantic representations could bring a leap forward in image retrieval or browsing. Despite the size of these datasets, the amount of annotated data for objects and parts can be costly and may not be available. In this paper, we propose to bypass this cost with a method able to learn to jointly label objects and parts without requiring exhaustively labeled data. We design a model architecture that can be trained under a proxy supervision obtained by combining standard image annotation (from ImageNet) with semantic part-based within-label relations (from WordNet). The model itself is designed to model both object image to object label similarities, and object label to object part label similarities in a single joint system. Experiments conducted on our combined data and a precisely annotated evaluation set demonstrate the usefulness of our approach.     

基于判别模型与生成模型的层叠图像自动标注

图像自动标注是模式识别与计算机视觉等领域中重要而又具有挑战性的问题.针对现有模型存在数据利用率低与易受正负样本不平衡影响等问题,提出了基于判别模型与生成模型的新型层叠图像自动标注模型.该模型第一层利用判别模型对未标注图像进行主题标注,获得相应的相关图像集;第二层利用提出的面向关键词的方法建立图像与关键词之间的联系,并使用提出的迭代算法分别对语义关键词与相关图像进行扩展;最后利用生成模型与扩展的相关图像集对未标注图像进行详细标注.该模型综合了判别模型与生成模型的优点,通过利用较少的相关训练图像来获得更好的标注结果.在Corel 5K图像库上进行的实验验证了该模型的有效性.     

结合相关反馈日志与语义网络的图像标注

连接高层语义和低层视觉特征的图像语义标注技术能够很好地表示图像的语义,提出并实现了一种结合相关反馈日志与语义网络的图像标注方法。该方法以收集的用户相关反馈日志为基础获得图像的语义信息,通过计算图像间的语义相似度进行语义聚类并采用语义传播的方式实现图像的语义标注。实验结果表明,随着相关反馈日志库的不断扩充,图像库中越来越多的图像会在反馈的过程中得到标注且标注的准确率会随着反馈次数的增加而趋于稳定。     

一种基于多层语义特征的图像理解方法

视觉场景理解包括检测和识别物体、推理被检测物体之间的视觉关系以及使用语句描述图像区域.为了实现对场景图像更全面、更准确的理解,将物体检测、视觉关系检测和图像描述视为场景理解中3种不同语义层次的视觉任务,提出一种基于多层语义特征的图像理解模型,并将这3种不同语义层进行相互连接以共同解决场景理解任务.该模型通过一个信息传递图将物体、关系短语和图像描述的语义特征同时进行迭代和更新,更新后的语义特征被用于分类物体和视觉关系、生成场景图和描述,并引入融合注意力机制以提升描述的准确性.在视觉基因组和COCO数据集上的实验结果表明,所提出的方法在场景图生成和图像描述任务上拥有比现有方法更好的性能.     

Methods for automatic and assisted image annotation

Personal memories composed of digital pictures are very popular at the moment. To retrieve these media items annotation is required. During the last years, several approaches have been proposed in order to overcome the image annotation problem. This paper presents our proposals to address this problem. Automatic and semi-automatic learning methods for semantic concepts are presented. The automatic method is based on semantic concepts estimated using visual content, context metadata and audio information. The semi-automatic method is based on results provided by a computer game. The paper describes our proposals and presents their evaluations.     

Image categorization using a semantic hierarchy model with sparse set of salient regions

Image categorization in massive image database is an important problem. This paper proposes an approach for image categorization, using sparse set of salient semantic information and hierarchy semantic label tree (HSLT) model. First, to provide more critical image semantics, the proposed sparse set of salient regions only at the focuses of visual attention instead of the entire scene was formed by our proposed saliency detection model with incorporating low and high level feature and Shotton’s semantic texton forests (STFs) method. Second, we also propose a new HSLT model in terms of the sparse regional semantic information to automatically build a semantic image hierarchy, which explicitly encodes a general to specific image relationship. And last, we archived image dataset using image hierarchical semantic, which is help to improve the performance of image organizing and browsing. Extension experimental results showed that the use of semantic hierarchies as a hierarchical organizing framework provides a better image annotation and organization, improves the accuracy and reduces human’s effort.     

A probabilistic semantic model for image annotation and multi-modal image retrieval

This paper addresses automatic image annotation problem and its application to multi-modal image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 images and 7736 automatically extracted annotation words from crawled Web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature.     

基于显著区域的图像自动标注*

为了提高图像自动标注的准确率,提出了一种基于图像显著区域的自动标注方法。首先提取图像的显著区域,然后提取图像的SIFT特征,利用K-均值聚类得到视觉词汇,并根据训练图像的SIFT特征是否位于显著区域进行不同的加权运算得到视觉词汇的词袋表示,最后利用支持向量机训练分类模型实现图像分类和标注。在一个包含1 255幅Corel图像的数据库进行实验,所提方法标注的准确率与整体考虑整幅图像特征相比有很大提高,表明提出的算法优于传统方法。     

Knowledge-based region labeling for remote sensing image interpretation

The increasing availability of High Spatial Resolution (HSR) satellite images is an opportunity to characterize and identify urban objects. Thus, the augmentation of the precision led to a need of new image analysis methods using region-based (or object-based) approaches. In this field, an important challenge is the use of domain knowledge for automatic urban objects identification, and a major issue is the formalization and exploitation of this knowledge. In this paper, we present the building steps of a knowledge-base of urban objects allowing to perform the interpretation of HSR images in order to help urban planners to automatically map the territory. The knowledge-base is used to assign segmented regions (i.e. extracted from the images) into semantic objects (i.e. concepts of the knowledge-base). A matching process between the regions and the concepts of the knowledge-base is proposed, allowing to bridge the semantic gap between the images content and the interpretation. The method is validated on Quickbird images of the urban areas of Strasbourg and Marseille (France). The results highlight the capacity of the method to automatically identify urban objects using the domain knowledge.