基于视觉语义主题与反馈日志的图像自动标注

为了提高图像标注性能,提出了一种基于视觉语义主题与反馈日志的图像自动标注方法。首 先,提取图像 前景与背景区域,分别进行处理;其次,基于WordNet构建标注词之间的语义关系模型,并 结合概率潜在语义分析(PLSA) 与高斯混合模型(GMM)建立图像底层特征、视觉语义主题与标注     

基于语义的相片标注研究及应用

随着数码相机的普及应用,日益增多的相片对高效的相片管理和检索技术提出了迫切的需求。传统的基于关键字检索和基于内容检索技术都不能很好满足人们的需求,因而基于语义的标注和检索技术受到越来越多的关注。在构建的相片领域本体基础上设计了相片语义标注模板,并结合自动和半自动的语义标注技术实现了一个相片标注系统。最后通过系统的运行和分析来验证该方案的有效性。     

基于稀疏编码的图像自动标注

对于图像的自动标注,探索合适的方法能提高系统标注结果在语义范畴的正确性。该文探讨了基于稀疏编码的图像自动标注。结合近邻及统计的思想,以corel-5k原有人工标注为基础,在matlab平台上对其测试图集进行自动标注。从结果上看,稀疏编码方法准确率相比常用方法偏低,但对于图像特征的学习明显优于其他方法。因此,稀疏编码在图像的自动标注领域有可行之处。     

基于人工神经网络的图像检索平台的设计与开发

用户描述图像的高层抽象语义与图像内在的底层特征之间存在差异,此时仅依靠图像内容特征进行检索的系统无法准确完成用户的检索任务。针对以上问题,提出了使用神经网络进行图像的匹配计算方法,通过样例自动学习和用户反馈学习两种学习方式,形成图像底层特征到图像分类的正确映射,学习后的神经网络可以进行图像的自动分类及检索。该方法结合了图像的底层特征描述及用户的高层语义反馈,有效地弥补了语义鸿沟。最后,系统通过整合Web前端、图像提取模块、神经网络模块及数据库模块,实现了神经网络学习及图像检索的完整流程。     

基于图正则化与非负组稀疏的自动图像标注

设计一个稳健的自动图像标注系统的重要环节是提取能够有效描述图像语义的视觉特征。由于颜色、纹理和形状等异构视觉特征在表示特定图像语义时所起作用的重要程度不同且同一类特征之间具有一定的相关性,该文提出了一种图正则化约束下的非负组稀疏(Graph Regularized Non-negative Group Sparsity, GRNGS)模型来实现图像标注,并通过一种非负矩阵分解方法来计算其模型参数。该模型结合了图正则化与l2,1-范数约束,使得标注过程中所选的组群特征能体现一定的视觉相似性和语义相关性。在Corel5K和ESP Game等图像数据集上的实验结果表明:相较于一些最新的图像标注模型,GRNGS模型的鲁棒性更强,标注结果更精确。     

一种基于GMM的图像语义标注方法

针对图像语义概念具体语义描述的问题,提出了一种基于GMM的图像语义标注方法。该方法对于每一个语义概念分别建立基于颜色特征和纹理特征的GMM模型,利用EM算法获取关键词内容,最后融合两个GMM模型求取的概率排序结果,对未知图像进行标注。实验结果表明,提出的方法能够准确地为待标注的图像预测出若干文本关键字,有效提高图像标注的查准率和查全率。     

基于区域相似度的图像自动标注方法

面对图片的数量与种类的快速增长,如何有效地组织和处理大量的图片信息并从其中检索出用户需要的信息成为一个重要的问题。图像检索技术是解决此类问题的核心技术。为了能够有效地标注和检索图像,提出了一种基于区域匹配的图像自动标注方法,实验证明,该方法能够有效地对图像进行标注。     

基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注

图像属性标注是一种更细化的图像标注，它能缩小认知与特征间"语义鸿沟".现有研究多基于单特征且未挖掘属性蕴含的深层语义，故无法准确刻画图像内容.改进有效区域基因选择算法融合图像特征，并设计迁移学习策略，实现材质属性标注；基于判别相关分析挖掘特征间跨模态语义，以改进相对属性模型，标注材质属性蕴含的深层语义-实用属性.实验表明：材质属性标注精准度达63.11%，较最强基线提升1.97%；实用属性标注精准度达59.15%，较最强基线提升2.85%；层次化的标注结果能全面刻画图像内容.     

综合语义与颜色特征的图像检索技术研究

针对多媒体搜索引擎系统中的图像检索技术，本文提出了应用图像的高层语义特征和底层颜色特征作为图像检索的综合指标，将图像文本和视觉信息融合起来，给出了一种综合语义和颜色特征的图像检索系统的体系架构．以填补多媒体底层特征和高层语义之间的差异，并在此基础上提出了相关算法，使图像检索能够满足用户的需求．提高图像检索的效率和精度。     

基于词汇树层次语义模型的图像检索算法

解决语义鸿沟必须建立图像低层特征到高层语义的映射,针对此问题,本文提出了一种基于词汇树层次语义模型的图像检索方法.首先提取图像包含颜色信息的SIFT特征来构造图像库的特征词汇树,生成描述图像视觉信息的视觉词汇.并在此基础上利用Bayesian决策理论实现视觉词汇到语义主题信息的映射,进而构造了一个层次语义模型,并在此模型基础上完成了基于内容的语义图像检索算法.通过检索过程中用户的相关反馈,不仅可以加入正反馈图像扩展图像查询库,同时能够修正高层语义映射.实验结果表明,基于该模型的图像检索算法性能稳定,并且随着反馈次数的增加,检索效果明显提升.     

Automatic Image Annotation Based on Generalized Relevance Models

This paper presents a generalized relevance model for automatic image annotation through learning the correlations between images and annotation keywords. Different from previous relevance models that can only propagate keywords from the training images to the test ones, the proposed model can perform extra keyword propagation among the test images. We also give a convergence analysis of the iterative algorithm inspired by the proposed model. Moreover, to estimate the joint probability of observing an image with possible annotation keywords, we define the inter-image relations through proposing a new spatial Markov kernel based on 2D Markov models. The main advantage of our spatial Markov kernel is that the intra-image context can be exploited for automatic image annotation, which is different from the traditional bag-of-words methods. Experiments on two standard image databases demonstrate that the proposed model outperforms the state-of-the-art annotation models.     

Cooperative sparse representation in two opposite directions for semi-supervised image annotation

Recent studies have shown that sparse representation (SR) can deal well with many computer vision problems, and its kernel version has powerful classification capability. In this paper, we address the application of a cooperative SR in semi-supervised image annotation which can increase the amount of labeled images for further use in training image classifiers. Given a set of labeled (training) images and a set of unlabeled (test) images, the usual SR method, which we call forward SR, is used to represent each unlabeled image with several labeled ones, and then to annotate the unlabeled image according to the annotations of these labeled ones. However, to the best of our knowledge, the SR method in an opposite direction, that we call backward SR to represent each labeled image with several unlabeled images and then to annotate any unlabeled image according to the annotations of the labeled images which the unlabeled image is selected by the backward SR to represent, has not been addressed so far. In this paper, we explore how much the backward SR can contribute to image annotation, and be complementary to the forward SR. The co-training, which has been proved to be a semi-supervised method improving each other only if two classifiers are relatively independent, is then adopted to testify this complementary nature between two SRs in opposite directions. Finally, the co-training of two SRs in kernel space builds a cooperative kernel sparse representation (Co-KSR) method for image annotation. Experimental results and analyses show that two KSRs in opposite directions are complementary, and Co-KSR improves considerably over either of them with an image annotation performance better than other state-of-the-art semi-supervised classifiers such as transductive support vector machine, local and global consistency, and Gaussian fields and harmonic functions. Comparative experiments with a nonsparse solution are also performed to show that the sparsity plays an important role in the cooperation of image representations in two opposite directions. This paper extends the application of SR in image annotation and retrieval.     

基于视觉与标注相关信息的图像聚类算法

算法首先按视觉相关程度对标注字进行打分,标注字的分值体现了语义一致图像的视觉连贯程度.利用图像语义类别固有的语言描述性,从图像标注中抽取具有明显视觉连贯性的标注字作为图像的语义类别,减少了数据库设计者繁琐的手工编目工作.按标注字信息对图像进行语义分类,提高了图像聚类的语义一致性.对4500幅Corel标注图像的聚类结果证实了算法的有效性.     

A Multi-Directional Search technique for image annotation propagation

Image annotation has attracted lots of attention due to its importance in image understanding and search areas. In this paper, we propose a novel Multi-Directional Search framework for semi-automatic annotation propagation. In this system, the user interacts with the system to provide example images and the corresponding annotations during the annotation propagation process. In each iteration, the example images are clustered and the corresponding annotations are propagated separately to each cluster: images in the local neighborhood are annotated. Furthermore, some of those images are returned to the user for further annotation. As the user marks more images, the annotation process goes into multiple directions in the feature space. The query movements can be treated as multiple path navigation. Each path could be further split based on the user’s input. In this manner, the system provides accurate annotation assistance to the user - images with the same semantic meaning but different visual characteristics can be handled effectively. From comprehensive experiments on Corel and U. of Washington image databases, the proposed technique shows accuracy and efficiency on annotating image databases.     

Simultaneous image classification and annotation based on probabilistic model

The paper proposes a novel probabilistic generative model for simultaneous image classification and annotation.The model considers the fact that the category information can provide valuable information for image annotation.Once the category of an image is ascertained,the scope of annotation words can be narrowed,and the probability of generating irrelevant annotation words can be reduced.To this end,the idea that annotates images according to class is introduced in the model.Using variational methods,the approximate inference and parameters estimation algorithms of the model are derived,and efficient approximations for classifying and annotating new images are also given.The power of our model is demonstrated on two real world datasets:a 1 600-images LabelMe dataset and a 1 791-images UIUC-Sport dataset.The experiment results show that the classification performance is on par with several state-of-the-art classification models,while the annotation performance is better than that of several state-of-the-art annotation models.     

Web and personal image annotation by mining label correlation with relaxed visual graph embedding

The number of digital images rapidly increases, and it becomes an important challenge to organize these resources effectively. As a way to facilitate image categorization and retrieval, automatic image annotation has received much research attention. Considering that there are a great number of unlabeled images available, it is beneficial to develop an effective mechanism to leverage unlabeled images for large-scale image annotation. Meanwhile, a single image is usually associated with multiple labels, which are inherently correlated to each other. A straightforward method of image annotation is to decompose the problem into multiple independent single-label problems, but this ignores the underlying correlations among different labels. In this paper, we propose a new inductive algorithm for image annotation by integrating label correlation mining and visual similarity mining into a joint framework. We first construct a graph model according to image visual features. A multilabel classifier is then trained by simultaneously uncovering the shared structure common to different labels and the visual graph embedded label prediction matrix for image annotation. We show that the globally optimal solution of the proposed framework can be obtained by performing generalized eigen-decomposition. We apply the proposed framework to both web image annotation and personal album labeling using the NUS-WIDE, MSRA MM 2.0, and Kodak image data sets, and the AUC evaluation metric. Extensive experiments on large-scale image databases collected from the web and personal album show that the proposed algorithm is capable of utilizing both labeled and unlabeled data for image annotation and outperforms other algorithms.     

Image distance metric learning based on neighborhood sets for automatic image annotation

Since there is semantic gap between low-level visual features and high-level image semantic, the performance of many existing content-based image annotation algorithms is not satisfactory. In order to bridge the gap and improve the image annotation performance, a novel automatic image annotation (AIA) approach using neighborhood set (NS) based on image distance metric learning (IDML) algorithm is proposed in this paper. According to IDML, we can easily obtain the neighborhood set of each image since obtained image distance can effectively measure the distance between images for AIA task. By introducing NS, the proposed AIA approach can predict all possible labels of the image without caption. The experimental results confirm that the introduction of NS based on IDML can improve the efficiency of AIA approaches and achieve better annotation performance than the existing AIA approaches.     

Large scale automatic image annotation based on convolutional neural network

Automatic image annotation is one of the most important challenges in computer vision, which is critical to many real-world researches and applications. In this paper, we focus on the issue of large scale image annotation with deep learning. Firstly, considering the existing image data, especially the network images, most of the labels of themselves are inaccurate or imprecise. We propose a Multitask Voting (MV) method, which can improve the accuracy of original annotation to a certain extent, thereby enhancing the training effect of the model. Secondly, the MV method can also achieve the adaptive label, whereas most existing methods pre-specify the number of tags to be selected. Additionally, based on convolutional neural network, a large scale image annotation model MVAIACNN is constructed. Finally, we evaluate the performance with experiments on the MIRFlickr25K and NUS-WIDE datasets, and compare with other methods, demonstrating the effectiveness of the MVAIACNN.     

Context-aware mobile image annotation for media search and sharing

In recent years, rapid advances in media technology including acquisition, processing and distribution have led to proliferation of many mobile applications. Amongst them, one of the emerging applications is mobile-based image annotation that uses camera phones to capture images with system-suggested tags before uploading them to the media sharing portals. This procedure can offer information to mobile users and also facilitate the retrieval and sharing of the image for Web users. However, context information that can be acquired from mobile devices is underutilized in many existing mobile image annotation systems. In this paper, we propose a new mobile image annotation system that utilizes content analysis, context analysis and their integration to annotate images acquired from mobile devices. Specifically, three types of context, location, user interaction and Web, are considered in the tagging processes. An image dataset of Nanyang Technological University (NTU) campus has been constructed, and a prototype mobile image tag suggestion system has been developed. The experimental results show that the proposed system performs well in both effectiveness and efficiency on NTU dataset, and shows good potential in domain-specific mobile image annotation for image sharing.