融合深度特征和语义邻域的自动图像标注*

传统图像标注方法中人工选取特征费时费力,传统标签传播算法忽视语义近邻,导致视觉相似而语义不相似,影响标注效果.针对上述问题,文中提出融合深度特征和语义邻域的自动图像标注方法.首先构建基于深度卷积神经网络的统一、自适应深度特征提取框架,然后对训练集划分语义组并建立待标注图像的邻域图像集,最后根据视觉距离计算邻域图像各标签的贡献值并排序得到标注关键词.在基准数据集上实验表明,相比传统人工综合特征,文中提出的深度特征维数更低,效果更好.文中方法改善传统视觉近邻标注方法中的视觉相似而语义不相似的问题,有效提升准确率和准确预测的标签总数.     

基于词频同现与WordNet的图像自动标注改善算法研究

图像自动标注是模式识别与计算机视觉等领域中的重要问题。针对现有图像自动标注模型普遍受到语义鸿沟问题的影响,提出了基于关键词同现的图像自动标注改善方法,该方法利用数据集中标注词间的关联性来改善图像自动标注的结果。此外,针对上述方法不能反映更广义的人的知识以及易受数据库规模影响等问题,提出了基于语义相似的图像自动标注改善方法,通过引入具有大量词汇、包含了人知识的结构化电子词典WordNet来计算词汇间的关系并改善图像自动标注结果。实验结果表明,提出的两个图像自动标注改善方法在各项评价指标上相比以往模型均有所提高。     

Concept-based indexing of annotated images using semantic DNA

One of the challenges in image retrieval is dealing with concepts which have no visual appearance in the images or are not used as keywords in their annotations. To address this problem, this paper proposes an unsupervised concept-based image indexing technique which uses a lexical ontology to extract semantic signatures called ‘semantic chromosomes’ from image annotations. A semantic chromosome is an information structure, which carries the semantic information of an image; it is the semantic signature of an image in a collection expressed through a set of semantic DNA (SDNA), each of them representing a concept. Central to the concept-based indexing technique discussed is the concept disambiguation algorithm developed, which identifies the most relevant ‘semantic DNA’ (SDNA) by measuring the semantic importance of each word/phrase in the annotation. The concept disambiguation algorithm is evaluated using crowdsourcing. The experiments show that the algorithm has better accuracy (79.4%) than the accuracy demonstrated by other unsupervised algorithms (73%) in the 2007 Semeval competition. It is also comparable with the accuracy achieved in the same competition by the supervised algorithms (82–83%) which contrary to the approach proposed in this paper have to be trained with large corpora. The approach is currently applied to the automated generation of mood boards used as an inspirational tool in concept design.     

基于核密度估计的图像自动标注方法

提出一种基于概率模型的图像自动语义标注方法,将图片自动标注看作一个多类分类问题,通过无参数的核密度估计,实现用含有共同标注词的图片组估计视觉特征和相应标注词之间关系的机制。选取表达能力较好的基于CPAM的视觉特征,无须对图像进行语义分割处理,有效提高核密度估计的效率。在基准数据集上进行实验,结果表明,该模型能够获得比当前其他相关方法更好的标注性能。     

考虑视觉焦点权重和词相关性的图像标注方法

为了弥补图像底层特征到高层语义之间的语义鸿沟,提出一种基于视觉焦点权重模型和词相关性的图像标注方法。由于人们对图像的认识过程中,对焦点区域有比较多的关注,因此可以通过视觉焦点权重模型计算图像各区域的视觉焦点权重来提取图像的焦点区域。同时焦点区域的标注词和其他区域的标注词在逻辑上是相关的,因此通过WordNet根据词汇相关性确定图像的最终标注向量。实验结果表明,通过该方法能提高图像自动语义标注的准确率。     

一种新的基于语义聚类和图算法的自动图像标注方法

针对图像检索中的语义鸿沟问题,提出了一种新颖的自动图像标注方法。该方法首先采用了一种基于软约束的半监督图像聚类算法(SHMRF-Kmeans)对已标注图像的区域进行语义聚类,这种聚类方法可以同时考虑图像的视觉信息和语义信息。并利用图算法——Manifold排序学习算法充分发掘语义概念与区域聚类中心的关系,得到两者的联合概率关系表。然后利用此概率关系表标注未知标注的图像。该方法与以前的方法相比可以更加充分地结合图像的视觉特征和高层语义。通过在通用图像集上的实验结果表明,本文提出的自动图像标注方法是有效的。     

一种基于区域特征关联的图像语义标注方法

图像语义的标注需要解决图像高层语义和底层特征间存在的语义鸿沟。采用基于图像分割、并结合图像区域特征抽取的方法,建立图像区域语义与底层特征间的关联,采用基于距离的分类算法,计算区域特征间的相似性,并对具有相同或相近特征的区域的语义采用关联关键字的方法进行区分,用关键字实现图像语义的自动标注。     

Statistical modeling and conceptualization of natural images

Multi-level annotation of images is a promising solution to enable semantic image retrieval by using various keywords at different semantic levels. In this paper, we propose a multi-level approach to interpret and annotate the semantics of natural images by using both the dominant image components and the relevant semantic image concepts. In contrast to the well-known image-based and region-based approaches, we use the concept-sensitive salient objects as the dominant image components to achieve automatic image annotation at the content level. By using the concept-sensitive salient objects for image content representation and feature extraction, a novel image classification technique is developed to achieve automatic image annotation at the concept level. To detect the concept-sensitive salient objects automatically, a set of detection functions are learned from the labeled image regions by using support vector machine (SVM) classifiers with an automatic scheme for searching the optimal model parameters. To generate the semantic image concepts, the finite mixture models are used to approximate the class distributions of the relevant concept-sensitive salient objects. An adaptive EM algorithm has been proposed to determine the optimal model structure and model parameters simultaneously. In addition, a large number of unlabeled samples have been integrated with a limited number of labeled samples to achieve more effective classifier training and knowledge discovery. We have also demonstrated that our algorithms are very effective to enable multi-level interpretation and annotation of natural images.     

TSVM-HMM: Transductive SVM based hidden Markov model for automatic image annotation

Automatic image annotation (AIA) is an effective technology to improve the performance of image retrieval. In this paper, we propose a novel AIA scheme based on hidden Markov model (HMM). Compared with the previous HMM-based annotation methods, SVM based semi-supervised learning, i.e. transductive SVM (TSVM), is triggered out for remarkably boosting the reliability of HMM with less users’ labeling effort involved (denoted by TSVM-HMM). This guarantees that the proposed TSVM-HMM based annotation scheme integrates the discriminative classification with the generative model to mutually complete their advantages. In addition, not only the relevance model between the visual content of images and the textual keywords but also the property of keyword correlation is exploited in the proposed AIA scheme. Particularly, to establish an enhanced correlation network among keywords, both co-occurrence based and WordNet based correlation techniques are well fused and are able to be helpful for benefiting from each other. The final experimental results reveal that the better annotation performance can be achieved at less labeled training images.     

基于分类融合和关联规则挖掘的图像语义标注

图像语义自动标注问题是现阶段一个具有挑战性的难题。在跨媒体相关模型基础上,提出了融合图像类别信息的图像语义标注新方法,并利用关联规则挖掘算法改善标注结果。首先对图像进行低层特征提取,用“视觉词袋”描述图像;然后对图像特征分别进行K-means聚类和基于支持向量机的多类别分类,得到图像相似性关系和类别信息;计算语义标签和图像之间的概率关系,并将图像类别信息作为权重融合到标签的统计概率中,得到候选标注词集;最后以候选标注词概率为依据,利用改善的关联规则挖掘算法挖掘文本关联度,并对候选标注词集进行等频离散化处理,从而得到最终标注结果。在图像集Corel上进行的标注实验取得了较为理想的标注结果。     

基于随机点积图的图像标注改善算法

针对自动图像标注中底层特征和高层语义之间的鸿沟问题,提出一种基于随机点积图的图像标注改善算法。该算法首先采用图像底层特征对图像候选标注词建立语义关系图,然后利用随机点积图对其进行随机重构,从而挖掘出训练图像集中丢失的语义关系,最后采用重启式随机游走算法,实现图像标注改善。该算法结合了图像的底层特征与高层语义,有效降低了图像集规模变小对标注的影响。在3种通用图像库上的实验证明了该算法能够有效改善图像标注,宏F值与微平均F值最高分别达到0.784与0.743。     

基于概念格的视觉单词约简方法

传统的视觉单词生成方式,仅通过无监督聚类方式获得,图像语义标注的精度和效率较低。采用概念格作为视觉单词约简工具,给出了一种新的视觉单词生成方法。首先,生成训练图像BOV模型的初始视觉词典,并将其归一化形成关于训练图像BOV模型的形式背景;其次构造BOV概念格,通过概念格的属性约简,实现对视觉单词的约简,从而生成最终的视觉单词。最后,通过实例表明了该方法是有效的。     

基于相关视觉关键词的图像自动标注方法研究

图像自动标注是计算机视觉与模式识别等领域中的重要问题.针对现有模型未对文本关键词的视觉描述形式进行建模,导致标注结果中大量出现与图像视觉内容无关的标注词等问题,提出了基于相关视觉关键词的图像自动标注模型VKRAM.该模型将标注词分为非抽象标注词与抽象标注词.首先建立非抽象标注词的视觉关键词种子,并提出了一个新方法抽取非抽象标注词对应的视觉关键词集合;接着根据抽象关键词的特点,运用提出的基于减区域的算法抽取抽象关键词对应的视觉关键词种子与视觉关键词集合;然后提出一个自适应参数方法与快速求解算法用于确定不同视觉关键词的相似度阈值;最后将上述方法相结合并用于图像自动标注中.该模型能从一定程度上解决标注结果中出现的大量无关标注词问题.实验结果表明,该模型在大多数指标上相比以往模型均有所提高.     

Effective image annotation for searches using multilevel semantics

There is an increasing need for automatic image annotation tools to enable effective image searching in digital libraries. In this paper, we present a novel probabilistic model for image annotation based on content-based image retrieval techniques and statistical analysis. One key difficulty in applying statistical methods to the annotation of images is that the number of manually labeled images used to train the methods is normally insufficient. Numerous keywords cannot be correctly assigned to appropriate images due to lacking or missing information in the labeled image databases. To deal with this challenging problem, we also propose an enhanced model in which the annotated keywords of a new image are defined in terms of their similarity at different semantic levels, including the image level, keyword level, and concept level. To avoid missing some relevant keywords, the model labels the keywords with the same concepts as the new image. Our experimental results show that the proposed models are effective for annotating images that have different qualities of training data.     

Learning semantic concepts from image database with hybrid generative/discriminative approach

Semantic gap has become a bottleneck of content-based image retrieval in recent years. In order to bridge the gap and improve the retrieval performance, automatic image annotation has emerged as a crucial problem. In this paper, a hybrid approach is proposed to learn the semantic concepts of images automatically. Firstly, we present continuous probabilistic latent semantic analysis (PLSA) and derive its corresponding Expectation–Maximization (EM) algorithm. Continuous PLSA assumes that elements are sampled from a multivariate Gaussian distribution given a latent aspect, instead of a multinomial one in traditional PLSA. Furthermore, we propose a hybrid framework which employs continuous PLSA to model visual features of images in generative learning stage and uses ensembles of classifier chains to classify the multi-label data in discriminative learning stage. Therefore, the framework can learn the correlations between features as well as the correlations between words. Since the hybrid approach combines the advantages of generative and discriminative learning, it can predict semantic annotation precisely for unseen images. Finally, we conduct the experiments on three baseline datasets and the results show that our approach outperforms many state-of-the-art approaches.     

基于关键词精化和句法树的商品图像句子标注

商品图像句子标注是图像标注中一项既有趣又富有挑战的研究任务.噪声单词干扰和句法结构错误是该项研究的制约因素,针对噪声单词干扰,提出关键词精化思想：用绝对排序特征强化关键词权重,完成第1次关键词精化;计算单词的语义相关度评分,进一步优选能准确刻画图像内容的单词,完成第2次关键词精化.设计词序列\     

A visual word weighting scheme based on emerging itemsets for video annotation

The method based on Bag-of-visual-Words (BoW) deriving from local keypoints has recently appeared promising for video annotation. Visual word weighting scheme has critical impact to the performance of BoW method. In this paper, we propose a new visual word weighting scheme which is referred as emerging patterns weighting (EP-weighting). The EP-weighting scheme can efficiently capture the co-occurrence relationships of visual words and improve the effectiveness of video annotation. The proposed scheme firstly finds emerging patterns (EPs) of visual keywords in training dataset. And then an adaptive weighting assignment is performed for each visual word according to EPs. The adjusted BoW features are used to train classifiers for video annotation. A systematic performance study on TRECVID corpus containing 20 semantic concepts shows that the proposed scheme is more effective than other popular existing weighting schemes.     

图像语义标注中的叙词查询方法*

为了在图像语义标注领域能更好地反映标注之间的关系,通过对已标注图像的标注进行分析来建立标 注之间的关系,并在此基础上将叙词查询的概念引入到图像语义标注中并提出了基于叙词查询的图像语义标注 方法,把语义标注问题统一在叙词查询与图像的语义关系相结合在统一的框架下,最后通过在Ｃｏｒｅｌ图像数据库中的验证表明,所提出的方法是有效的并且标注率得到了明显的提高。