图像语义标注

作为图像检索技术中重要环节的语义标注,其标注的准确度决定着最终检索效果。介绍了语义标注的基础(即语义层次模型),总结了语义标注常用的方法:人工手动标注和计算机标注系统,并且分析了两种方法的具体实现以及优缺点。     

基于本体的语义标注原型评述

实现语义Web构想的关键是利用本体词汇来标注Web资源,如Web页、服务等,基于本体的语义标注原型就是用于支持内容创建者在Web页中添加语义元数据,使其内容被人和机器所理解。本文首先简介现有基于本体的标注原型,然后从不同角度综述了各原型,并进行了对照比较,最后指出了现有原型的不足。     

图像自动语义标注技术综述

近年来,随着对基于内容图像检索技术研究的深入,图像自动语义标注已成为了该领域的研究热点。针对目前广泛研究的图像语义标注技术,从其分类、关键技术、存在问题及发展方向进行了进行了论述,以期为从事该方向研究的人员提供一定的借鉴意义和参考价值。     

数据库元数据的自动语义标注

语义异构是异构数据库信息集成中要解决的关键问题.为了使关系数据库的表和字段具有语义信息,将数据库元数据自动标注成语义元数据成为研究的热点.基于概念名和概念结构的语义相似度计算,提出了一种数据库元数据自动语义标注方法.首先从关系数据库的元数据中提取隐含的语义信息,并据此创建领域本体,然后通过计算元数据与本体实体间的语义相似度对提取的元数据进行自动语义标注,提出的相似度算法综合考虑了概念名称和结构的相似性,并采取了必要的优化措施进行改进.经实验测试证明,该方法具有较高的标注正确率,是一种行之有效的语义标注方法.     

融合语义主题的图像自动标注

由于语义鸿沟的存在,图像自动标注已成为一个重要课题.在概率潜语义分析的基础上,提出了一种融合语义主题的方法以进行图像的标注和检索.首先,为了更准确地建模训练数据,将每幅图像的视觉特征表示为一个视觉"词袋";然后设计一个概率模型分别从视觉模态和文本模态中捕获潜在语义主题,并提出一种自适应的不对称学习方法融合两种语义主题.对于每个图像文档,它在各个模态上的主题分布通过加权进行融合,而权值由该文档的视觉词分布的熵值来确定.于是,融合之后的概率模型适当地关联了视觉模态和文本模态的信息,因此能够很好地预测未知图像的语义标注.在一个通用的Corel图像数据集上,将提出的方法与几种前沿的图像标注方法进行了比较.实验结果表明,该方法具有更好的标注和检索性能.     

大规模图像语义相关性自动标注

针对大规模图像集合的自动标注问题,提出一种图像语义相关性自动标注方法.首先提取图像的视觉特征,将每个样本表示为局部邻域样本点的稀疏线性组合;然后采用一种基于最大后验概率准则的多标签学习方法得到每幅图像的单特征标签相关度;最终对单个特征和特定标签的相关度阈值进行无偏估计,并采用无监督组合方法融合多种视觉特征和标签的相关度.互联网数据集测试结果表明,该方法是有效的.     

中文篇章级句间语义关系体系及标注

篇章句间关系(Discourse Relation)是篇章级语义分析的重要内容,该文在英文篇章句间关系研究的基础上分析了中英文间的差异,总结了中文篇章级语义分析的特点,并在此基础上提出面向中文篇章句间关系的层次化语义关系体系,对句间关系类型进行详细描述。为了验证体系的合理性和完备性,我们在互联网新闻语料上进行了标注实践,分析了标注中遇到的难点并给出解决方案,为进一步的中文篇章级语义分析工作奠定基础。     

汉语逻辑补足义标注框架研究

逻辑补足义是指附加在以谓词为中心的基本命题成分之上的否定、程度、时体、模态和语气等, 具体表现为逻辑语义算子对谓词的语义约束关系,是基本命题成分所表达语义关系的有效补充。在句子中,逻辑补足义所表达的语义是句子深度语义理解的重要层面。该文以深层语义理解为目标,在逻辑补足义已有的研究基础上,建立了否定、程度、时体和语气分类体系,构建了相应的算子词典;制定标注规范,对已经标注了基本命题义语义角色的句子进行各类逻辑补足义的标注;最后,对标注的结果进行统计并对标注过程中出现的问题进行了分析。     

一种基于稀疏编码的语义标注方法

语义标注是实现语义网的一个重要研究内容,目前已有很多标注方法取得了不错的效果。但这些方法几乎都没有注意到本体所描述的知识往往稀疏地分布在文档中,也未能有效地利用文档的组织结构信息,使得这些方法对质量较差的文档的标注不理想。为此提出了一种基于稀疏编码的本体语义自动标注方法((Semantic Annotation Method based on Sparse Coding, SAMSC),该方法先按本体知识描述从文档中识别出一定的语义作为初始值,再通过迭代解析文档段落结构和描述主题,完成本体知识与文档资源的相关系数矩阵计算,最后在全局文档空间中通过最小化损失函数来实现用本体对文档的语义标注。实验表明,该方法能有效地对互联网中大量良芬不齐的文档进行自动语义标注,对质量差的文档资源能取得让人接受的结果。     

篇章连贯语义关系的自动标注方法

为实现篇章连贯语义关系的判定与自动标注,提出一种综合运用关联词多种语法信息的自动标注方法。该方法利用关联词的词性分布规则排除非关联词,标注出潜在关联词,对比关联词库中的模式表,并综合利用搭配距离、搭配强度和句法位置获取合法的篇章连贯模式,在此基础上标注出其语义关系。通过实验验证了该方法的有效性。     

基于语义分类的文物图像标注研究

由于图像数据中普遍存在的“语义鸿沟”问题，传统的基于内容的图像检索技术对于数字图书馆中的图像检索往往力不从心。而图像标注能有效地弥补语义的缺失。文中分析了图像语义标注的现状以及存在的问题，提出了基于语义分类的文物语义标注方法。算法首先通过构建一个Bayes语义分类器对待标注图像进行语义分类，进而通过在语义类内部建立基于统计的标注模型，实现了图像的语义标注。在针对文物图像进行标注的实验中，该方法获得了较好的标注准确率和效率。     

基于语义分类的文物图像标注研究

由于图像数据中普遍存在的“语义鸿沟”问题,传统的基于内容的图像检索技术对于数字图书馆中的图像检索往往力不从心。而图像标注能有效地弥补语义的缺失。文中分析了图像语义标注的现状以及存在的问题,提出了基于语义分类的文物语义标注方法。算法首先通过构建一个Bayes语义分类器对待标注图像进行语义分类,进而通过在语义类内部建立基于统计的标注模型,实现了图像的语义标注。在针对文物图像进行标注的实验中,该方法获得了较好的标注准确率和效率。     

一种自适应的Web图像语义自动标注方法

提出了一种自适应的Web图像语义自动标注方法:首先利用Web标签资源自动获取训练数据;然后通过带约束的分段惩罚加权回归模型将关联文本权重分布自适应学习和先验知识约束有机地结合在一起,实现Web图像语义的自动标注.在4 000幅从Web获得的图像数据集上的实验结果验证了该文自动获取训练集方法以及Web图像语义标注方法的有效性.     

An Integrated Framework for Semantic Annotation and Adaptation

Tools for the interpretation of significant events from video and video clip adaptation can effectively support automatic extraction and distribution of relevant content from video streams. In fact, adaptation can adjust meaningful content, previously detected and extracted, to the user/client capabilities and requirements. The integration of these two functions is increasingly important, due to the growing demand of multimedia data from remote clients with limited resources (PDAs, HCCs, Smart phones). In this paper we propose an unified framework for event-based and object-based semantic extraction from video and semantic on-line adaptation. Two cases of application, highlight detection and recognition from soccer videos and people behavior detection in domotic^* applications, are analyzed and discussed.Domotics is a neologism coming from the Latin word domus (home) and informatics.Marco Bertini has a research grant and carries out his research activity at the Department of Systems and Informatics at the University of Florence, Italy. He received a M.S. in electronic engineering from the University of Florence in 1999, and Ph.D. in 2004. His main research interest is content-based indexing and retrieval of videos. He is author of more than 25 papers in international conference proceedings and journals, and is a reviewer for international journals on multimedia and pattern recognition.Rita Cucchiara (Laurea Ingegneria Elettronica, 1989; Ph.D. in Computer Engineering, University of Bologna, Italy 1993). She is currently Full Professor in Computer Engineering at the University of Modena and Reggio Emilia (Italy). She was formerly Assistant Professor (‘93–‘98) at the University of Ferrara, Italy and Associate Professor (‘98–‘04) at the University of Modena and Reggio Emilia, Italy. She is currently in the Faculty staff of Computer Engenering where has in charges the courses of Computer Architectures and Computer Vision.Her current interests include pattern recognition, video analysis and computer vision for video surveillance, domotics, medical imaging, and computer architecture for managing image and multimedia data.Rita Cucchiara is author and co-author of more than 100 papers in international journals, and conference proceedings. She currently serves as reviewer for many international journals in computer vision and computer architecture (e.g. IEEE Trans. on PAMI, IEEE Trans. on Circuit and Systems, Trans. on SMC, Trans. on Vehicular Technology, Trans. on Medical Imaging, Image and Vision Computing, Journal of System architecture, IEEE Concurrency). She participated at scientific committees of the outstanding international conferences in computer vision and multimedia (CVPR, ICME, ICPR, ...) and symposia and organized special tracks in computer architecture for vision and image processing for traffic control. She is in the editorial board of Multimedia Tools and Applications journal. She is member of GIRPR (Italian chapter of Int. Assoc. of Pattern Recognition), AixIA (Ital. Assoc. Of Artificial Intelligence), ACM and IEEE Computer Society.Alberto Del Bimbo is Full Professor of Computer Engineering at the Università di Firenze, Italy. Since 1998 he is the Director of the Master in Multimedia of the Università di Firenze. At the present time, he is Deputy Rector of the Università di Firenze, in charge of Research and Innovation Transfer. His scientific interests are Pattern Recognition, Image Databases, Multimedia and Human Computer Interaction. Prof. Del Bimbo is the author of over 170 publications in the most distinguished international journals and conference proceedings. He is the author of the “Visual Information Retrieval” monography on content-based retrieval from image and video databases edited by Morgan Kaufman. He is Member of IEEE (Institute of Electrical and Electronic Engineers) and Fellow of IAPR (International Association for Pattern Recognition). He is presently Associate Editor of Pattern Recognition, Journal of Visual Languages and Computing, Multimedia Tools and Applications Journal, Pattern Analysis and Applications, IEEE Transactions on Multimedia, and IEEE Transactions on Pattern Analysis and Machine Intelligence. He was the Guest Editor of several special issues on Image databases in highly respected journals.Andrea Prati (Laurea in Computer Engineering, 1998; PhD in Computer Engineering, University of Modena and Reggio Emilia, 2002). He is currently an assistant professor at the University of Modena and Reggio Emilia (Italy), Faculty of Engineering, Dipartimento di Scienze e Metodi dell’Ingegneria, Reggio Emilia. During last year of his PhD studies, he has spent six months as visiting scholar at the Computer Vision and Robotics Research (CVRR) lab at University of California, San Diego (UCSD), USA, working on a research project for traffic monitoring and management through computer vision. His research interests are mainly on motion detection and analysis, shadow removal techniques, video transcoding and analysis, computer architecture for multimedia and high performance video servers, video-surveillance and domotics. He is author of more than 60 papers in international and national conference proceedings and leading journals and he serves as reviewer for many international journals in computer vision and computer architecture. He is a member of IEEE, ACM and IAPR.     

一种基于文档内容的语义标注方法

互联网上存在海量数据,如何在大量的信息中查找到有用信息就变成了一个至关重要的问题。语义网为解决这一问题带来了曙光。然而当今网络现状与语义网之间存在巨大差距,即海量非结构化的页面内容难直接转化为语义的知识。提出了一种基于文档内容的语义标注方法,利用本体所表达的语义环境,即本体知识相关词汇及其所处的语义上下文环境在文档中出现频率,实现对文档的语义标注。实验显示方法取得良好的效果,但受本体知识质量和标注文档质量两个因素影响较大。     

图像语义自动标注及其粒度分析方法

缩小图像低层视觉特征与高层语义之间的鸿沟, 以提高图像语义自动标注的精度, 进而快速满足用户检索图像的需求,一直是图像语义自动标注研究的关键. 粒度分析方法是一种层次的、重要的数据分析方法, 为复杂问题的求解提供了新的思路. 图像理解与分析的粒度不同, 图像语义标注的精度则不同, 检索的效率及准确度也就不同. 本文对目前图像语义自动标注模型的方法进行综述和分析, 阐述了粒度分析方法的思想、模型及其在图像语义标注过程中的应用, 探索了以粒度分析为基础的图像语义自动标注方法并给出进一步的研究方向.     

一种适合弱标签数据集的图像语义标注方法

真实环境下数据集中广泛存在着标签噪声问题,数据集的弱标签性已严重阻碍了图像语义标注的实用化进程.针对弱标签数据集中的标签不准确、不完整和语义分布失衡现象,提出了一种适用于弱标签数据集的图像语义标注方法.首先,在视觉内容与标签语义的一致性约束、标签相关性约束和语义稀疏性约束下,通过直推式学习填充样本标签,构建样本的近似语义平衡邻域.鉴于邻域中存在噪声干扰,通过多标签语义嵌入的邻域最大边际学习获得距离测度和图像语义的一致性,使得近邻处于同一语义子空间.然后,以近邻为局部坐标基,通过邻域非负稀疏编码获得目标图像和近邻的部分相关性,并构建局部语义一致邻域.以邻域内的语义近邻为指导并结合语境相关信息,进行迭代式降噪与标签预测.实验结果表明了方法的有效性.     

Image Annotation by Propagating Labels from Semantic Neighbourhoods

Automatic image annotation aims at predicting a set of semantic labels for an image. Because of large annotation vocabulary, there exist large variations in the number of images corresponding to different labels (“class-imbalance”). Additionally, due to the limitations of human annotation, several images are not annotated with all the relevant labels (“incomplete-labelling”). These two issues affect the performance of most of the existing image annotation models. In this work, we propose 2-pass k-nearest neighbour (2PKNN) algorithm. It is a two-step variant of the classical k-nearest neighbour algorithm, that tries to address these issues in the image annotation task. The first step of 2PKNN uses “image-to-label” similarities, while the second step uses “image-to-image” similarities, thus combining the benefits of both. We also propose a metric learning framework over 2PKNN. This is done in a large margin set-up by generalizing a well-known (single-label) classification metric learning algorithm for multi-label data. In addition to the features provided by Guillaumin et al. (2009) that are used by almost all the recent image annotation methods, we benchmark using new features that include features extracted from a generic convolutional neural network model and those computed using modern encoding techniques. We also learn linear and kernelized cross-modal embeddings over different feature combinations to reduce semantic gap between visual features and textual labels. Extensive evaluations on four image annotation datasets (Corel-5K, ESP-Game, IAPR-TC12 and MIRFlickr-25K) demonstrate that our method achieves promising results, and establishes a new state-of-the-art on the prevailing image annotation datasets.     

建模连续视觉特征的图像语义标注方法

针对图像检索中存在的"语义鸿沟"问题,提出一种对连续视觉特征直接建模的图像自动标注方法.首先对概率潜语义分析(PLSA)模型进行改进,使之能处理连续量,并推导对应的期望最大化算法来确定模型参数;然后根据不同模态数据各自的特点,提出一个对不同模态数据分别处理的图像语义标注模型,该模型使用连续PLSA建模视觉特征,使用标准PLSA建模文本关键词,并通过不对称的学习方法学习2种模态之间的关联,从而能较好地对未知图像进行标注.通过在一个包含5000幅图像的标准Corel数据集中进行实验,并与几种典型的图像标注方法进行比较的结果表明,文中方法具有更高的精度和更好的效果.