基于语义学习的图像多模态检索

针对语义鸿沟问题,在语义学习的基础上设计图像的多模态检索系统。该系统结合3种查询方式进行图像检索。基于视觉特征的查询通过特征提取与相似度匹配进行排位。基于标签的查询建立在图像自动标注的基础上,但在语义空间之外的泛化能力较差。基于语义图例的查询能够在很大程度上克服这个缺陷,通过在显式或隐式的语义空间上进行查询,使检索结果更符合人类感知。实验结果表明,与基于纹理特征的图像检索相比,基于语义图例的检索具有更高的精度及召回率。     

基于语义分析的查询扩展方法

查询扩展是优化信息检索的有效途径。为此,提出一种基于语义分析的查询扩展方法,利用基于互信息的共现模型分析初检文档,并将其作为部分扩展源,用模型的统计结果剪枝由语义词典WordNet生成的语义树,限制扩展范围。从初检文档和语义词典两方面选取扩展词对原查询进行扩展形成新的查询集。对返回结果进行重排序,调整前n篇文档的查准率。实验证明该方法是切实可行的。     

基于本体的查询扩展研究

查询扩展是优化信息检索的一种有效方法。基于关键词的查询扩展对语义信息的忽略为结果带来了不好的影响,因而提出一种基于本体的查询扩展方法。首先建立本体模型,通过计算本体中的概念语义相似度和实例语义相似度,实现语义查询扩展。     

一种新的基于语义聚类和图算法的自动图像标注方法

针对图像检索中的语义鸿沟问题,提出了一种新颖的自动图像标注方法。该方法首先采用了一种基于软约束的半监督图像聚类算法(SHMRF-Kmeans)对已标注图像的区域进行语义聚类,这种聚类方法可以同时考虑图像的视觉信息和语义信息。并利用图算法——Manifold排序学习算法充分发掘语义概念与区域聚类中心的关系,得到两者的联合概率关系表。然后利用此概率关系表标注未知标注的图像。该方法与以前的方法相比可以更加充分地结合图像的视觉特征和高层语义。通过在通用图像集上的实验结果表明,本文提出的自动图像标注方法是有效的。     

Multimodal Retrieval using Mutual Information based Textual Query Reformulation

Multimodal Retrieval is a well-established approach for image retrieval. Usually, images are accompanied by text caption along with associated documents describing the image. Textual query expansion as a form of enhancing image retrieval is a relatively less explored area. In this paper, we first study the effect of expanding textual query on both image and its associated text retrieval. Our study reveals that judicious expansion of textual query through keyphrase extraction can lead to better results, either in terms of text-retrieval or both image and text-retrieval. To establish this, we use two well-known keyphrase extraction techniques based on tf-idf and KEA. While query expansion results in increased retrieval efficiency, it is imperative that the expansion be semantically justified. So, we propose a graph-based keyphrase extraction model that captures the relatedness between words in terms of both mutual information and relevance feedback. Most of the existing works have stressed on bridging the semantic gap by using textual and visual features, either in combination or individually. The way these text and image features are combined determines the efficacy of any retrieval. For this purpose, we adopt Fisher-LDA to adjudge the appropriate weights for each modality. This provides us with an intelligent decision-making process favoring the feature set to be infused into the final query. Our proposed algorithm is shown to supersede the previously mentioned keyphrase extraction algorithms for query expansion significantly. A rigorous set of experiments performed on ImageCLEF-2011 Wikipedia Retrieval task dataset validates our claim that capturing the semantic relation between words through Mutual Information followed by expansion of a textual query using relevance feedback can simultaneously enhance both text and image retrieval.     

A review on automatic image annotation techniques

Nowadays, more and more images are available. However, to find a required image for an ordinary user is a challenging task. Large amount of researches on image retrieval have been carried out in the past two decades. Traditionally, research in this area focuses on content based image retrieval. However, recent research shows that there is a semantic gap between content based image retrieval and image semantics understandable by humans. As a result, research in this area has shifted to bridge the semantic gap between low level image features and high level semantics. The typical method of bridging the semantic gap is through the automatic image annotation (AIA) which extracts semantic features using machine learning techniques. In this paper, we focus on this latest development in image retrieval and provide a comprehensive survey on automatic image annotation. We analyse key aspects of the various AIA methods, including both feature extraction and semantic learning methods. Major methods are discussed and illustrated in details. We report our findings and provide future research directions in the AIA area in the conclusions     

基于MPEG-7的视频语义检索系统

设计和实现一个支持语义的分布式视频检索系统:"语寻"。该系统利用一个改进的视频语义处理工具(该工具基于IBM VideoAnnEx标注工具,并增加镜头语义图标注和自然语言处理的功能)对视频进行语义分析和标注,生成包含语义信息的MPEG-7描述文件,然后对视频的MPEG-7描述文件建立分布式索引,并同时分布式存储视频文件;系统提供丰富的Web查询接口,包括关键字语义扩展查询,语义图查询以及自然语句查询,当用户提交语义查询意图后,便能够迅速地检索到感兴趣的视频和片段,并且可以浏览点播;整个系统采用分布式架构,具备良好的可扩展性,并能够支持海量视频信息的索引和检索。     

Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos

This paper targets at the problem of automatic semantic indexing of news videos by presenting a video annotation and retrieval system which is able to perform automatic semantic annotation of news video archives and provide access to the archives via these annotations. The presented system relies on the video texts as the information source and exploits several information extraction techniques on these texts to arrive at representative semantic information regarding the underlying videos. These techniques include named entity recognition, person entity extraction, coreference resolution, and semantic event extraction. Apart from the information extraction components, the proposed system also encompasses modules for news story segmentation, text extraction, and video retrieval along with a news video database to make it a full-fledged system to be employed in practical settings. The proposed system is a generic one employing a wide range of techniques to automate the semantic video indexing process and to bridge the semantic gap between what can be automatically extracted from videos and what people perceive as the video semantics. Based on the proposed system, a novel automatic semantic annotation and retrieval system is built for Turkish and evaluated on a broadcast news video collection, providing evidence for its feasibility and convenience for news videos with a satisfactory overall performance.     

Leveraging semantic resources in diversified query expansion

A search query, being a very concise grounding of user intent, could potentially have many possible interpretations. Search engines hedge their bets by diversifying top results to cover multiple such possibilities so that the user is likely to be satisfied, whatever be her intended interpretation. Diversified Query Expansion is the problem of diversifying query expansion suggestions, so that the user can specialize the query to better suit her intent, even before perusing search results. In this paper, we consider the usage of semantic resources and tools to arrive at improved methods for diversified query expansion. In particular, we develop two methods, those that leverage Wikipedia and pre-learnt distributional word embeddings respectively. Both the approaches operate on a common three-phase framework; that of first taking a set of informative terms from the search results of the initial query, then building a graph, following by using a diversity-conscious node ranking to prioritize candidate terms for diversified query expansion. Our methods differ in the second phase, with the first method Select-Link-Rank (SLR) linking terms with Wikipedia entities to accomplish graph construction; on the other hand, our second method, Select-Embed-Rank (SER), constructs the graph using similarities between distributional word embeddings. Through an empirical analysis and user study, we show that SLR ourperforms state-of-the-art diversified query expansion methods, thus establishing that Wikipedia is an effective resource to aid diversified query expansion. Our empirical analysis also illustrates that SER outperforms the baselines convincingly, asserting that it is the best available method for those cases where SLR is not applicable; these include narrow-focus search systems where a relevant knowledge base is unavailable. Our SLR method is also seen to outperform a state-of-the-art method in the task of diversified entity ranking.     

基于数据融合和相关度反馈的信息检索方法

数据融合和基于相关度反馈的查询扩展是两种有效的检索过程优化技术。前者通过集成多个检索结果提高检索性能,后者执行多次查询,依据前次结果修改/扩展用户查询,以求更好地反映用户信息需求。在混合数据融合和查询扩展技术的基础上提出一种检索过程优化方法——HQD方法,由相关度反馈结果生成多个替代查询,检索这些替代查询后采用求和余弦方法生成最终检索结果。HQD方法能有效提高检索性能。     

知识网格中基于领域本体的智能检索

本文提出知识网格环境下基于领域本体的智能检索模型,采用OWL DL语言对领域知识进行形式化描述,支持推理和深层语义检索.＂标注＂和＂查询优化＂是检索的两个关键技术.通过规范的概念和概念间语义关系对文档片段进行标注,并针对＂一词多义＂问题提出＂主题-概念＂两阶段消歧算法.＂查询优化＂过程中,基于OWL DL推理的优化算法实现查询概念的自动扩展,提高了查全率和查准率.基于以上方法,建立航天领域本体,利用网上数据库开放资源作为测试集进行评测.实验显示,与传统基于     

具有概念联想功能的语义关系库的自动构建

语义检索的关键技术就是语义扩展。文中利用基于带衰减因子的词共现模型计算公式来获得词与词之间相关性信息资源．从而给出了用于信息检索系统中的语义关系库完整的自动构建方法。将生成的语义关系库用于SMART信息检索系统中以实现语义扩展：实验结果证明：具有语义关系库的SMART信息检索系统比原不具有语义关系库的SMART信息检索系统提高了检索效率，特别是大大地提高了查全率。     

一种基于领域本体的资源反馈检索模型研究

针对现有资源平台无法互通共享资源,资源库检索系统仅依靠用户输入的单词关键字描述检索资源而无法挖 掘用户需求中的语义信息的问题,提出了一种基于本体的资源反馈检索模型。该模型通过构建本体、概念相似度计算、查询关 键字扩展等关键技术,利用了用户多次反馈中的包含语义知识,满足了用户的查询需求。实验表明,该模型能够有效提高检索 的性能。     

基于本体的林业领域语义查询扩展模型

传统信息检索模型仅考虑考虑关键词本身的匹配程度,在林业领域信息检索时得到的检索结果不全面或不准确.为了改善检索质量,提出了一种基于本体的林业领域语义查询扩展模型.该模型利用了本体的语义推理的能力和语义结构对关键词进行语义查询扩展,最终实现提高检索查全率和查准率的目的,是对传统基于关键词匹配的信息检索模型的语义补充.结果表明该模型在一定程度上改善了林业领域信息检索结果的查准率和查全率.     

Automatic localization and annotation of facial features using machine learning techniques

Content-based image retrieval (CBIR) systems traditionally find images within a database that are similar to query image using low level features, such as colour histograms. However, this requires a user to provide an image to the system. It is easier for a user to query the CBIR system using search terms which requires the image content to be described by semantic labels. However, finding a relationship between the image features and semantic labels is a challenging problem to solve. This paper aims to discover semantic labels for facial features for use in a face image retrieval system. Face image retrieval traditionally uses global face-image information to determine similarity between images. However little has been done in the field of face image retrieval to use local face-features and semantic labelling. Our work aims to develop a clustering method for the discovery of semantic labels of face-features. We also present a machine learning based face-feature localization mechanism which we show has promise in providing accurate localization.     

基于领域本体的语义查询扩展

在信息检索研究领域,资源与查询词的匹配决定信息检索质量。现有检索方法的检索结果存在过多不相关信息,不能很好满足用户需求。针对传统信息检索存在的问题与当前语义查询扩展方法的特点,本文在分析各种语义查询扩展方法及其相关研究的基础上,提出一种改进的基于领域本体的语义查询扩展方法。该方法论通过本体模型和概念相似度的计算对检索信息进行检索意图树的构建并扩展;然后在资源本体中以最短路径的方式搜索资源。实验结果表明,本文方法相较其他查询扩展方法能得到更好的检索结果。     

Deep Learning and Semantic Concept Spaceare Used in Query Expansion

In the practice of information retrieval, there are some problems such as the lack of accurate expression of user query requests, the mismatch between document and query and query optimization. Focusing on these problems, we propose the query expansion method based on conceptual semantic space with deep learning, this hybrid query expansion technique include deep learning and pseudocorrelation feedback, use the deep learning and semantic network WordNet to construct query concept tree in the level of concept semantic space, the pseudo-correlation feedback documents are processed by observation window, compute the co-occurrence weight of the words by using the average mutual information and get the final extended words set. The results of experiment show that the expansion algorithm based on conceptual semantic space with deep learning has better performance than the traditional pseudo-correlation feedback algorithm on query expansion.