基于文本与语义相关性分析的图像检索

为了更加有效地检索到符合用户复杂语义需求的图像，提出一种基于文本描述与语义相关性分析的图像检索算法。该方法将图像检索分为两步：基于文本语义相关性分析的图像检索和基于SIFT特征的相似图像扩展检索。根据自然语言处理技术分析得到用户文本需求中的关键词及其语义关联，在选定图像库中通过语义相关性分析得到“种子”图像；接下来在图像扩展检索中，采用基于SIFT特征的相似图像检索，利用之前得到的“种子”图像作为查询条件，在网络图像库中进行扩展检索，并在结果集上根据两次检索的图像相似度进行排序输出，最终得到更加丰富有效的图像检索结果。为了证明算法的有效性，在标准数据集Corel5K和网络数据集Deriantart8K上完成了多组实验，实验结果证明该方法能够得到较为精确地符合用户语义要求的图像检索结果，并且通过扩展算法可以得到更加丰富的检索结果。     

中文文本的信息自动抽取和相似检索机制

目前信息抽取成为提供高质量信息服务的重要手段,提出面向中文文本信息的自动抽取和相似检索机制,其基本思想是将用户兴趣表示为语义模板,对关键字进行概念扩充,通过搜索引擎获得初步的候选文本集合,在概念触发机制和部分分析技术基础上,利用语义关系到模板槽的映射机制,填充文本语义模板,形成结构化文本数据库.基于文本数据表述的模糊性,给出用户查询与文本语义模板的相似关系,实现了相似检索,可以更加全面地满足用户的信息需求.     

基于查询扩展词条加权的文本检索研究

本文分析了关键词检索文本,由于其查询词没有扩展导致检全率低;而概念检索文本虽然部分有检索词扩展,但是查询词权重与原查询词没有区分.为此,本文利用词条间的语义相似度,提出一种查询扩展词条权重计算方法--展开减小法,并将查询词以及扩展词经展开减小法计算权重后构建向量空间模型检索文本.实验表明,构建的检索模型检索文本,其综合...     

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.     

The effects of multiple query evidences on social image retrieval

System performance assessment and comparison are fundamental for large-scale image search engine development. This article documents a set of comprehensive empirical studies to explore the effects of multiple query evidences on large-scale social image search. The search performance based on the social tags, different kinds of visual features and their combinations are systematically studied and analyzed. To quantify the visual query complexity, a novel quantitative metric is proposed and applied to assess the influences of different visual queries based on their complexity levels. Besides, we also study the effects of automatic text query expansion with social tags using a pseudo relevance feedback method on the retrieval performance. Our analysis of experimental results shows a few key research findings: (1) social tag-based retrieval methods can achieve much better results than content-based retrieval methods; (2) a combination of textual and visual features can significantly and consistently improve the search performance; (3) the complexity of image queries has a strong correlation with retrieval results’ quality—more complex queries lead to poorer search effectiveness; and (4) query expansion based on social tags frequently causes search topic drift and consequently leads to performance degradation.     

Multimodal retrieval with relevance feedback based on genetic programming

This paper presents a framework for multimodal retrieval with relevance feedback based on genetic programming. In this supervised learning-to-rank framework, genetic programming is used for the discovery of effective combination functions of (multimodal) similarity measures using the information obtained throughout the user relevance feedback iterations. With these new functions, several similarity measures, including those extracted from different modalities (e.g., text, and content), are combined into one single measure that properly encodes the user preferences. This framework was instantiated for multimodal image retrieval using visual and textual features and was validated using two image collections, one from the Washington University and another from the ImageCLEF Photographic Retrieval Task. For this image retrieval instance several multimodal relevance feedback techniques were implemented and evaluated. The proposed approach has produced statistically significant better results for multimodal retrieval over single modality approaches and superior effectiveness when compared to the best submissions of the ImageCLEF Photographic Retrieval Task 2008.     

融合词向量与位置信息的关键词提取算法

针对现有的基于图的关键词提取方法未能有效整合文本序列中词与词之间的潜在语义关系的问题,提出了一个融合词向量与位置信息的基于图的关键词提取算法EPRank。通过词向量表示模型学得目标文档中每个词的表示向量;将该反映词与词之间的潜在语义关系的词向量与位置特征相结合融合到PageRank评分模型中;选择几个排名靠前的单词或短语作为目标文档的关键词。实验结果表明,提出的EPRank方法在KDD和SIGIR两个数据集上的各项评估指标均高于5个现有的关键词提取方法。     

深度多模态融合服装风格检索

目的 服装检索方法是计算机视觉与自然语言处理领域的研究热点,其包含基于内容与基于文本的两种查询模态。然而传统检索方法通常存在检索效率低的问题,且很少研究关注服装在风格上的相似性。为解决这些问题,本文提出深度多模态融合的服装风格检索方法。方法 提出分层深度哈希检索模型,基于预训练的残差网络ResNet（residual network）进行迁移学习,并把分类层改造成哈希编码层,利用哈希特征进行粗检索,再用图像深层特征进行细检索。设计文本分类语义检索模型,基于LSTM（long short-term memory）设计文本分类网络以提前分类缩小检索范围,再以基于doc2vec提取的文本嵌入语义特征进行检索。同时提出相似风格上下文检索模型,其参考单词相似性来衡量服装风格相似性。最后采用概率驱动的方法量化风格相似性,并以最大化该相似性的结果融合方法作为本文检索方法的最终反馈。结果 在Polyvore数据集上,与原始ResNet模型相比,分层深度哈希检索模型的top5平均检索精度提高11.6%,检索速度提高2.57 s/次。与传统文本分类嵌入模型相比,本文分类语义检索模型的top5查准率提高29.96%,检索速度提高16.53 s/次。结论 提出的深度多模态融合的服装风格检索方法获得检索精度与检索速度的提升,同时进行了相似风格服装的检索使结果更具有多样性。     

Enhancing passage retrieval in log files by query expansion based on explicit and pseudo relevance feedback

Passage retrieval is usually defined as the task of searching for passages which may contain the answer for a given query. While these approaches are very efficient when dealing with texts, applied to log files (i.e. semi-structured data containing both numerical and symbolic information) they usually provide irrelevant or useless results. Nevertheless one appealing way for improving the results could be to consider query expansions that aim at adding automatically or semi-automatically additional information in the query to improve the reliability and accuracy of the returned results. In this paper, we present a new approach for enhancing the relevancy of queries during a passage retrieval in log files. It is based on two relevance feedback steps. In the first one, we determine the explicit relevance feedback by identifying the context of the requested information within a learning process. The second step is a new kind of pseudo relevance feedback. Based on a novel term weighting measure it aims at assigning a weight to terms according to their relatedness to queries. This measure, called TRQ (Term Relatedness to Query), is used to identify the most relevant expansion terms.The main advantage of our approach is that is can be applied both on log files and documents from general domains. Experiments conducted on real data from logs and documents show that our query expansion protocol enables retrieval of relevant passages.     

TF–DF indexing for mocap data segments in measuring relevance based on textual search queries

Many techniques have been proposed to address the problem of mocap data retrieval by using a short motion as input, and they are commonly categorized as content-based retrieval. However, it is difficult for users who do not have equipments to create mocap data samples to take advantage of them. On the contrary, simple retrieval methods which only require text as input can be used by everyone. Nevertheless, not only that it is not clear how to measure mocap data relevance in regard to textual search queries, but the search results will also be limited to the mocap data samples, the annotations of which contain the words in the search query. In this paper, the authors propose a novel method that builds on the TF (term frequency) and IDF (inverse document frequency) weights, commonly used in text document retrieval, to measure mocap data relevance in regard to textual search queries. We extract segments from mocap data samples and regard these segments as words in text documents. However, instead of using IDF which prioritizes infrequent segments, we opt to use DF (document frequency) to prioritize frequent segments. Since motions are not required as input, everybody will be able to take advantage of our approach, and we believe that our work also opens up possibilities for applying developed text retrieval methods in mocap data retrieval.     

Deep Learning and Semantic Concept Spaceare Used in Query Expansion

In the practice of information retrieval, there are some problems such as the lack of accurate expression of user query requests, the mismatch between document and query and query optimization. Focusing on these problems, we propose the query expansion method based on conceptual semantic space with deep learning, this hybrid query expansion technique include deep learning and pseudocorrelation feedback, use the deep learning and semantic network WordNet to construct query concept tree in the level of concept semantic space, the pseudo-correlation feedback documents are processed by observation window, compute the co-occurrence weight of the words by using the average mutual information and get the final extended words set. The results of experiment show that the expansion algorithm based on conceptual semantic space with deep learning has better performance than the traditional pseudo-correlation feedback algorithm on query expansion.     

教学资源概念查询扩展研究

查询扩展是在原查询词的基础上加入与用户查询词相关的词或者词组,组成新的、更准确的查询序列,使扩展后的查询序列能更清晰地表达用户的查询请求,克服自然语言的“二义性”。基于《计算机网络》概念语义网络能更加有效地找出计算机网络领域内查询词的概念词及扩展概念词,并向上拓展将各个查询词的原始语义关系联接起来,解决了查询词之间缺乏联系的问题,为扩展检索的实现奠定基础。描述了概念语义网络的生成方法、关联概念树的抽取方法和查询扩展检索的计算机实现流程,为教学资源领域的在线学习提供了技术支持。     

A survey of content-based image retrieval with high-level semantics

In order to improve the retrieval accuracy of content-based image retrieval systems, research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the ‘semantic gap’ between the visual features and the richness of human semantics. This paper attempts to provide a comprehensive survey of the recent technical achievements in high-level semantic-based image retrieval. Major recent publications are included in this survey covering different aspects of the research in this area, including low-level image feature extraction, similarity measurement, and deriving high-level semantic features. We identify five major categories of the state-of-the-art techniques in narrowing down the ‘semantic gap’: (1) using object ontology to define high-level concepts; (2) using machine learning methods to associate low-level features with query concepts; (3) using relevance feedback to learn users’ intention; (4) generating semantic template to support high-level image retrieval; (5) fusing the evidences from HTML text and the visual content of images for WWW image retrieval. In addition, some other related issues such as image test bed and retrieval performance evaluation are also discussed. Finally, based on existing technology and the demand from real-world applications, a few promising future research directions are suggested.     

Visual and textual fusion for semantically supervised region-based retrieval

This paper presents a unified annotation and retrieval framework, which integrates region annotation with image retrieval for performance reinforcement. To integrate semantic annotation with region-based image retrieval, visual and textual fusion is proposed for both soft matching and Bayesian probabilistic formulations. To address sample insufficiency and sample asymmetry in the annotation classifier training phase, we present a region-level multi-label image annotation scheme based on pair-wise coupling support vector machine (SVM) learning. In the retrieval phase, to achieve semantic-level region matching we present a novel retrieval scheme which differs from former work: the query example uploaded by users is automatically annotated online, and the user can judge its annotation quality. Based on the user’s judgment, two novel schemes are deployed for semantic retrieval: (1) if the user judges the photo to be well annotated, Semantically supervised Integrated Region Matching is adopted, which is a keyword-integrated soft region matching method; (2) If the user judges the photo to be poorly annotated, Keyword Integrated Bayesian Reasoning is adopted, which is a natural integration of a Visual Dictionary in online content-based search. In the relevance feedback phase, we conduct both visual and textual learning to capture the user’s retrieval target. Better annotation and retrieval performance than current methods were reported on both COREL 10,000 and Flickr web image database (25,000 images), which demonstrated the effectiveness of our proposed framework.     

一种基于词共现图的受限领域自动问答系统

提出了一种基于高频词和共现词的文本主题词抽取方法。该方法充分考虑到文档的统计信息和语义信息, 通过对提问问题和答案库中答案的相似度计算排序, 输出候选答案。提出一种具体的应用模型, 分别从问题的分析、信息检索和答案抽取三个模块进行系统的设计, 具有一定的应用价值。     

一个智能化的图像检索系统IIRS

本文深入研究了支持语义查询的大型图像数据库检索技术，提出了一种新的基于内容的图像检索系统：IIRS(Intelligent Irnage Retrieval System)。该系统支持多种特征的组合查询，引入了反馈机制及语义查询，具有自动学习及再学习的能力。实验结果表明，引入相关反馈及语义查询，可大大提高检索性能，     

一种基于稀疏典型性相关分析的图像检索方法

图像语义检索的一个关键问题就是要找到图像底层特征与语义之间的关联,由于文本是表达语义的一种有效手段,因此提出通过研究文本与图像两种模态之间关系来构建反映两者间潜在语义关联的有效模型的思路,基于该模型,可使用自然语言形式(文本语句)来表达检索意图,最终检索到相关图像.该模型基于稀疏典型性相关分析(sparse canonical correlation analysis,简称sparse CCA),按照如下步骤训练得到:首先利用隐语义分析方法构造文本语义空间,然后以视觉词袋(bag of visual words)来表达文本所对应的图像,最后通过Sparse CCA算法找到一个语义相关空间,以实现文本语义与图像视觉单词间的映射.使用稀疏的相关性分析方法可以提高模型可解释性和保证检索结果稳定性.实验结果验证了Sparse CCA方法的有效性,同时也证实了所提出的图像语义检索方法的可行性.     

融合Copulas理论和关联规则挖掘的查询扩展

将Copulas理论引入文本特征词关联模式挖掘,提出融合Copulas理论和关联规则挖掘的查询扩展算法.从初检文档集中提取前列n篇文档构建伪相关反馈文档集或用户相关反馈文档集,利用基于Copulas理论的支持度和置信度对相关反馈文档集挖掘含有原查询词项的特征词频繁项集和关联规则模式,从这些规则模式中提取扩展词,实现查询扩展.在NTCIR-5 CLIR中英文本语料上的实验表明,文中算法可有效遏制查询主题漂移和词不匹配问题,改善信息检索性能,提升扩展词质量,减少无效扩展词.     

An effective technique for the content based image retrieval to reduce the semantic gap based on an optimal classifier technique

Content Based Image Retrieval (CBIR) systems use Relevance Feedback (RF) in order to improve the retrieval accuracy. Research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the “semantic gap” between the visual features and the richness of human semantics. In this paper, a novel system is proposed to enhance the gain of long-term relevance feedback. In the proposed system, the general CBIR involves two steps—ABC based training and image retrieval. First, the images other than the query image are pre-processed using median filter and gray scale transformation for removal of noise and resizing. Secondly, the features such as Color, Texture and shape of the image are extracted using Gabor Filter, Gray Level Co-occurrence Matrix and Hu-Moment shape feature techniques and also extract the static features like mean and standard deviation. The extracted features are clustered using k-means algorithm and each cluster are trained using ANN based ABC technique. A method using artificial bee colony (ABC) based artificial neural network (ANN) to update the weights assigned to features by accumulating the knowledge obtained from the user over iterations. Eventually, the comparative analysis performed using the commonly used methods namely precision and recall were clearly shown that the proposed system is suitable for the better CBIR and it can reduce the semantic gap than the conventional systems.