User specific context construction for personalized multimedia retrieval

The onset of Web 2.0 has given the freedom of tagging to the users. The popularization of social networking and the expansion of the smartphone market in the past decade has led to an increase of data being accumulated on the social media platforms, particularly images and videos. The exponential and ever increasing data have made information retrieval cumbersome, especially for the social network users, and this has turned out to be a huge challenge in the evolution of algorithms and technologies. In this paper, we present a novel framework and techniques for retrieving user’s multimedia content like images from the user’s profile using the context of the image/media file. We apply the Logical Itemset mining on the image Metadata consisting of the textual data (Hashtags, Caption, Date and Time) associated with the images. Through this work, we intend to bridge the semantic gap between the images and the data representation that the user associated with them. Our framework also addresses the paraphrase problem of variation in words (synonyms) used to describe a context of a media file. To evaluate the applicability of our framework, we performed tests on large Instagram image dataset extracted from various user profiles containing monolingual metadata, which show promising results for real-time applications. Furthermore, we compare and evaluate our framework with another context-based image retrieval framework, Krumbs.     

From multimedia retrieval to knowledge management

Developments in information retrieval technologies can make multimedia data as pervasive and important as textual sources in knowledge management systems. The authors suggest ways in which speech-based multimedia information retrieval technologies can evolve into full-fledged knowledge management systems     

Personalization in multimedia retrieval: A survey

With the explosive broadcast of multimedia (text documents, image, video etc.) in our life, how to annotate, search, index, browse and relate various forms of information efficiently becomes more and more important. Combining these challenges by relating them to user preference and customization only complicates the matter further. The goal of this survey is to give an overview of the current situation in the branches of research that are involved in annotation, relation and presentation to a user by preference. This paper will present some current models and techniques being researched to model ontology, preference, context, and presentation and bring them together in a chain of ideas that leads from raw uninformed data to an actual usable user interface that adapts with user preference and customization.     

一种基于用户配置文件的个性化检索方法*

为了进一步改进个性化搜索方法,通过对现有个性化搜索方法的研究,提出了一种新的搜索方法。该方法从用户兴趣相关性出发,将用户配置文件与传统个性化搜索相结合。在传统TF-IDF方法的基础上,提出了一种综合考虑标签总引用次数和配置文件中标签总数的新方法,用于获取用户配置文件与资源配置文件中的标签权重;设计了基于余弦相似性计算并综合匹配的标签个数的资源相关性计算方法。通过MovieLens数据集试验,验证了本文提出方法的有效性。     

基于内容的多媒体检索技术

基于内容的多媒体信息检索在图像检索中意义重大,其检索的依据主要是多媒体的特征向量值,因此多媒体信息特别是图像的特征向量的组织直接影响到数据检索的效率.对基于内容的多媒体检索技术进行了研究,同时借鉴基于关键字的检索技术,引进了二叉排序树来组织图像的特征向量,利用成熟的二叉排序树算法进行特征向量值的检索,提高了图像检索的效率.     

A survey on content-based retrieval for multimedia databases

Conventional database systems are designed for managing textual and numerical data, and retrieving such data is often based on simple comparisons of text/numerical values. However, this simple method of retrieval is no longer adequate for multimedia data, since the digitized representation of images, video, or data itself does not convey the reality of these media items. In addition, composite data consisting of heterogeneous types of data also associates with the semantic content acquired by a user's recognition. Therefore, content-based retrieval for multimedia data is realized taking such intrinsic features of multimedia data into account. Implementation of the content-based retrieval facility is not based on a single fundamental, but is closely related to an underlying data model, a priori knowledge of the area of interest, and the scheme for representing queries. This paper surveys recent studies on content-based retrieval for multimedia databases from the point of view of three fundamental issues. Throughout the discussion, we assume databases that manage only nontextual/numerical data, such as image or video, are also in the category of multimedia databases     

A research on social network-based personalized semantic retrieval model

Personalized semantic retrieval extends the query process and optimizes query results by mapping user preference of information to ontology. It can fetch different results according to the same queries from different users. This paper proposes a personalized semantic retrieval model based on social network. It implements the organization, presentation, acquisition and maintenance of user preference data. Finally, it uses these personalization data in the process of information retrieval.     

A personalized query expansion approach for engineering document retrieval

Engineers create engineering documents with their own terminologies, and want to search existing engineering documents quickly and accurately during a product development process. Keyword-based search methods have been widely used due to their ease of use, but their search accuracy has been often problematic because of the semantic ambiguity of terminologies in engineering documents and queries. The semantic ambiguity can be alleviated by using a domain ontology. Also, if queries are expanded to incorporate the engineer’s personalized information needs, the accuracy of the search result would be improved. Therefore, we propose a framework to search engineering documents with less semantic ambiguity and more focus on each engineer’s personalized information needs. The framework includes four processes: (1) developing a domain ontology, (2) indexing engineering documents, (3) learning user profiles, and (4) performing personalized query expansion and retrieval. A domain ontology is developed based on product structure information and engineering documents. Using the domain ontology, terminologies in documents are disambiguated and indexed. Also, a user profile is generated from the domain ontology. By user profile learning, user’s interests are captured from the relevant documents. During a personalized query expansion process, the learned user profile is used to reflect user’s interests. Simultaneously, user’s searching intent, which is implicitly inferred from the user’s task context, is also considered. To retrieve relevant documents, an expanded query in which both user’s interests and intents are reflected is then matched against the document collection. The experimental results show that the proposed approach can substantially outperform both the keyword-based approach and the existing query expansion method in retrieving engineering documents. Reflecting a user’s information needs precisely has been identified to be the most important factor underlying this notable improvement.     

基于P2P的个性化Web信息检索

为了克服Web搜索引擎在可扩展性、协作性和个性化等方面存在的不足,提出了一种基于Peer to Peer 的全分布、协作式、自组织的个性化Web信息检索,定义了以查询主题为中心进行主题聚类、数据组织和查询路由的用户协作共享策略,设计了协作生成用户兴趣列表向量、对相似语义查询进行主题聚类和更新、基于查询集建立倒排索引以及基于查询主题进行语义路由等算法和机制,以提供人性化、协作式、个性化的搜索。模拟实验表明,原型系统可以加快查询速度,减轻网络负荷,提高搜索的准确率。     

多媒体信息检索研究与展望

对近几年在多媒体信息检索领域的研究成果进行分析,总结了多媒体信息检索的研究现状,指出了该研究领域的发展方向,最后提出了多媒体信息检索技术研究面临的挑战.     

Instance based personalized multi-form image browsing and retrieval

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.     

mwKAT: A meta-tool-generated knowledge acquisition tool for multimedia workstations development

mwKAT is an interactive knowledge acquisition tool for acquiring domain knowledge about multimedia components. It constructs knowledge bases for a consulting system that produces the design specification for a multimedia workstation according to the user requirements.mwKAT is generated from and executed inGAS, a primitives-based generic knowledge acquisition meta-tool. It contains three acquisition primitives, namely, parameter proposing, constraint proposing, and fix proposing to construct an intermediate knowledge base represented by a dependency model. These primitives identify necessary domain knowledge and guide users to propose significant components, constraints, and fix methods into the dependency model.mwKAT also invokes knowledge verification and validation primitives to verify the completeness, consistency, compilability, and correctness of the intermediate knowledge base.     

A multimedia retrieval framework based on semi-supervised ranking and relevance feedback

We present a new framework for multimedia content analysis and retrieval which consists of two independent algorithms. First, we propose a new semi-supervised algorithm called ranking with Local Regression and Global Alignment (LRGA) to learn a robust Laplacian matrix for data ranking. In LRGA, for each data point, a local linear regression model is used to predict the ranking scores of its neighboring points. A unified objective function is then proposed to globally align the local models from all the data points so that an optimal ranking score can be assigned to each data point. Second, we propose a semi-supervised long-term Relevance Feedback (RF) algorithm to refine the multimedia data representation. The proposed long-term RF algorithm utilizes both the multimedia data distribution in multimedia feature space and the history RF information provided by users. A trace ratio optimization problem is then formulated and solved by an efficient algorithm. The algorithms have been applied to several content-based multimedia retrieval applications, including cross-media retrieval, image retrieval, and 3D motion/pose data retrieval. Comprehensive experiments on four data sets have demonstrated its advantages in precision, robustness, scalability, and computational efficiency.     

基于概念的多媒体数据语义检索模型*

基于概念的检索是多媒体数据语义检索的解决途径之一。对概念进行了语义扩展,提出了基于概念的多媒体数据语义检索模型,包括人机接口、知识获取、数据获取、概念检索以及语义获取等模块,并对各模块所涉及到的问题进行了分析和探讨。     

个性化跨语言信息检索中结果重排序研究

目前,Web的不断发展使得针对其内容搜索的精确度有所降低,尤其在不同的语言中进行搜索时,情况变得愈发复杂。跨语言信息检索提供了一种跨越语言障碍、获取信息的有效方法。以往的跨语言信息检索研究大多采取以检索系统为中心的研究方法,并未考虑到用户在翻译和检索过程中发挥的作用。结果重排序技术已经广泛应用于单语个性化信息检索,但是在个性化跨语言信息检索中还较少有相关研究。通过结果重排序技术来研究个性化跨语言信息检索,提出了两种个性化跨语言结果重排序方法。一种基于隐含语义,而另外一种则基于外部语义进行,将首轮搜索结果根据用户的偏好进一步进行处理和优化,使用户感兴趣的内容置于搜索结果列表的前列。在真实用户搜索日志数据上的实验结果表明,结果重排序能够有效提高个性化跨语言信息检索的搜索准确率。