A system for visualizing the viewpoints of information and itsapplication to intelligent activity support

The authors have built and evaluated several systems to assist creative concept formation by professional engineers and scientists. Through experiments, it has become clear that a system that can reveal different viewpoints automatically is strongly needed by many users to support their creative activities. We are all surrounded by an almost infinite amount of information. If we can elicit different viewpoints from large information sources, we can arrive at new understandings that could not have been possible through discussions with other persons alone. This paper presents a system that automatically elicits and visualizes different viewpoints of authors concerning certain topics from a text database of journal and conference papers. Users can review different viewpoints of a topic of interest by looking at a space configured by the system. Through interaction with the system, users are expected to build their own concepts creatively. The effect of promoting creative concept formation and other benefits, such as enhanced information retrieval and knowledge sharing, have been validated by several experiments     

一种基于TSP-KNN的事件相关故事单元检索算法

在传统的基于内容视频检索的方法中,由于视频的领域较宽,视频的低级视觉特征和高级概念之间存在着较大的语义鸿沟,常导致检索效果不佳．本文认为更有现实意义的做法是,以含有比镜头更多语义信息的事件相关故事单元为检索单位,通过提取事件相关媒体中的文本信息并利用机器学习方法自动建立事件类的模型,从而提供概念化的故事单元查询方式．本文提出了组合特征选择方法和一种二阶段修剪KNN：TSP-KNN,组合特征选择方法相对于MI方法更适合事件相关故事单元的检索．二阶段修剪KNN先对训练集进行修剪,然后再用KNN训练得到分类器,该方法解决了样本混叠以及多中心分布问题．实验结果表明所提出的方法是有效的,明显地提高了事件相关故事单元的检索性能．     

Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing

Digital video now plays an important role in medical education, health care, telemedicine and other medical applications. Several content-based video retrieval (CBVR) systems have been proposed in the past, but they still suffer from the following challenging problems: semantic gap, semantic video concept modeling, semantic video classification, and concept-oriented video database indexing and access. In this paper, we propose a novel framework to make some advances toward the final goal to solve these problems. Specifically, the framework includes: 1) a semantic-sensitive video content representation framework by using principal video shots to enhance the quality of features; 2) semantic video concept interpretation by using flexible mixture model to bridge the semantic gap; 3) a novel semantic video-classifier training framework by integrating feature selection, parameter estimation, and model selection seamlessly in a single algorithm; and 4) a concept-oriented video database organization technique through a certain domain-dependent concept hierarchy to enable semantic-sensitive video retrieval and browsing.     

基于视觉感知的图像检索的研究

基于内容图像检索的一个突出问题是图像低层特征与高层语义之间存在的巨大鸿沟.针对相关反馈和感兴趣区检测在弥补语义鸿沟时存在主观性强、耗时的缺点,提出了视觉信息是一种客观反映图像高层语义的新特征,基于视觉信息进行图像检索可以有效减小语义鸿沟;并在总结视觉感知的研究进展和实现方法的基础上,给出了基于视觉感知的图像检索在感兴趣区检测、图像分割、相关反馈和个性化检索四个方面的研究思路.     

基于语义的视频检索关键技术综述

随着大量视频的出现,视频内容检索是当今多媒体应用的一个重要研究方向。现有的视频检索技术多是基于低层特征,这些低层特征与高层语义概念相差较多,严重影响了视频内容检索系统的实用性。由于低层特征和高层语义概念间的语义鸿沟,如何从视频内容中提取人类思维中的语义概念,正成为目前视频内容检索中最具有挑战性的研究内容。文中介绍了语义视频检索出现的背景和国内外最新研究动态,分析了现有方法的优缺点,对现有的关键技术进行综述。     

Guest Editors' Introduction: The Many Faces of Semantics

The Semantic Web enables programs and agents to automatically understand what data is about, and therefore bridge the so-called semantic gap between the ways in which users request Web resources and the real needs of those users, ultimately improving the quality of Web information retrieval. This issue presents four expanded articles from The First International Workshop on the Many Faces of Multimedia Semantics.     

A unified image retrieval framework on local visual and semantic concept-based feature spaces

This paper presents a learning-based unified image retrieval framework to represent images in local visual and semantic concept-based feature spaces. In this framework, a visual concept vocabulary (codebook) is automatically constructed by utilizing self-organizing map (SOM) and statistical models are built for local semantic concepts using probabilistic multi-class support vector machine (SVM). Based on these constructions, the images are represented in correlation and spatial relationship-enhanced concept feature spaces by exploiting the topology preserving local neighborhood structure of the codebook, local concept correlation statistics, and spatial relationships in individual encoded images. Finally, the features are unified by a dynamically weighted linear combination of similarity matching scheme based on the relevance feedback information. The feature weights are calculated by considering both the precision and the rank order information of the top retrieved relevant images of each representation, which adapts itself to individual searches to produce effective results. The experimental results on a photographic database of natural scenes and a bio-medical database of different imaging modalities and body parts demonstrate the effectiveness of the proposed framework.     

A Multi-Pronged Approach to Improving Semantic Extraction of News Video

In this paper we describe a multi-strategy approach to improving semantic extraction from news video. Experiments show the value of careful parameter tuning, exploiting multiple feature sets and multilingual linguistic resources, applying text retrieval approaches for image features, and establishing synergy between multiple concepts through undirected graphical models. We present a discriminative learning framework called Multi-concept Discriminative Random Field (MDRF) for building probabilistic models of video semantic concept detectors by incorporating related concepts as well as the low-level observations. The model exploits the power of discriminative graphical models to simultaneously capture the associations of concept with observed data and the interactions between related concepts. Compared with previous methods, this model not only captures the co-occurrence between concepts but also incorporates the raw data observations into a unified framework. We also describe an approximate parameter estimation algorithm and present results obtained from the TRECVID 2006 data. No single approach, however, provides a consistently better result for all concept detection tasks, which suggests that extracting video semantics should exploit multiple resources and techniques rather than naively relying on a single approach     

A Human-Centered Multiple Instance Learning Framework for Semantic Video Retrieval

This paper proposes a human-centered interactive framework for automatically mining and retrieving semantic events in videos. After preprocessing, the object trajectories and event models are fed into the core components of the framework for learning and retrieval. As trajectories are spatiotemporal in nature, the learning component is designed to analyze time series data. The human feedback to the retrieval results provides progressive guidance for the retrieval component in the framework. The retrieval results are in the form of video sequences instead of contained trajectories for user convenience. Thus, the trajectories are not directly labeled by the feedback as required by the training algorithm. A mapping between semantic video retrieval and multiple instance learning (MIL) is established in order to solve this problem. The effectiveness of the algorithm is demonstrated by experiments on real-life transportation surveillance videos.      

Similarity-based online feature selection in content-based image retrieval.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.     

A novel long-term learning algorithm for relevance feedback in content-based image retrieval

Relevance feedback has proven to be a powerful tool to bridge the semantic gap between low-level features and high-level human concepts in content-based image retrieval (CBIR). However, traditional short-term relevance feedback technologies are confined to using the current feedback record only. Log-based long-term learning captures the semantic relationships among images in a database by analyzing the historical relevance information to boost the retrieval performance effectively. In this paper, we propose an expanded-judging model to analyze the historical log data’s semantic information and to expand the feedback sample set from both positive and negative relevant information. The index table is used to facilitate the log analysis. The expanded-judging model is applied in image retrieval by combining with short-term relevance feedback algorithms. Experiments were carried out to evaluate the proposed algorithm based on the Corel image database. The promising experimental results validate the effectiveness of our proposed expanded-judging model.     

Web信息检索中信息分类技术研究

随着Internet/Intranet的快速发展和普及,丰富的Web资源构成一个巨大的全球信息仓库。在海量数据空间中快速、准确地获取用户所需成为Web检索系统研究的焦点。将一种全新的网页自动分类技术引入WWW信息抽取领域,解决网上信息有效获取的问题。获取网站分类体系,设计的Web信息自动归类算法,可通过Web数据抽取机制以及Web信息分类技术实现检索结果的分类和层次化展示,使用户快捷准确地从WWW上获取所需信息。     

Semantic clustering for region-based image retrieval

With the proliferation of applications that demand content-based image retrieval, two merits are becoming more desirable. The first is the reduced search space, and the second is the reduced “semantic gap.” This paper proposes a semantic clustering scheme to achieve these two goals. By performing clustering before image retrieval, the search space can be significantly reduced. The proposed method is different from existing image clustering methods as follows: (1) it is region based, meaning that image sub-regions, instead of the whole image, are grouped into. The semantic similarities among image regions are collected over the user query and feedback history; (2) the clustering scheme is dynamic in the sense that it can evolve to include more new semantic categories. Ideally, one cluster approximates one semantic concept or a small set of closely related semantic concepts, based on which the “semantic gap” in the retrieval is reduced.     

Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on the semantic content rather than keywords. To enable intelligent web interactions, or even web filtering, we need to be capable of interpreting the information base in an intelligent manner. For a number of years research has been ongoing in the field of ontological engineering with the aim of using ontologies to add such (meta) knowledge to information. In this paper, we describe the architecture of a system (Dynamic REtrieval Analysis and semantic metadata Management (DREAM)) designed to automatically and intelligently index huge repositories of special effects video clips, based on their semantic content, using a network of scalable ontologies to enable intelligent retrieval. The DREAM Demonstrator has been evaluated as deployed in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. This paper provides its performance and usability results and highlights the scope for future enhancements of the DREAM architecture which has proven successful in its first and possibly most challenging proving ground, namely film production, where it is already in routine use within our test bed Partners’ creative processes.     

A unified framework for image retrieval using keyword and visual features.

In this paper, a unified image retrieval framework based on both keyword annotations and visual features is proposed. In this framework, a set of statistical models are built based on visual features of a small set of manually labeled images to represent semantic concepts and used to propagate keywords to other unlabeled images. These models are updated periodically when more images implicitly labeled by users become available through relevance feedback. In this sense, the keyword models serve the function of accumulation and memorization of knowledge learned from user-provided relevance feedback. Furthermore, two sets of effective and efficient similarity measures and relevance feedback schemes are proposed for query by keyword scenario and query by image example scenario, respectively. Keyword models are combined with visual features in these schemes. In particular, a new, entropy-based active learning strategy is introduced to improve the efficiency of relevance feedback for query by keyword. Furthermore, a new algorithm is proposed to estimate the keyword features of the search concept for query by image example. It is shown to be more appropriate than two existing relevance feedback algorithms. Experimental results demonstrate the effectiveness of the proposed framework.     

一种新闻视频文本语义信息提取与分析的新方法

本文提出一种视频文本语义信息分析的新思路,即在文本区域提取后结合文种识别理论来提取新闻视频的来源和身份等高级语义信息,同时文种识别结果可为OCR的选择提供先验知识。主要工作包括:1)针对视频中的字幕,提出一种基于时-空分析的算法来检测视频中的字幕,然后对检测到的字幕通过投影分析进行定位、增强和二值化;2)对提取到的文本区域提出一种基于PCA和小波变换的文种识别算法。     

Semantic-gap-oriented active learning for multilabel image annotation

User interaction is an effective way to handle the semantic gap problem in image annotation. To minimize user effort in the interactions, many active learning methods were proposed. These methods treat the semantic concepts individually or correlatively. However, they still neglect the key motivation of user feedback: to tackle the semantic gap. The size of the semantic gap of each concept is an important factor that affects the performance of user feedback. User should pay more efforts to the concepts with large semantic gaps, and vice versa. In this paper, we propose a semantic-gap-oriented active learning method, which incorporates the semantic gap measure into the information-minimization-based sample selection strategy. The basic learning model used in the active learning framework is an extended multilabel version of the sparse-graph-based semisupervised learning method that incorporates the semantic correlation. Extensive experiments conducted on two benchmark image data sets demonstrated the importance of bringing the semantic gap measure into the active learning process.     

Support Vector Machine Learning for Region‐Based Image Retrieval with Relevance Feedback

We present a relevance feedback approach based on multi‐class support vector machine (SVM) learning and cluster‐merging which can significantly improve the retrieval performance in region‐based image retrieval. Semantically relevant images may exhibit various visual characteristics and may be scattered in several classes in the feature space due to the semantic gap between low‐level features and high‐level semantics in the user's mind. To find the semantic classes through relevance feedback, the proposed method reduces the burden of completely re‐clustering the classes at iterations and classifies multiple classes. Experimental results show that the proposed method is more effective and efficient than the two‐class SVM and multi‐class relevance feedback methods.