基于事件项语义图聚类的多文档摘要方法

基于事件的抽取式摘要方法一般首先抽取那些描述重要事件的句子,然后把它们重组并生成摘要。该文将事件定义为事件项以及与其关联的命名实体,并聚焦从外部语义资源获取的事件项语义关系。首先基于事件项语义关系创建事件项语义关系图并使用改进的DBSCAN算法对事件项进行聚类,接着为每类选择一个代表事件项或者选择一类事件项来表示文档集的主题,最后从文档抽取那些包含代表项并且最重要的句子生成摘要。该文的实验结果证明在多文档自动摘要中考虑事件项语义关系是必要的和可行的。     

基于衰退理论的Flickr热点事件检测方法

该文提出了一种基于衰退理论对Flickr数据进行热点事件检测的方法。该方法首先将从Flickr图像中提取的视觉词汇(Visual Words)与图像的文本信息加权合并成文档。然后训练LDA模型获得文档的主题分布作为其最终向量表示。在此基础上提出了一种改进的Single-Pass算法进行事件检测,该算法不仅考虑了图片的地理位置信息,而且基于衰退理论(Aging Theory)对检测到的事件进行生命周期建模,以便计算事件在每个时间段的能量值。最后,根据能量值进行事件排序,获得给定时间段内的热点事件。在真实Flickr数据集上的实验结果表明所提出的方法在精确率、召回率和F₁测度上优于传统事件检测方法。     

An Aging Theory for Event Life-Cycle Modeling

An event can be described by a sequence of chronological documents from several information sources that together describe a story or happening. The goal of event detection and tracking is to automatically identify events and their associated documents during their life cycles. Conventional document clustering and classification techniques cannot effectively detect and track sequential events, as they ignore the temporal relationships among documents related to an event. The life cycle of an event is analogous to living beings. With abundant nourishment (i.e., related documents for the event), the life cycle is prolonged; conversely, an event or living fades away when nourishment is exhausted. Improper tracking algorithms often unnecessarily prolong or shorten the life cycle of detected events. In this paper, we propose an aging theory to model the life cycle of sequential events, which incorporates a traditional single-pass clustering algorithm to detect and track events. Our experiment results show that the proposed method achieves a better overall performance for both long-running and short-term events than previous approaches. Moreover, we find that the aging parameters of the aging schemes are profile dependent and that using proper profile-specific aging parameters improves the detection and tracking performance further     

一种面向协作标签系统的图片检索聚类方法

为了更有效地进行图片检索,提出了一种面向Web2.0协作标签系统的图片检索聚类方法。该算法首先针对标签空间由于标签表达多样性带来的不一致问题,并通过挖掘标签间的词汇关系实现语义级查询扩展来得到语义可能相关的扩展图片结果集;然后根据标签间的相关度度量选出图片结果集中与查询标签高相关的标签集,接着采用一种自顶向下启发式的图划分算法来自动对次相关标签集进行分类。最后图片结果集即根据标签分类结果被聚类。为验证该方法的效果,从标签图片共享网站Flickr上随机下载了大量真实图片集以及所含带的标签元数据,在已实现的图片检索原型系统PivotBrowser上进行了大量实验,结果证明,该聚类算法能有效解决标签空间存在的标签表达不一致问题和标签查询歧义性问题,能提供更满意的用户检索。     

Tag-based personalized image ranking in event browsing

Many image sharing websites, e.g. Flickr, Google+ , allow users to upload images as an event, and users can browse the images others uploaded as events. The fact that people usually browse only the first few images of an event then decide whether the event is what they want makes us believe that it is necessary to present those images people favor on the very first position for each event. Here we propose a new tag- based personalized image-ranking algorithm in event browsing such that it gives image higher score if it: a) is important in the event, b) matches user’s preference. c) matches user’s query. To this end, we first adopt a local matching model to assign images an original score based on whether this image satisfies user’s query and preference. We then propose a global ranking model to take the local scores as initial values and make the salience scores iteratively smooth with respect to all images returned from the events of the query.     

一种基于感兴趣内容的图像检索方法

在基于内容图像检索研究中,针对如何提取用户感兴趣的内容以提高检索效果的问题,该文根据用户选择的相似图像自动获取“感兴趣”区域,使用Fisher判别进行区域中“感兴趣”特征分量阈值的动态选择,并以该“感兴趣”特征分量及“感兴趣”区域作为用户感兴趣的内容进行图像检索。试验结果表明了该方法的有效性。     

Multimodal Retrieval using Mutual Information based Textual Query Reformulation

Multimodal Retrieval is a well-established approach for image retrieval. Usually, images are accompanied by text caption along with associated documents describing the image. Textual query expansion as a form of enhancing image retrieval is a relatively less explored area. In this paper, we first study the effect of expanding textual query on both image and its associated text retrieval. Our study reveals that judicious expansion of textual query through keyphrase extraction can lead to better results, either in terms of text-retrieval or both image and text-retrieval. To establish this, we use two well-known keyphrase extraction techniques based on tf-idf and KEA. While query expansion results in increased retrieval efficiency, it is imperative that the expansion be semantically justified. So, we propose a graph-based keyphrase extraction model that captures the relatedness between words in terms of both mutual information and relevance feedback. Most of the existing works have stressed on bridging the semantic gap by using textual and visual features, either in combination or individually. The way these text and image features are combined determines the efficacy of any retrieval. For this purpose, we adopt Fisher-LDA to adjudge the appropriate weights for each modality. This provides us with an intelligent decision-making process favoring the feature set to be infused into the final query. Our proposed algorithm is shown to supersede the previously mentioned keyphrase extraction algorithms for query expansion significantly. A rigorous set of experiments performed on ImageCLEF-2011 Wikipedia Retrieval task dataset validates our claim that capturing the semantic relation between words through Mutual Information followed by expansion of a textual query using relevance feedback can simultaneously enhance both text and image retrieval.     

OCRS: an interactive object-based image clustering and retrieval system

In this paper, we propose an Interactive Object-based Image Clustering and Retrieval System (OCRS). The system incorporates two major modules: Preprocessing and Object-based Image Retrieval. In preprocessing, an unsupervised segmentation method called WavSeg is used to segment images into meaningful semantic regions (image objects). This is an area where a huge number of image regions are involved. Therefore, we propose a Genetic Algorithm based algorithm to cluster these images objects and thus reduce the search space for object-based image retrieval. In the learning and retrieval module, the Diverse Density algorithm is adopted to analyze the user’s interest and generate the initial hypothesis which provides a prototype for future learning and retrieval. Relevance Feedback technique is incorporated to provide progressive guidance to the learning process. In interacting with user, we propose to use One-Class Support Vector Machine (SVM) to learn the user’s interest and refine the returned result. Performance is evaluated on a large image database and the effectiveness of our retrieval algorithm is demonstrated through comparative studies.

Skew detection in document images based on rectangular active contour

The digitalization processes of documents produce frequently images with small rotation angles. The skew angles in document images degrade the performance of optical character recognition (OCR) tools. Therefore, skew detection of document images plays an important role in automatic document analysis systems. In this paper, we propose a Rectangular Active Contour Model (RAC Model) for content region detection and skew angle calculation by imposing a rectangular shape constraint on the zero-level set in Chan–Vese Model (C-V Model) according to the rectangular feature of content regions in document images. Our algorithm differs from other skew detection methods in that it does not rely on local image features. Instead, it uses global image features and shape constraint to obtain a strong robustness in detecting skew angles of document images. We experimented on different types of document images. Comparing the results with other skew detection algorithms, our algorithm is more accurate in detecting the skews of the complex document images with different fonts, tables, illustrations, and layouts. We do not need to pre-process the original image, even if it is noisy, and at the same time the rectangular content region of a document image is also detected.     

Document cleanup using page frame detection

When a page of a book is scanned or photocopied, textual noise (extraneous symbols from the neighboring page) and/or non-textual noise (black borders, speckles, ...) appear along the border of the document. Existing document analysis methods can handle non-textual noise reasonably well, whereas textual noise still presents a major issue for document analysis systems. Textual noise may result in undesired text in optical character recognition (OCR) output that needs to be removed afterwards. Existing document cleanup methods try to explicitly detect and remove marginal noise. This paper presents a new perspective for document image cleanup by detecting the page frame of the document. The goal of page frame detection is to find the actual page contents area, ignoring marginal noise along the page border. We use a geometric matching algorithm to find the optimal page frame of structured documents (journal articles, books, magazines) by exploiting their text alignment property. We evaluate the algorithm on the UW-III database. The results show that the error rates are below 4% each of the performance measures used. Further tests were run on a dataset of magazine pages and on a set of camera captured document images. To demonstrate the benefits of using page frame detection in practical applications, we choose OCR and layout-based document image retrieval as sample applications. Experiments using a commercial OCR system show that by removing characters outside the computed page frame, the OCR error rate is reduced from 4.3 to 1.7% on the UW-III dataset. The use of page frame detection in layout-based document image retrieval application decreases the retrieval error rates by 30%.     

MyOwnLife: incremental and hierarchical classification of a personal image collection on mobile devices

Browsing multimedia collection on mobile devices raises the needs for new multimedia indexing solutions. In this paper, we focus on the management of personal image collections. We propose a method to simplify the browsing task on such a collection. The contributions reside in an incremental hierarchical algorithm, a method to provide a textual representation of the groups obtained and an algorithm to build a geo-temporal view of the collection. The proposed incremental hierarchical algorithm builds a temporal tree from the time stamp of each image. We opt here for a combination of a supervised clustering and an incremental algorithm based on mixture model. Good properties of the hierarchy are determined automatically thanks to the Integrated Likelihood Criterion (ICL). Based on the events obtained, a textual representation is proposed and then used to improve our temporal classification, combining geographical and temporal information. Results are validated on several real user collections with our prototype MyOwnLife.     

复杂彩色文本图像中字符的提取

从复杂彩色文本图像中提取和识别字符已经成为一个既困难又有趣的问题。本文给出了一个具有创新性和实用性的区域生长算法用于彩色图像的分割:彩色图像游程邻接算法CRAG(color run-length adjacency graph algorithm)。我们将该算法用于彩色文本图像,首先得到图像的彩色连通域,再对这些连通域的平均颜色进行颜色聚类,可得到若干个聚类中心,然后根据不同的颜色中心将图像分为相应的彩色层面,最后通过连通域分析判断所需的文字层。该生长算法修改并扩展了传统的BAG算法,并将其运用于彩色印刷体文本图像中,充分利用了彩色图像的颜色和位置信息。实验结果表明新的方法能很好的从彩色印刷图像中提取多种常见的艺术字,并具有较高的提取速度,同时保留了文字和背景图像的原始色彩,便于将来的图像恢复。     

聚类核值相似区特征点的医学影像分类

针对传统方法通常选取角点或极值点作为特征点,忽略了局部纹理变化从而影响医学影像分类性能的问题,提出一种新的特征点检测和描述方法,并基于其应用Bag-of-Keypoints模型实现医学影像分类。首先改进自适应的K-means对影像进行像素级聚类,构建核值相似区并选取邻域内聚类分布变化急剧的像素点作为特征点,然后在极坐标系中定义特征点描述符并生成视觉词典,通过视觉词直方图描述影像,最后利用直方图交集方法度量影像间的相似度来扩展KNN完成分类。遵循IRMA的医学影像类别编码标准严格选择实验数据,结果表明该算法较传统方法F1值平均提高4.5%,对于不同类别影像效果更加稳定鲁棒,从而更好地满足临床应用需求。     

An information fusion approach to integrate image annotation and text mining methods for geographic knowledge discovery

Due to the steady increase in the number of heterogeneous types of location information on the internet, it is hard to organize a complete overview of the geospatial information for the tasks of knowledge acquisition related to specific geographic locations. The text- and photo-types of geographical dataset contain numerous location data, such as location-based tourism information, therefore defining high dimensional spaces of attributes that are highly correlated. In this work, we utilized text- and photo-types of location information with a novel approach of information fusion that exploits effective image annotation and location based text-mining approaches to enhance identification of geographic location and spatial cognition. In this paper, we describe our feature extraction methods to annotating images, and utilizing text mining approach to analyze images and texts simultaneously, in order to carry out geospatial text mining and image classification tasks. Subsequently, photo-images and textual documents are projected to a unified feature space, in order to generate a co-constructed semantic space for information fusion. Also, we employed text mining approaches to classify documents into various categories based upon their geospatial features, with the aims to discovering relationships between documents and geographical zones. The experimental results show that the proposed method can effectively enhance the tasks of location based knowledge discovery.     

Multimodal classification of events in social media

A large amount of social media hosted on platforms like Flickr and Instagram is related to social events. The task of social event classification refers to the distinction of event and non-event-related contents as well as the classification of event types (e.g. sports events and concerts). In this paper, we provide an extensive study of textual, visual, as well as multimodal representations for social event classification. We investigate the strengths and weaknesses of the modalities and study the synergy effects between the modalities. Experimental results obtained with our multimodal representation outperform state-of-the-art methods and provide a new baseline for future research.     

基于跨媒体字典的图像检索

当前存在的图像检索大多是基于内容的检索,为提高检索的准确率,通过整合文本及视觉信息,提出一种自动将文本查询转化为可视化表示的方法,实现基于跨媒体字典的图像检索。采用标注图像集挖掘文本和图像间的关系,训练建立一个类似于双语字典的跨媒体字典,自动将文本查询转化为视觉查询,分别进行基于文本和基于视觉的图像检索,将2种方法检索到的图像合并作为最终检索结果。实验结果表明,该方法能有效地提高图像的查准率。