Constructing and application of multimedia TV-news archives

This paper addresses an integrated information mining techniques for broadcasting TV-news. This utilizes technique from the fields of acoustic, image, and video analysis, for information on news story title, newsman and scene identification. The goal is to construct a compact yet meaningful abstraction of broadcast TV-news, allowing users to browse through large amounts of data in a non-linear fashion with flexibility and efficiency. By adding acoustic analysis, a news program can be partitioned into news and commercial clips, with 90% accuracy on a data set of 400 h TV-news recorded off the air from July 2005 to August 2006. By applying speaker identification and/or image detection techniques, each news stories can be segmented with a better accuracy of 95.92%. On-screen captions or subtitles are recognized by OCR techniques to produce the text title of each news stories. The extracted title words can be used to link or to navigate more related news contents on the WWW. In cooperation with facial and scene analysis and recognition techniques, OCR results can provide users with multimodal query on specific news stories. Some experimental results are presented and discussed for the system reliability, performance evaluation and comparison.     

Video OCR: indexing digital news libraries by recognition of superimposed captions

The automatic extraction and recognition of news captions and annotations can be of great help locating topics of interest in digital news video libraries. To achieve this goal, we present a technique, called Video OCR (Optical Character Reader), which detects, extracts, and reads text areas in digital video data. In this paper, we address problems, describe the method by which Video OCR operates, and suggest applications for its use in digital news archives. To solve two problems of character recognition for videos, low-resolution characters and extremely complex backgrounds, we apply an interpolation filter, multi-frame integration and character extraction filters. Character segmentation is performed by a recognition-based segmentation method, and intermediate character recognition results are used to improve the segmentation. We also include a method for locating text areas using text-like properties and the use of a language-based postprocessing technique to increase word recognition rates. The overall recognition results are satisfactory for use in news indexing. Performing Video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.     

一种基于K-L变换和聚类的视频摘要方法*

提出了一种基于K-L变换和聚类的视频摘要方法。首先通过对视频帧原始RGB空间进行K-L变换,得到由主轴构成的参数模型;其次运用滑动窗口法进行镜头检测;再次,根据最邻近规则对每个镜头的视频帧进行聚类;最后通过后处理优化聚类结果,提取最靠近聚类中心的帧作为关键帧,组成视频摘要。以新闻视频为例,实验结果证明了算法的有效性。     

视频摘要技术综述

目的 类似于文本摘要,视频摘要是对视频内容的总结。为了合理地评估视频摘要领域的研究进展,正确导向视频摘要的继续研究,本文归纳总结视频摘要技术的主要研究方法和显著性成果,对视频摘要技术进行综述。方法 依据视频摘要的两个主要生成步骤:视频内容分析和摘要生成分别介绍视频摘要的主要研究方法。同时,分析了近5年视频摘要领域的研究状况,对视频摘要发展的新趋势:实时视频摘要和多视角视频摘要进行了阐述。最后,还对视频摘要的评价系统进行了分类总结。结果 对视频摘要进行综述,对摘要中的语义获取难题提出了2种指导性建议。并依据分析结果,展望了视频摘要技术未来的发展方向。结论 视频摘要技术作为视频内容理解的重要组成部分,有较大研究价值。而目前,视频摘要在视频语义表达和摘要评价系统方面并不精确完善,还需进一步的深入研究。     

新闻视频主持人镜头检测方法

提出用于新闻视频主持人镜头检测的基于扩展人脸区域的模板匹配方法。利用一段新闻节目中主持人服装不变的重要线索,在线提取主持人镜头的扩展人脸区域模板,采用分块HSV颜色直方图作为模板参数,利用该模板对新闻视频镜头中检测出的扩展人脸区域进行匹配,根据匹配结果确定主持人镜头。实验结果表明,该方法计算简单、检测精度高、实时性强,具有很好的通用性。     

A framework for video abstraction systems analysis and modelling from an operational point of view

Nowadays the huge amount of video material stored in multimedia repositories makes its search and retrieval a very slow and usually difficult task. Existing video abstraction systems aim to relieve this problem by providing short versions of the original content which ease the search and navigation processes and reduce the browsing time. There are many approaches for video abstraction based on the optimal selection and presentation of a subset of fragments (keyframes, shots, etc.) from the original video attending to different criteria, usually dependent on the application scenario. Nevertheless, given the huge size and growth rate of existing video repositories there is an increasing need for providing efficient techniques. This paper presents a unified taxonomy and a generic architectural model aimed for the study of existing abstraction systems computational performance and characteristics. The taxonomy has been developed taking into account and identifying the operative characteristics of current state of the art video abstraction techniques. The proposed video abstraction architecture model characterizes the stages needed to build a generic abstraction process and establishes the basic architectural aspects and requirements for the modeling of systems with specific operative requirements.     

Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos

This paper targets at the problem of automatic semantic indexing of news videos by presenting a video annotation and retrieval system which is able to perform automatic semantic annotation of news video archives and provide access to the archives via these annotations. The presented system relies on the video texts as the information source and exploits several information extraction techniques on these texts to arrive at representative semantic information regarding the underlying videos. These techniques include named entity recognition, person entity extraction, coreference resolution, and semantic event extraction. Apart from the information extraction components, the proposed system also encompasses modules for news story segmentation, text extraction, and video retrieval along with a news video database to make it a full-fledged system to be employed in practical settings. The proposed system is a generic one employing a wide range of techniques to automate the semantic video indexing process and to bridge the semantic gap between what can be automatically extracted from videos and what people perceive as the video semantics. Based on the proposed system, a novel automatic semantic annotation and retrieval system is built for Turkish and evaluated on a broadcast news video collection, providing evidence for its feasibility and convenience for news videos with a satisfactory overall performance.     

INFORMATION FILTERING OF ON-LINE NEWS USING DYNAMIC ABSTRACT GENERATION

With the information explosion from the Internet, there is a need to efficiently determine the relevance of information. This paper discusses an approach to information filtering using dynamic abstract generation techniques. Different abstract generation techniques such as the location method, indicative-phrases, keyword frequency, and title-keyword method are incorporated into a retrieval interface for on-line news articles. During news retrieval, abstract generation, an extract containing a set of verbatim sentences from the news article will be automatically produced. This will form an indicative abstract from which the prospective reader can then decide whether to read the full-length news article. In this way, a reader can filter out irrelevant news articles without having to review the entire article.     

Automatic composition techniques for video production

Video production involves the process of capturing, editing and composing video segments for delivery to a consumer. A composition must yield a coherent presentation of an event or narrative. This process can be automated if appropriate domain-specific metadata are associated with video segments and composition techniques are established. Automation leads to the support of dynamic composition and customization for applications such as news on demand. In this paper, we present techniques to achieve dynamic, real-time and cohesive video composition and customization. We also identify metrics for evaluating our techniques with respect to existing manually produced video-based news. The results of such an evaluation show that the quality of automatic composition is comparable to-and in some cases, better than-broadcast news video composition. The results also validate the assertions on which the automatic composition techniques are based     

Automatic and fast temporal segmentation for personalized news consuming

Automatic news program segmentation and classification becomes a hot topic, which reorganizes the news program according to the news’ topics, and provides the on-demand services to mobile consumers or Internet/home TV consumers. This paper presents a personalized news consuming system, including the system architecture, consumption steps and key techniques. Then, focused on the core technique, i.e., video temporal segmentation, the automatic video temporal segmentation method is proposed, evaluated and compared with existing ones. Experimental results show that the proposed scheme is computational efficient and gets higher correct detection rate. These properties make it a suitable choice for the personalized news consuming system.     

新闻视频中口播帧检测方法的研究

新闻视频分析是视频分析领域的重要课题.提出了一种基于知识的新闻视频分析方法——二阶段模板匹配法,用于检测新闻节目中主持人口播镜头,从而为新闻单元的定位提供基本依据.该方法具有通用性和实时性的特点,可以在新闻视频的自动分析或自动索引系统中得到实际应用.     

新闻视频挖掘技术研究

新闻视频挖掘是一个新兴的研究领域,也是多媒体数据挖掘的典型代表。本文对新闻视频挖掘技术进行了全面深入的讨论,首先从概念上对新闻视频挖掘进行了界定,提出了新闻视频挖掘的层次框架和技术框架,指出新闻视频挖掘包括低层视频挖掘和高层视频挖掘两个层次。其中,低层视频挖掘是利用数据挖掘的方法对视频内容进行分析的过程,而高层数据挖掘则是在低层挖掘的基础上进一步发现视频中的知识的过程。新闻视频挖掘的技术框架则对挖掘所涉及到的具体技术进行了分析。最后,对新闻视频挖掘中的结构挖掘、语义内容挖掘、视频摘要、趋势挖掘、关联挖掘等任务进行了详细的阐述,并对各种任务举出了具体的示例加以说明。     

基于流形学习和混合模型的视频摘要方法

视频摘要是进行视频浏览、视频检索、视频索引等视频应用的前提,而且视频摘要类似于文本的摘要,也是对视频内容的一个简短概括。为了自动获得既包含视频的主要信息,而冗余信息又少的视频摘要,提出了一种基于流形学习和有限混合模型的自动视频摘要方法。该方法通过对视频序列进行流形建模,首先得到视频场景的初次分割;然后对包含内容较多的场景,使用等距降维方法计算视频帧的特征向量;最后将视频帧的特征向量输入到混合模型进行聚类分析,得到更细粒度的摘要结果。为了实现视频摘要的自动处理,所采用的混合模型需要具有模型选择功能。混合模型的聚类结果和流形建模的结果共同构成了视频摘要。视频分割片段的实验结果表明,在不需人为干预的情况下,所提供的视频摘要不仅包含视频主要内容,而且冗余信息少。     

基于内容的新闻视频检索技术研究

新闻视频的检索具有较大的实用意义。本文结合新闻视频的层次结构,从具体的每一步骤对现有的基于内容的新闻视频检索的常用技术,尤其是关于利用音视特征来进行新闻单元分割,进行了总结和探讨比较,指出了目前研究中存在的主要问题并提出了今后的研究方向。     

多模态特征融合的新闻视频故事分割方法

新闻视频故事分割是新闻视频分析的重要底层支持技术,本文提出了一种融合音频、视频等多模态特征的新闻视频故事分割方法.首先分析音频特征的静音片段作为音频特征候选点,对视频进行镜头分割,并将镜头分割结果分类为播音员镜头和新闻报道镜头,将所有的镜头分割点和播音员镜头片段提取为视频片段候选点;然后通过对新闻视频编辑规则的研究,对视频、音频特征候选点融合分析来获取新闻视频的故事分割,实验表明该方法在不同新闻视频编辑规则下都具有较好的分割效率.     

基于主题音乐和镜头切换检测新闻提要

新闻提要是一段新闻的简要介绍,它高度概括了整段新闻的主要内容,有效地从新闻视频中提取新闻摘要对于基于内容建立新闻视频数据库非常重要。通过分析新闻视频的特点,给出了详细的检测算法并通过实验对算法进行了检验。     

Semantic Based Prefetching in News-on-Demand Video Servers

In video databases, a video document has two abstractions. The high level abstraction corresponds to the view in which the contents of that video document are seen by end users, and the low level abstraction corresponds to the physical organization of that video document. Due to the huge size of continuous data, reducing I/O has become a key issue. The latter has been mostly addressed by developing appropriate buffering techniques. In addition, prefetching techniques play a major role to meet the video data requirements. In this paper, we propose a novel prefetching strategy based not only on run-time information (objects access frequencies for example) but also on knowledge about clips structures. The proposed technique merges the two views of a video document to trigger prefetching at the video server level. Simulation experiments for a News-on-Demand application performed on different request scenarios show an improvement of about 18% in the buffer hit-rate with respect, first to the available buffer size and second to the request arrival rate.     

基于领域本体的新闻视频检索

新闻视频作为视频数据中有代表性的一种媒体,受到人们的广泛关注,对新闻视频的检索要求也越来越高.传统的新闻视频检索大多是非语义层面的,采用的是基于关键词的检索方法，难于获得令人满意的查准率和查全率．本文提出一种基于领域本体的新闻视频检索框架，定义了新闻视频检索中的新闻视频对象，使用语义表达能力强的领域本体来指导视频语义对象的标注，并针对“一词多义”问题提出了“概念域-概念”两阶段概念消歧算法；针对自然语言检索问题，使用领域本体进行查询优化和查询扩展，并提出了查询语句自动生成方法．实验表明，基于领域本体的新闻视频检索方法可以有效的提高检索性能．     

新闻视频的场景分段索引及摘要生成

在提出一种新闻视频检索系统结构的基础上，介绍了基于标题条检测的新闻场景分段索引方法，并阐述了两种新闻摘要的生成策略．该方法利用新闻节目标题条出现的时空位置线索，结合标题关键词的识别，建立起一个层次化的新闻视频索引结构，并帮助用户通过新闻摘要实现不同要求的视频浏览．实验证明该方法有较高的检索成功率，并且简单快捷，为新闻视频检索提供了新的有效途径．     

一种多模态融合新闻视频条目分割算法

新闻视频条目分割是新闻视频检索和浏览中重要的底层支持技术,本文提出了一种融合主持人模板匹配和主题字幕帧检测的多模态新闻视频条目分割算法。先用基于主持人模板的算法进行第一次分割,再用基于改进的字幕检测方法进行第二次分割,最后将两次分割的结果融合并去除重复的分割点。实验证明,该算法对新闻视频条目分割具有较好的效果。