A motion-based scene tree for browsing and retrieval of compressed videos

This paper describes a fully automatic content-based approach for browsing and retrieval of MPEG-2 compressed video. The first step of the approach is the detection of shot boundaries based on motion vectors available from the compressed video stream. The next step involves the construction of a scene tree from the shots obtained earlier. The scene tree is shown to capture some semantic information as well as to provide a construct for hierarchical browsing of compressed videos. Finally, we build a new model for video similarity based on global as well as local motion associated with each node in the scene tree. To this end, we propose new approaches to camera motion and object motion estimation. The experimental results demonstrate that the integration of the above techniques results in an efficient framework for browsing and searching large video databases.     

基于视频对象的视频内容分级描述模型

本文提出了一种基于视频对象的视频内容分级描述模型.视频序列首先被分成一个个的镜头,在每个镜头内对视频对象进行分割和跟踪.按照镜头、视频对象、视频对象平面和视频对象区域四级框架提取特征,对视频内容进行分级描述.本文对视频内容信息的描述可以用于视频检索、视频注释等应用.     

Keyframe-based video summarization using Delaunay clustering

Recent advances in technology have made tremendous amounts of multimedia information available to the general population. An efficient way of dealing with this new development is to develop browsing tools that distill multimedia data as information oriented summaries. Such an approach will not only suit resource poor environments such as wireless and mobile, but also enhance browsing on the wired side for applications like digital libraries and repositories. Automatic summarization and indexing techniques will give users an opportunity to browse and select multimedia document of their choice for complete viewing later. In this paper, we present a technique by which we can automatically gather the frames of interest in a video for purposes of summarization. Our proposed technique is based on using Delaunay Triangulation for clustering the frames in videos. We represent the frame contents as multi-dimensional point data and use Delaunay Triangulation for clustering them. We propose a novel video summarization technique by using Delaunay clusters that generates good quality summaries with fewer frames and less redundancy when compared to other schemes. In contrast to many of the other clustering techniques, the Delaunay clustering algorithm is fully automatic with no user specified parameters and is well suited for batch processing. We demonstrate these and other desirable properties of the proposed algorithm by testing it on a collection of videos from Open Video Project. We provide a meaningful comparison between results of the proposed summarization technique with Open Video storyboard and K-means clustering. We evaluate the results in terms of metrics that measure the content representational value of the proposed technique.     

A two-level queueing system for interactive browsing and searching of video content

This paper presents a two-level queueing system for dynamic summarization and interactive searching of video content. Video frames enter the queueing system; some insignificant and redundant frames are removed; the remaining frames are pulled out of the system as top-level key frames. Using an energy-minimization method, the first queue removes the video frames that constitute the gradual transitions of video shots. The second queue measures the content similarity of video frames and reduces redundant frames. In the queueing system, all key frames are linked in a directed-graph index structure, allowing video content to be accessed at any level-of-detail. Furthermore, this graph-based index structure enables interactive video content exploration, and the system is able to retrieve the video key frames that complement the video content already viewed by users. Experimental results on four full-length videos show that our queueing system performs much better than two existing methods on video key frame selection at different compression ratios. The evaluation on video content search shows that our interactive system is more effective than other systems on eight video searching tasks. Compared with the regular media player, our system reduces the average content searching time by half.     

Searching and editing MPEG-compressed video in a distributed online environment

WebClip (on-line demo at http://www.ctr.columbia.edu/webclip) is a compressed video searching and editing system operating over the World Wide Web. WebClip uses a distributed client-server model including a server engine for content analysis/editing, and clients for interactive controls of video browsing/editing. It specializes several unique features, including compressed-domain video feature extraction and manipulation, multi-resolution video access, content based video browsing/retrieval, and a distributed network architecture.     

Video collage: presenting a video sequence using a single image

The explosive growth of video data demands the video presentation technique which supports fast browsing of video content. In this paper, we present an automatic procedure for constructing a compact synthesized collage from a video sequence. The synthesized image, called “Video Collage”, is a kind of static video summary—to select the most representative images from video, to extract salient regions of interest (ROIs) from these images, and to seamlessly arrange ROIs on a given canvas with the temporal structure of video content preserved. We formulate the generation of Video Collage as a unified energy minimization problem in which each of above desirability is represented by an energy term. We start from the basic setting of Video Collage in which both the shape of ROIs and collage are fixed as rectangular, and then show how it can support arbitrary shapes of ROIs, as well as a variety of collage templates and region of interest (ROI) arrangement layouts (i.e., book, diagonal, and spiral). The experiments show its effectiveness to present a video in a very compact and visually appealing form while preserving the necessary information to understand the video.

一种层次视频摘要生成方法

视频摘要是视频内容的一种压缩表示方式。为了能够更好地浏览视频，提出了一种根据浏览或检索的粒度不同来建立两种层次视频摘要(镜头级和场景级)的思想，并给出了一种视频摘要生成方法：首先用一种根据内容变化自动提取镜头内关键帧的方法来实现关键帧的提取；继而用一种改进的时间自适应算法通过镜头的组合来得到场景；最后在场景级用最小生成树方法提取代表帧。由于关键帧和代表帧分别代表了它们所在镜头和场景的主要内容，因此它们的序列就构成了视频总结。一些电影视频片段检验的实验结果表明，这种生成方法能够较好地提供粗细两种粒度的视频内容总结。     

语义视频检索的现状和研究进展

概述了图像的可视化特征如颜色、纹理、形状和运动信息,时空关系分析,以及多特征目标提取和相似度量度;分析了视频语义的提取,语义查询、检索;探讨了视频语义检索的性能评估,存在的问题和发展方向。     

视频相册系统

为了对视频数据进行有效的管理,提出了一种新的视频检索与浏览系统——视频相册系统。在该系统中,首先用相册生成方案挑选出用户数字视频库的一组代表性的关键帧;接着筛选出的关键帧被预训练的形状模板(如圆形、心形、扇形、邮票形等)所裁剪,最终被打印成册。当用户想浏览视频时,可事先浏览该视频相册,就像浏览普通相册一样,若用户想观看相册中某个关键帧所代表的视频片段,即可首先方便地用摄像手机等设备拍摄该关键帧,并通过无线网络(如蓝牙)把拍摄的图像传输给计算机终端;此后,视频相册系统采用基于自训练与Snakes轮廓进化的活动形状模型算法来定位关键帧在拍摄的图像中的轮廓位置,并纠正其成像畸变。最终,系统即可在视频数据库中自动找到与纠正后的关键帧最相似的一幅,并为用户回放其代表的视频片段。实验评测结果表明,该视频相册系统可在数字视频与模拟相册间建立有效的联系。     

A hierarchical organization scheme for video data

Video stream is based on bits of imagery and is thus difficult to be perceived (by machine) in the content level. To access video content, a suitable organization of video data is critical. This paper proposes a hierarchical structure and a process scheme for organizing video data to facilitate indexing, browsing and querying. Four layers can be distinguished, that is: video program, episode, shot and frame. This hierarchy provides an efficient and flexible structure as well as compact and meaningful abstraction of video program. To achieve such an organization, not only the boundary detection of shots and episodes, but also the extraction of key-frames for shots and the selection of representative shots and frames for episodes are important. Suitable criteria and methods for above tasks are proposed and these techniques have been integrated into a workable system. A number of organization experiments using real video data are performed and some results are presented, which show the effectiveness of the proposed organization scheme and techniques.     

基于内容检索的视频处理技术

从分析视频数据的结构和特点出发,总结了基于内容检索的视频处理方法的一般步骤,即视频分割、关键帧选取、静态和动态特征提取以及视频聚类等,然后深入介绍了各个处理过程中的一些最新方法,并分析了各种方法和技术的优缺点;最后,对基于内容的视频检索提出一些值得进一步研究的问题。     

视频摘要技术研究*

对视频摘要技术进行了研究,将视频摘要按其表现形式分为以标题、关键帧和故事板为代表的静态视频摘要以及以缩略视频为代表的动态视频摘要,并对其中涉及的关键技术进行了探讨,最后对视频摘要技术未来的发展方向进行了总结与展望。     

On fast microscopic browsing of MPEG-compressed video

MPEG has been established as a compression standard for efficient storage and transmission of digital video. However, users are limited to VCR-like (and tedious) functionalities when viewing MPEG video. The usefulness of MPEG video is presently limited by the lack of tools available for fast browsing, manipulation and processing of MPEG video. In this paper, we first address the problem of rapid access to individual shots and frames in MPEG video. We build upon the compressed-video-processing framework proposed in [1, 8], and propose new and fast algorithms based on an adaptive mixture of approximation techniques for extracting spatially reduced image sequence of uniform quality from MPEG video across different frame types and also under different motion activities in the scenes. The algorithms execute faster than real time on a Pentium personal computer. We demonstrate how the reduced images facilitate fast and convenient shot- and frame-level video browsing and access, shot-level editing and annotation, without the need for frequent decompression of MPEG video. We further propose methods for reducing the auxiliary data size associated with the reduced images through exploitation of spatial and temporal redundancy. We also address how the reduced images lead to computationally efficient algorithms for video analysis based on intra- and inter-shot processing for video database and browsing applications. The algorithms, tools for browsing and techniques for video processing presented in this paper have been used by many in IBM Research on more than 30 h of MPEG-1 video for video browsing and analysis.     

基于内容的视频浏览界面的设计与实现

基于内容的视频浏览是在基于内容视频检索领域中,技术现状无法满足应用需求情况下的一种折衷手段。ＶｉｄｅｏＣＡＲ是我们设计的一个用于视频内容分析与表现的原型实验平台。在该系统的界面设计中我们充分利用面向对象的设计思想和 ＭＦＣ提供的各种ＡｃｔｉｖｅＸ控件,解决了在屏幕空间有限的条件下提供有效导航机制和高效内容表现的问题,提供了多种浏览手段,帮助用户在各个层次上快速定位感兴趣的视频资料。本文介绍了该系统界面的设计要点和主要实现方法。     

基于AP聚类和频繁模式挖掘的视频摘要生成方法

为了有效支持视频数据库浏览和检索,通过视频摘要来对视频进行紧凑表达变得十分重要.提出了一种新颖的基于近邻传播聚类AP(Affinity Propagation)和频繁镜头模式挖掘的视频摘要自动生成算法.视频频繁镜头模式被定义为在一定时间窗口内经常出现的镜头系列.首先通过近邻传播聚类,将相似镜头聚合到一起;然后采用频繁镜头模式挖掘的方法对视频聚类内容进行挖掘,去掉视频中冗余内容部分;最后通过覆盖视频语义信息的频繁镜头模式生成视频摘要.实验结果表明,视频摘要算法取得了良好的效果.     

J2EE架构下智能视频检索系统集成框架研究

提出了一种基于J2EE平台的智能视频检索系统集成框架,并实现了具有视频分析、内容管理、基于WEB检索和浏览等功能的视频检索系统iVideo。该系统参照MPEG7标准描述视频数据,这种描述便于视频内容的管理;系统采取高层语义特征与底层视觉特征融合以及相关反馈等手段有效提高检索的准确度,并能根据不同的终端设备自适应地显示查询结果。     

视频编码技术研究进展

视频编码是进行视频传输的一个关键环节，随着视频传输的广泛应用，视频编码技术也受到了越来越多的关注。该文系统地讨论了当前视频编码关键技术的研究进展，分析了各种技术的特点，指出了进一步发展的前景。