视频分割中特殊编辑的一种检测算法

在现有的编辑以及镜头操作的检测方法中，对突然变化的检测相对容易一些，而对镜头操作的检测比较困难，提出了一种在压缩域上，对MPEG流DCT变换的直流分量沿时间轴形成的3维数据空间中，运动向量随时间累积的分布效果进行分析的方法，来对特殊编辑与镜头操作作用进行检测，与现有的视频分割方法相比较，有较好的对光照度变化，目标移动造成的干扰的鲁棒性。     

Efficient Segmentation and Camera Motion Indexing of Compressed Video

In order to provide sophisticated access methods to the contents of video servers, it is necessary to automatically process and represent each video through a number of visual indexes. We focus on two tasks, namely the hierarchical representation of a video as a sequence of uniform segments (shots), and the characterization of each shot by a vector describing the camera motion parameters. For the first task we use a Bayesian classification approach to detecting scene cuts by analysing motion vectors. Adaptability to different compression qualities is achieved by learning different classification masks. For the second task, the optical flow is processed in order to distinguish between stationary and moving shots. A least-squares fitting procedure determines the pan/tilt/zoom camera parameters within shots that present regular motion. Each shot is then indexed by a vector representing the dominant motion components and the type of motion. In order to maximize processing speed, all techniques directly process and analyse MPEG-1 motion vectors, without the need for video decompression. An overall processing rate of 59 frames/s is achieved on software. The successful classification performance, evaluated on various news video clips for a total of 61 023 frames, attains 97.7% for the shot segmentation, 88.4% for the stationary vs. moving shot classification, and 94.7% for the detailed camera motion characterization.     

基于综合相似度的二次差分法在镜头检测中的应用

本文提出了一种基于综合相似度的二次差分法,综合考虑小波分析和统计量等多种方法对镜头进行检测。在镜头突变检测中考虑了相邻帧的二次差分法;同时,在渐变检测中引入了自适应阈值法,考虑了非相邻帧的二次差分法,避免了人为因素,克服了其他方法仅适用于一类视频序列或几类视频序列的限制,因而具有较好的鲁棒性。实际视频数数据的实验结果表明,本文所用方法对镜头边界检测具有很好的效果。     

H．264压缩域中利用Biased-SVM检测镜头边界

为了直接从H．264码流中检测镜头边界,提出了利用H．264压缩域多特征和Biased—SVM（不平衡支持向量机）分类算法的检测方法。分析帧类型、宏块类型、运动矢量、帧内预测模式等信息,以获得发生镜头突变和渐变的特征。针对镜头边界帧的数量远少于视频帧总数的特点,用Biased—SVM分类方法将视频帧分为突变帧、渐变帧和非镜头边界帧。在TRECVID视频集上的实验结果表明,与其他H．264压缩域的算法相比,该算法有更好的性能。     

视频层次结构挖掘

视频处理的关键是视频信息的结构化,视频基本结构是由帧、镜头、场景和视频节目构成的层次结构。视频层次结构挖掘的一个简单框架是对视频进行镜头分割、抽取镜头特征和视频场景构造。论文在镜头分割的基础上提出了基于多特征的镜头聚类分析和基于镜头的场景边界检测两种视频场景构造方法,从而实现视频层次结构挖掘。实验表明,基于镜头的场景边界检测性能优于基于多特征的镜头聚类分析。     

Detecting motion regions in presence of strong parallax from a moving camera by multi-view geometric constraints

We present a method for detecting motion regions in video sequences observed by a moving camera in the presence of a strong parallax due to static 3D structures. The proposed method classifies each image pixel into planar background, parallax, or motion regions by sequentially applying 2D planar homographies, the epipolar constraint, and a novel geometric constraint called the "structure consistency constraint." The structure consistency constraint, being the main contribution of this paper, is derived from the relative camera poses in three consecutive frames and is implemented within the "Plane + Parallax" framework. Unlike previous planar-parallax constraints proposed in the literature, the structure consistency constraint does not require the reference plane to be constant across multiple views. It directly measures the inconsistency between the projective structures from the same point under camera motion and reference plane change. The structure consistency constraint is capable of detecting moving objects followed by a moving camera in the same direction, a so-called degenerate configuration where the epipolar constraint fails. We demonstrate the effectiveness and robustness of our method with experimental results of real-world video sequences.     

一种新的视频镜头分割算法

镜头是视频分析和索引的基础，但是自动的镜头分割，尤其是渐变切换的检测还是一个很有挑战性的课题。本文提出了一种利用直方图与模板匹配相结合来进行视频镜头切变检测的算法和一种利用图象灰度级平均(MGL)来进行渐变检测的算法，该渐变检测算法能有效区分摄象机镜头的运动和渐变。实际测试证明，利用本文算法进行视频镜头检测
能取得比较好的效果。     

一种基于模型的扫换检测方法

视频自动分割是实现视频数据库检索必不可少的一个过程,其基础是镜头边界检测.当前已有的算法能够较准确地检测出镜头突变,但对于镜头的渐变则常常会漏检,这是由于镜头渐变时帧间差没有一个明显的峰值,因而其检测比突变检测要困难得多.扫换是一种常用的视频空间编辑手段,用于实现多种镜头变化.通过分析各种类型的扫换,提出了一种新的基于视频空间编辑模型的扫换检测算法,其性能优于Alattar提出的基于统计特征的算法.对用AdobePremiere5.1生成的各种扫换视频进行检测.实验结果表明,这种扫换检测算法能够较好地适应     

Statistical sequential analysis for real-time video scene change detection on compressed multimedia bitstream

The increased availability and usage of multimedia information have created a critical need for efficient multimedia processing algorithms. These algorithms must offer capabilities related to browsing, indexing, and retrieval of relevant data. A crucial step in multimedia processing is that of reliable video segmentation into visually coherent video shots through scene change detection. Video segmentation enables subsequent processing operations on video shots, such as video indexing, semantic representation, or tracking of selected video information. Since video sequences generally contain both abrupt and gradual scene changes, video segmentation algorithms must be able to detect a large variety of changes. While existing algorithms perform relatively well for detecting abrupt transitions (video cuts), reliable detection of gradual changes is much more difficult. A novel one-pass, real-time approach to video scene change detection based on statistical sequential analysis and operating on a compressed multimedia bitstream is proposed. Our approach models video sequences as stochastic processes, with scene changes being reflected by changes in the characteristics (parameters) of the process. Statistical sequential analysis is used to provide an unified framework for the detection of both abrupt and gradual scene changes.     

Direct recovery of planar-parallax from multiple frames

We present an algorithm that estimates dense planar-parallax motion from multiple uncalibrated views of a 3D scene. This generalizes the "plane+parallax" recovery methods to more than two frames. The parallax motion of pixels across multiple frames (relative to a planar surface) is related to the 3D scene structure and the camera epipoles. The parallax field, the epipoles, and the 3D scene structure are estimated directly from image brightness variations across multiple frames, without precomputing correspondences.     

Joint radiometric calibration and feature tracking system with an application to stereo

To capture the full brightness range of natural scenes, cameras automatically adjust the exposure value which causes the brightness of scene points to change from frame to frame. Given such a video sequence, we introduce a system for tracking features and estimating the radiometric response function of the camera and the exposure difference between frames simultaneously. We model the global and nonlinear process that is responsible for the changes in image brightness rather than adapting to the changes locally and linearly which makes our tracking more robust to the change in brightness. We apply our system to perform structure-from-motion and stereo to reconstruct a texture-mapped 3D surface from a video taken in a high dynamic range environment.     

InsightVideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval

Hierarchical video browsing and feature-based video retrieval are two standard methods for accessing video content. Very little research, however, has addressed the benefits of integrating these two methods for more effective and efficient video content access. In this paper, we introduce InsightVideo, a video analysis and retrieval system, which joins video content hierarchy, hierarchical browsing and retrieval for efficient video access. We propose several video processing techniques to organize the content hierarchy of the video. We first apply a camera motion classification and key-frame extraction strategy that operates in the compressed domain to extract video features. Then, shot grouping, scene detection and pairwise scene clustering strategies are applied to construct the video content hierarchy. We introduce a video similarity evaluation scheme at different levels (key-frame, shot, group, scene, and video.) By integrating the video content hierarchy and the video similarity evaluation scheme, hierarchical video browsing and retrieval are seamlessly integrated for efficient content access. We construct a progressive video retrieval scheme to refine user queries through the interactions of browsing and retrieval. Experimental results and comparisons of camera motion classification, key-frame extraction, scene detection, and video retrieval are presented to validate the effectiveness and efficiency of the proposed algorithms and the performance of the system.     

视频镜头时域分割方法的研究

视频时域分割指将视频序列分成若干镜头，是视频内容分析以及基于内容的视频浏览和检索的第一步．该文首先对视频结构以及视频镜头种类进行了简要的描述，然后对为计算不连续性而采用的提取特征和建立测量准则的常用方法进行概述．其后，文章介绍了检测镜头切变和渐变的算法及其优缺点．在压缩域上检测镜头变换边界的问题也在文中予以分析．在结论与展望中，提出了一些这一领域的难点和对今后工作的展望．     

Video parsing and browsing using compressed data

Parsing video content is an important first step in the video indexing process. This paper presents algorithms to automate the video parsing task, including partitioning a source video into clips and classifying those clips according to camera operations, using compressed video data. We have developed two algorithms and a hybrid approach to partitioning video data compressed according to the JPEG and MPEG standards. The algorithms utilize both the video content encoded in DCT (Discrete Cosine Transform) coefficients and the motion vectors between frames. The hybrid approach integrates the two algorithms and incorporates multi-pass strategies and motion analyses to improve both accuracy and processing speed. Also, we present content-based video browsing tools which utilize the information, particularly about the shot boundaries and key frames, obtained from parsing.     

Shot clustering techniques for story browsing

Automatic video segmentation is the first and necessary step for organizing a long video file into several smaller units. The smallest basic unit is a shot. Relevant shots are typically grouped into a high-level unit called a scene. Each scene is part of a story. Browsing these scenes unfolds the entire story of a film, enabling users to locate their desired video segments quickly and efficiently. Existing scene definitions are rather broad, making it difficult to compare the performance of existing techniques and to develop a better one. This paper introduces a stricter scene definition for narrative films and presents ShotWeave, a novel technique for clustering relevant shots into a scene using the stricter definition. The crux of ShotWeave is its feature extraction and comparison. Visual features are extracted from selected regions of representative frames of shots. These regions capture essential information needed to maintain viewers' thought in the presence of shot breaks. The new feature comparison is developed based on common continuity-editing techniques used in film making. Experiments were performed on full-length films with a wide range of camera motions and a complex composition of shots. The experimental results show that ShotWeave outperforms two recent techniques utilizing global visual features in terms of segmentation accuracy and time.     

基于颜色空间的自适应阈值镜头分割算法

镜头分割是基于内容的视频检索的关键步骤,它会直接影响到视频检索的效果。文中介绍了几种常用的镜头分割方法,并且根据YUV颜色空间的各分量可分离的特点,提出了一种改进的基于YUV颜色空间的自适应阈值镜头分割方法。在突变镜头检测模块,为了消除由噪声带来的误检,加入了帧差值比法。在渐变镜头检测模块,使用了滑动窗口值方法。镜头分割的难点是渐变检测,算法在渐变检测上也取得了不错的成果。经过大量实验结果表明,改进的算法对镜头分割有很好的实验效果,算法计算复杂度低,易于实现。     

Multi-frame estimation of planar motion

Traditional plane alignment techniques are typically performed between pairs of frames. We present a method for extending existing two-frame planar motion estimation techniques into a simultaneous multi-frame estimation, by exploiting multi-frame subspace constraints of planar surfaces. The paper has three main contributions: 1) we show that when the camera calibration does not change, the collection of all parametric image motions of a planar surface in the scene across multiple frames is embedded in a low dimensional linear subspace; 2) we show that the relative image motion of multiple planar surfaces across multiple frames is embedded in a yet lower dimensional linear subspace, even with varying camera calibration; and 3) we show how these multi-frame constraints can be incorporated into simultaneous multi-frame estimation of planar motion, without explicitly recovering any 3D information, or camera calibration. The resulting multi-frame estimation process is more constrained than the individual two-frame estimations, leading to more accurate alignment, even when applied to small image regions.     

Shot Change Detection via Local Keypoint Matching

Shot change detection is an essential step in video content analysis. However, automatic shot change detection often suffers from high false detection rates due to camera or object movements. To solve this problem, we propose an approach based on local keypoint matching of video frames. This approach aims to detect both abrupt and gradual transitions between shots without modeling different kinds of transitions. Our experiment results show that the proposed algorithm is effective for most kinds of shot changes.      

An Accumulation Algorithm for Video Shot Boundary Detection

In this paper, an accumulation algorithm for video shot detection is introduced. The algorithm considers the properties of gradual transition. In a gradual transition, there is only a small difference between consecutive frames. The algorithm remembers the differences between consecutive frames and accumulates them. When the accumulation difference exceeds a threshold, an occurrence of shot transition is declared. Our main contributions are to introduce a frame C that remembers the changes from the beginning of a shot and detect the different types of boundaries (cut, fade, dissolve) at one process. We tested our algorithm with clips extracted from MPEG VCDs. The algorithm showed a good performance in detecting the gradual transitions as well as the abrupt cuts and has the ability to identify different types of boundaries.