利用等价关系理论进行视频片段检索的方法

视频片段检索是基于内容的视频检索的主要方式,可是现有的片段检索方法大多只是对预先分割好的片段进行检索。为了从连续的视频节目中自动分割出多个相似的片段,提出了一种新的有效的视频片段检索方法,并首次尝试将等价关系理论应用于视频片段的检索.该方法首先用等价关系理论定义了片段匹配函数,同时采用滑动镜头窗自动分割出多个真正相似的片段;然后把等价类映射为矩阵表达形式,再通过矩阵的特性来度量影响片段相似度的不同因子,实现了相似片段的排序。实验结果表明,该方法能够一次性快速准确地从连续视频库中自动分割出与查询片段相似的多个片段。     

Mining human movement evolution for complex action recognition

In this paper, a novel and efficient system is proposed to capture human movement evolution for complex action recognition. First, camera movement compensation is introduced to extract foreground object movement. Secondly, a mid-level feature representation called trajectory sheaf is proposed to capture the temporal structural information among low-level trajectory features based on key frames selection. Thirdly, the final video representation is obtained by training a sorting model with each key frame in the video clip. At last, the hierarchical version of video representation is proposed to describe the entire video with higher level representation. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on UCF Sports, and comparable results on several challenge benchmarks, such as Hollywood2 and HMDB51 dataset.     

Query by video clip

Typical digital video search is based on queries involving a single shot. We generalize this problem by allowing queries that involve a video clip (say, a 10-s video segment). We propose two schemes: (i) retrieval based on key frames follows the traditional approach of identifying shots, computing key frames from a video, and then extracting image features around the key frames. For each key frame in the query, a similarity value (using color, texture, and motion) is obtained with respect to the key frames in the database video. Consecutive key frames in the database video that are highly similar to the query key frames are then used to generate the set of retrieved video clips. (ii) In retrieval using sub-sampled frames, we uniformly sub-sample the query clip as well as the database video. Retrieval is based on matching color and texture features of the sub-sampled frames. Initial experiments on two video databases (basketball video with approximately 16,000 frames and a CNN news video with approximately 20,000 frames) show promising results. Additional experiments using segments from one basketball video as query and a different basketball video as the database show the effectiveness of feature representation and matching schemes.     

Affective video content representation and modeling

This paper looks into a new direction in video content analysis - the representation and modeling of affective video content . The affective content of a given video clip can be defined as the intensity and type of feeling or emotion (both are referred to as affect) that are expected to arise in the user while watching that clip. The availability of methodologies for automatically extracting this type of video content will extend the current scope of possibilities for video indexing and retrieval. For instance, we will be able to search for the funniest or the most thrilling parts of a movie, or the most exciting events of a sport program. Furthermore, as the user may want to select a movie not only based on its genre, cast, director and story content, but also on its prevailing mood, the affective content analysis is also likely to contribute to enhancing the quality of personalizing the video delivery to the user. We propose in this paper a computational framework for affective video content representation and modeling. This framework is based on the dimensional approach to affect that is known from the field of psychophysiology. According to this approach, the affective video content can be represented as a set of points in the two-dimensional (2-D) emotion space that is characterized by the dimensions of arousal (intensity of affect) and valence (type of affect). We map the affective video content onto the 2-D emotion space by using the models that link the arousal and valence dimensions to low-level features extracted from video data. This results in the arousal and valence time curves that, either considered separately or combined into the so-called affect curve, are introduced as reliable representations of expected transitions from one feeling to another along a video, as perceived by a viewer.     

Example-based video remixing

A video remix is generally created by arranging selected video clips and combining them with other media streams such as audio clips and video transition effects. This paper proposes a system for semi-automatically creating video remixes of good expressive quality. Given multiple original video clips, audio clips, and transition effects as the input, the proposed system generates a video remix by five processes: I) video clip sequence generation, II) audio clip selection, III) audio boundary extraction, IV) video segment extraction, and V) transition effect selection, based on the spatial and temporal structural patterns automatically learned from professionally created video remix examples. Experiments using movie trailers of action genre as video remix examples not only demonstrate that video remixing by professionals can be imitated based on examples but also reveal that the video clip sequence generation and audio clip selection are the most important processes to improve the perceived expressive quality of video remixes.     

一种基于小波的视频片断检索方法

视频片断检索是基于内容的视频检索的主要方式,该文首先利用颜色信息对视频库进行初步筛选,然后根据集合的相似性度量进行精确检索,通过实验,该方法可以获得更快的检索速度。     

A spatio-temporal pyramid matching for video retrieval

An efficient video retrieval system is essential to search relevant video contents from a large set of video clips, which typically contain several heterogeneous video clips to match with. In this paper, we introduce a content-based video matching system that finds the most relevant video segments from video database for a given query video clip. Finding relevant video clips is not a trivial task, because objects in a video clip can constantly move over time. To perform this task efficiently, we propose a novel video matching called Spatio-Temporal Pyramid Matching (STPM). Considering features of objects in 2D space and time, STPM recursively divides a video clip into a 3D spatio-temporal pyramidal space and compares the features in different resolutions. In order to improve the retrieval performance, we consider both static and dynamic features of objects. We also provide a sufficient condition in which the matching can get the additional benefit from temporal information. The experimental results show that our STPM performs better than the other video matching methods.     

相似动画视频片段探测方法

为了实现相似视频片段的快速探测,以动画视频片段为研究对象,提出一种建立在视频单元层上的动画视频片段探测方法.在视频特征描述阶段,采用更符合动画图像的Markov平稳特征来描述动画视频帧的视觉特征,并利用视频距离轨迹(VDT)来挖掘视频片段特征,同时采用线性拟合特征的描述方法来描述VDT的特征;在特征匹配阶段,将视频片段匹配问题转换为网络流优化的问题,通过将视频单元的时间一致性嵌入到匹配网络中来寻找最佳对齐方式,大幅度减少了匹配的数据量.实验结果表明,该方法极大地改善了相似视频片段的探测效果,与传统的视频匹配方法相比,其具有更好的鲁棒性以及更高的效率.     

Abnormal visual event detection based on multi-instance learning and autoregressive integrated moving average model in edge-based Smart City surveillance

The abnormal visual event detection is an important subject in Smart City surveillance where a lot of data can be processed locally in edge computing environment. Real-time and detection effectiveness are critical in such an edge environment. In this paper, we propose an abnormal event detection approach based on multi-instance learning and autoregressive integrated moving average model for video surveillance of crowded scenes in urban public places, focusing on real-time and detection effectiveness. We propose an unsupervised method for abnormal event detection by combining multi-instance visual feature selection and the autoregressive integrated moving average model. In the proposed method, each video clip is modeled as a visual feature bag containing several subvideo clips, each of which is regarded as an instance. The time-transform characteristics of the optical flow characteristics within each subvideo clip are considered as a visual feature instance, and time-series modeling is carried out for multiple visual feature instances related to all subvideo clips in a surveillance video clip. The abnormal events in each surveillance video clip are detected using the multi-instance fusion method. This approach is verified on publically available urban surveillance video datasets and compared with state-of-the-art alternatives. Experimental results demonstrate that the proposed method has better abnormal event detection performance for crowded scene of urban public places with an edge environment.     

基于三维小波变换视频片断检索技术研究

提出了一种基于三维小波的视频片断检索的新方法,该方法克服了传统视频检索方法中未考虑的时空关系.首先利用低频子图小波系数的标准方差均值对目标视频库进行初步筛选,然后对利用八叉树算法提取细节特征,进一步的把细节特征和近似特征组合起来对图像进行精确的检索.实验证明,该方法具有更好的检索精度.     

一种通过视频片段进行视频检索的方法

视频片段检索是基于内容的视频检索的主要方式,它需要解决两个问题:(1) 从视频库里自动分割出与查询片段相似的多个片段;(2) 按照相似度从高到低排列这些相似片段.首次尝试运用图论的匹配理论来解决这两个问题.针对问题(1),把检索过程分为两个阶段:镜头检索和片段检索.在镜头检索阶段,利用相机运动信息,一个变化较大的镜头被划分为几个内容一致的子镜头,两个镜头的相似性通过对应子镜头的相似性计算得到;在片段检索阶段,通过考察相似镜头的连续性初步得到一个个相似片段,再运用最大匹配的Hungarian算法来确定真正的相似片段.针对问题(2),考虑了片段相似性判断的视觉、粒度、顺序和干扰因子,提出用最优匹配的Kuhn-Munkres算法和动态规划算法相结合,来解决片段相似度的度量问题.实验对比结果表明,所提出的方法在片段检索中可以取得更高的检索精度和更快的检索速度.     

一种基于颜色空间最短距离的编码误差修正新方法

在视频编码的过程中, 编码后的数据不可避免地会发生一定程度的失真。在这些失真的色点中会有一部分的色点跃出颜色空间的范围, 因此在视频编码框架的重建部分使用了误差修正(clip)来修正这些误差点。现有的误差修正方法仅仅是对YUV数据中各个分量进行了简单的超出0~255范围的修正, 而忽略了颜色空间变换后YUV的颜色空间模型的变化。针对该问题, 提出了一种基于颜色空间最短距离的编码误差修正（minimum distance clip）新方法。该方法利用YUV颜色空间模型以及空间几何的理论, 通过寻找这些误差点在该模型上的最短距离点来修正上述误差。采用H. 264参考软件JM13. 0进行实验验证, 测试结果表明, 该方法较传统的修正方法在主观质量和客观性能上均有一定的提高, 对有较高色彩质量需求的先进视频编码应用, 如超高清视频、3DTV等有一定的贡献。     

基于关键帧序列的视频片段检索

提出了一种基于关键帧融合的视频片段检索方法。使用特征联合分布直方图将视频分割为子镜头,子镜头用关键帧表示。检索时,对范例视频片段的每个关键帧检索到相似的关键帧,所有的相似关键帧按照时间连续性融合为视频片段。提出一种快速的视频片段相似度计算模型。实验表明,本文的方法快速有效。     

A framework for automatic static and dynamic video thumbnail extraction

Video thumbnails enable users to see quick snapshots of video collections. To display the video thumbnails, the first frame or a frame selected by using simple low level features in each video clip has been set to the default thumbnail for the sake of computational efficiency and implementation simplicity. However, such methods often fail to represent the gist of the clip. To overcome this limitation, we present a new framework for both static and dynamic video thumbnail extraction. First, we formulate energy functions using the features which incorporate mid-level information to obtain superior thumbnailing. Since it is considered that frames whose layouts are similar to others in the clip are relevant in video thumbnail extraction, scene layouts are also considered in computing overall energy. For dynamic thumbnail generation, a time slot is determined by finding the duration showing the minimum energy. Experimental results show that the proposed method achieves comparable performance on a variety of challenging videos, and the subjective evaluation demonstrates the effectiveness of our method.     

一种基于确信度的双向视频分割方法

提出了一种将前景物体从一个视频序列中分割出来的方法.提出了确信度模型,该模型由局部颜色配置信息求得.该算法首先对视频所有帧做了分水岭预处理,然后对关键帧做了图分割处理.接下来经由双向过程计算确信度.具体为先通过正向过程计算确信度,再通过逆向过程并辅以光流算法对小部分确信度进行修正,最后根据确信度进行前背景标注.该确信度模型对前背景较为相似的视频有良好的分割效果,并且由于通过双向过程进行分割,对视频中部分遮挡物体的分割也有不错的改善.     

基于时空灰度序特征的视频片段定位算法

基于灰度序特征的视频片段定位算法是解决视频片段定位问题的典型算法.这类算法存在的不足是：特征的唯一性表示能力不够,使得在召回率较高的情况下,定位检索的精度下降得较快;二次多项式级的时间复杂度使得响应时间过长,并对查询视频长度敏感.针对上述两个问题,提出了一种基于时空灰度序特征的视频片段定位算法,其关键步骤包括：(1) 在精确定位之前,通过引入线性时间复杂度的基于时空二值模式直方图特征(spatio-temporal binary pattern histogram,简称STBPH)的实时过滤算法以及基于二值时间灰度序特征(binarytemporal ordinal measure,简称BTOM)的快速过滤算法,大幅度减少精确定位阶段需要进行比较的候选视频片段个数;(2) 在精确定位阶段,通过引入唯一性表示能力更好且保持了较好鲁棒性的时空统一灰度序特征(jointspatio-temporal ordinal measure,简称JSTOM)进行序列匹配,显著提高了定位检索的精度.实验结果表明,该算法能够快速、准确地进行视频片段定位,大幅降低了对查询视频长度的敏感度.     

基于帧数据量波动特性的压缩域视频快速检索方法

为实现压缩域视频快速检索,提出基于帧数据量波动特性的检索方法。该方法首先计算压缩域各图像帧的数据量,得出查询片段和目标视频等长内的数据量曲线,然后在I帧对齐的基础上将查询片段在目标视频上进行滑动,滑动窗长为单个图组长度。再在每次滑动后计算查询片段与目标视频数据量曲线波动的差异程度,同时每次滑动后要更新目标视频的数据量曲线。最后结合设定门限进行相似判决并返回结果。该方法不需要为每一帧抽取高维特征向量,用一个向量而不是一组高维向量来表述一段视频。实验结果表明,相比现有快速检索算法,该方法使检索速度得到提高,同时也能达到较高的准确率。 另外,该方法既可用于基于压缩域视频库的快速检索,也可用于在线的视频片段匹配,实时发现与设定目标相似的视频。     

Adobe Premiere CS4中数字视频合成技术的实现

视频合成就是要将多个视频画面融合为一个视频画面,它是数字视频编辑中的一个重要问题。在Adobe Premiere CS4视频编辑软件中,提供了多种技术手段来解决这一问题。本文介绍通过调整素材的透明度和设置视频剪辑的＂混合模式＂来完成视频合成;通过视频特效组中的滤镜,根据素材的颜色、亮度、无用信号以及遮罩进行抠像,以达到视频合成的目的;使用＂轨道遮罩键＂特效实现跟踪画面的视频合成技术。实践证明,灵活地使用这些方法可以制作出高效、自然的数字视频作品。