一种基于聚类的关键帧提取算法

关键帧的提取是视频数据结构化的一部分,在获得关键帧之后就可以进入基于内容的图像检索阶段,实现基于内容的视频检索。本文提出了一种在镜头边界检测之后再进行视频帧聚类的方法来提取关键帧。聚类形成了数据更小的子镜头,最后从子镜头中选择与聚类中心距离最小的一帧作为关键帧。最后,通过仿真实验表明该方法能够快速有效地提取出视频关键帧。     

级联手工特征与深度特征的视频关键帧检测方法

关键帧检测是有效的视频内容分析的关键环节。常用的基于手工特征的方法运行效率高但很难有效表征关键帧特征,因而性能不好。基于深度特征的方法因为网络结构复杂,导致效率不高。在体育比赛类视频中,关键帧常为比赛转播中镜头变化的最后一帧。但广播视频中除了包含比赛视频还包括很多其他类型的镜头如中场休息、渐变镜头等。因此检测最后一帧包含很多比赛无关内容。针对这一问题,本文提出了一种手工特征与深度特征相结合的视频关键帧检测方法。首先基于颜色直方图特征进行镜头边界检测获取最后一帧。进一步基于直方图相似性提出一种类似聚类的方法得到候选关键帧。最后,基于深度神经网络对候选关键帧进行分类,得到真正的关键帧。在冰壶比赛视频和篮球比赛视频上的对比实验结果表明,相对于传统的背景差分法、光流法等,本文提出方法能够快速、可靠地提取关键帧。      

HEVC压缩域的视频摘要关键帧提取方法

为了提高关键帧提取的准确率，改善视频摘要的质量，提出了一种HEVC压缩域的视频摘要关键帧提取方法。首先，对视频序列进行编解码，在解码中统计HEVC帧内编码PU块的亮度预测模式数目。然后，特征提取是利用统计得到的模式数目构建成模式特征向量，并将其作为视频帧的纹理特征用于关键帧的提取。最后，利用融合迭代自组织数据分析算法(ISODATA）的自适应聚类算法对模式特征向量进行聚类，在聚类结果中选取每个类内中间向量对应的帧作为候选关键帧，并通过相似度对候选关键帧进行再次筛选，剔除冗余帧，得到最终的关键帧。实验结果表明，在Open Video Project数据集上进行的大量实验验证，该方法提取关键帧的精度为79.9%、召回率达到93.6%、F-score为86.2%，有效地改善了视频摘要的质量。      

基于压缩传感和EMD距离的视频镜头关键帧提取

关键帧提取是视频内容分析与检索技术的核心问题。提出一种基于压缩传感和EMD距离的关键帧提取方法,首先构造一个符合有限等距性质的稀疏矩阵,将帧高维特征投影到低维空间,然后通过计算帧低维特征之间的调节余弦相似度完成子镜头分割。在各子镜头中,利用EMD距离计算帧与子镜头中心的差异,并选择差异最小值所对应的帧作为该子镜头的关键帧。实验结果表明,该方法提取的关键帧能够对视频内容进行准确的描述。     

基于帧间似然比的关键帧提取算法

借助基于内容分析的关键帧提取算法的基本思路，提出了一种基于帧间似然比的关键帧提取算法，算法利用Y、Cb、Cr3个颜色分量的均值和方差作为图像帧的特征参数，有效地描述了帧间相似性，对镜头突变和渐变有较强的鲁棒性，实验表明该方法能较好地提取出视频序列的关键帧。     

探讨基于Tsallis熵的视频关键帧提取技术

关键帧是视频中的一组有限数量的帧的子集,一个视频的关键帧序列能够合理地概括该视频信息,从而减少过大的视频数据对生产生活带来的承载负重.本文讨论了基于Tsallis熵的Jensen距离公式——JTD在视频关键帧提取中的应用.根据得到的差异性距离值JTD,首先检查子镜头边界,进而从每个子镜头中抽取一帧作为该镜头的代表帧,最终得到该段视频的关键帧序列.     

基于聚类的视频镜头分割和关键帧提取

镜头分割是基于内容的视频检索和浏览首先要解决的关键技术。视频分割为镜头后,下一步的工作就是进行关键帧提取,用以描述镜头的主要内容。提出了一种改进的基于聚类的镜头分割和关键帧提取算法．在无监督聚类中引入一个参考变量,解决了利用无监督聚类进行镜头分割和关键帧提取时可能产生的帧序不连续或分割错误的问题。在关键帧提取阶段,将镜头分割为子镜头后,引入图像熵的概念提取关键帧。实验结果表明了改进算法在镜头分割和关键帧提取方面的有效性。     

基于压缩感知和熵计算的关键帧提取算法

针对关键帧提取问题,提出了一种基于压缩感知理 论和熵计算的关键帧提取算法, 首先通过构造符合有限等距性质要求的稀疏随机投影矩阵,将高维多尺度帧图像特征变换为 低维多尺度帧图像特征, 并形成视频镜头低维多尺度特征列向量组;然后通过随机权值向量与低维多尺度特征向量的 阿达玛乘积运算生成各 帧图像的匹配特征,并根据匹配特征的相似性度量完成镜头内部的子镜头分割;最后通过交 叉熵计算在每个子镜头 中得到可能的关键帧,并由图像熵计算确定最终的关键帧。实验表明,与传统方法相比,本 文算法提取的关键帧能够更精确、更稳定描述视频镜头内容。     

一种压缩域原始视频资料的摘要方法

本文基于MPEG视频流的结构特点,给出了一种针对原始未编辑视频资料的摘要方法.该方法主要应用B帧预测编码宏块的统计特性来检测场景变换,利用Ⅰ帧的分布特性检测复杂运动,Ⅰ帧亮度DC系数检测闪光,综合多种因素进行关键帧提取,形成适于实际工作需要、计算量小、能较好地反映视频情节的原始视频资料的视频摘要.     

一种压缩域中基于镜头的视频检索方法

镜头是视频的基本单元。文章提出了一个计算镜头纹理直方图和在压缩域中基于镜头的视频检索方法。对镜头中所有Ⅰ帧的DC图分别提取颜色和纹理直方图，然后分别形成镜头的可变阿尔法颜色和纹理直方图．并用镜头的这些特征在镜头层次上用不同的距离度量方法进行视频检索。实验结果表明，基于镜头的检索用L1度量距离比用L2和Χ^2度量距离有更好的检索性能，而且避免了基于关键帧检索中由于镜头运动等原因可能导致镜头关键帧选择不当对检索性能的影响，因而比基于关键帧检索具有更好的检索性能。     

Memorable and rich video summarization

Video summarization can facilitate rapid browsing and efficient video indexing in many applications. A good summary should maintain the semantic interestingness and diversity of the original video. While many previous methods extracted key frames based on low-level features, this study proposes Memorability-Entropy-based video summarization. The proposed method focuses on creating semantically interesting summaries based on image memorability. Further, image entropy is introduced to maintain the diversity of the summary. In the proposed framework, perceptual hashing-based mutual information (MI) is used for shot segmentation. Then, we use a large annotated image memorability dataset to fine-tune Hybrid-AlexNet. We predict the memorability score by using the fine-tuned deep network and calculate the entropy value of the images. The frame with the maximum memorability score and entropy value in each shot is selected to constitute the video summary. Finally, our method is evaluated on a benchmark dataset, which comes with five human-created summaries. When evaluating our method, we find it generates high-quality results, comparable to human-created summaries and conventional methods.     

Video abstraction based on the visual attention model and online clustering

With the fast evolution of digital video, research and development of new technologies are greatly needed to lower the cost of video archiving, cataloging and indexing, as well as improve the efficiency and accessibility of stored video sequences. A number of methods to respectively meet these requirements have been researched and proposed. As one of the most important research topics, video abstraction helps to enable us to quickly browse a large video database and to achieve efficient content access and representation. In this paper, a video abstraction algorithm based on the visual attention model and online clustering is proposed. First, shot boundaries are detected and key frames in each shot are extracted so that consecutive key frames in a shot have the same distance. Second, the spatial saliency map indicating the saliency value of each region of the image is generated from each key frame and regions of interest (ROI) is extracted according to the saliency map. Third, key frames, as well as their corresponding saliency map, are passed to a specific filter, and several thresholds are used so that the key frames containing less information are discarded. Finally, key frames are clustered using an online clustering method based on the features in ROIs. Experimental results demonstrate the performance and effectiveness of the proposed video abstraction algorithm.     

Video key frame extraction through dynamic Delaunay clustering with a structural constraint

Key frame based video summarization has emerged as an important area of research for the multimedia community. Video key frames enable an user to access any video in a friendly and meaningful way. In this paper, we propose an automated method of video key frame extraction using dynamic Delaunay graph clustering via an iterative edge pruning strategy. A structural constraint in form of a lower limit on the deviation ratio of the graph vertices further improves the video summary. We also employ an information-theoretic pre-sampling where significant valleys in the mutual information profile of the successive frames in a video are used to capture more informative frames. Various video key frame visualization techniques for efficient video browsing and navigation purposes are incorporated. A comprehensive evaluation on 100 videos from the Open Video and YouTube databases using both objective and subjective measures demonstrate the superiority of our key frame extraction method.     

基于音视频模板匹配的新闻视频识别方法

本文提出了一种基于音视模板匹配的新闻视频识别方法。在模板建立过程中,从新闻视频片头中的主题音乐提取音频模板,从主持人镜头中的扩展人脸区域提取视觉模板,这两者共同构成音视模板;在识别过程中,对电视视频流先进行音频模板匹配,然后由匹配通过的候选时间点定位到相应的视频镜头,接着通过视觉模板对镜头中的扩展人脸区域进行匹配,进而确定主持人镜头,最后完成新闻视频识别。实验结果表明,该方法计算效率高、简单易操作,具有较好的实用价值。     

基于颜色和灰度熵的视频镜头检测方法

视频镜头分割是视频检索和视频分类等应用的关键步骤,因此提出一种基于颜色直方图和灰度熵的镜头检测算法。该方法以HSV颜色直方图和图像的灰度熵为特征,计算视频相邻两图像帧的相似度,从而判断出是否发生了镜头变换。实验证明,该方法能准确检测出视频的突变镜头,对渐变镜头也有理想的检测效果。     

新闻视频片花检索新方法

针对新闻视频中精品小栏目的点播需要靠人工剪辑,费时费力,提出了一种基于时空切片的片花检索方法。该方法根据片花的特点过滤掉大部分不相关的镜头,然后通过镜头合并规则得到候选片段,最后将候选片段融合为时空切片,并在候选片段时空切片的基础上采用分块主色调特征和纹理特征相结合的方法得到最后的相似片段。实验从查全率和定位精度两方面证明了该方法的有效性。     

基于类内和类间损失的场景构造方法

为了便于在不同结构层次上对视频进行检索和浏览。可以把视频序列分为不同层次的逻辑单元。逻辑单元的层次由上到下可以分成序列、场景、镜头和帧。其中场景是时间上具有一定顺序关系的相似镜头的集合。文章提出了基于类内和类间损失的场景构造算法。首先利用时间约束和颜色直方图求得镜头距离；然后基于类内和类间损失对相似镜头进行聚类，得到镜头类；最后在分析镜头类的基础上构造场景。实验证明，构造的场景比较好的反映了视频的内容。     

A novel video abstraction method based on fast clustering of the regions of interest in key frames

Online video nowadays has become one of the top activities for users and has become easy to access. In the meantime, how to manage such huge amount of video data and retrieve them efficiently has become a big issue. In this article, we propose a novel method for video abstraction based on fast clustering of the regions of interest (ROIs). Firstly, the key-frames in each shot are extracted using the average histogram algorithm. Secondly, the saliency and edge maps are generated from each key-frame. According to these two maps, the key points for the visual attention model can be determined. Meanwhile, in order to expand the regions surrounding the key points, several thresholds are calculated from the corresponding key-frame. Thirdly, based on the key points and thresholds, several regions of interest are expanded and thus the main content in each frame is obtained. Finally, the fast clustering method is performed on the key frames by utilizing their ROIs. The performance and effectiveness of the proposed video abstraction algorithm is demonstrated by several experimental results.