Memorable and rich video summarization

Video summarization can facilitate rapid browsing and efficient video indexing in many applications. A good summary should maintain the semantic interestingness and diversity of the original video. While many previous methods extracted key frames based on low-level features, this study proposes Memorability-Entropy-based video summarization. The proposed method focuses on creating semantically interesting summaries based on image memorability. Further, image entropy is introduced to maintain the diversity of the summary. In the proposed framework, perceptual hashing-based mutual information (MI) is used for shot segmentation. Then, we use a large annotated image memorability dataset to fine-tune Hybrid-AlexNet. We predict the memorability score by using the fine-tuned deep network and calculate the entropy value of the images. The frame with the maximum memorability score and entropy value in each shot is selected to constitute the video summary. Finally, our method is evaluated on a benchmark dataset, which comes with five human-created summaries. When evaluating our method, we find it generates high-quality results, comparable to human-created summaries and conventional methods.     

Video key frame extraction through dynamic Delaunay clustering with a structural constraint

Key frame based video summarization has emerged as an important area of research for the multimedia community. Video key frames enable an user to access any video in a friendly and meaningful way. In this paper, we propose an automated method of video key frame extraction using dynamic Delaunay graph clustering via an iterative edge pruning strategy. A structural constraint in form of a lower limit on the deviation ratio of the graph vertices further improves the video summary. We also employ an information-theoretic pre-sampling where significant valleys in the mutual information profile of the successive frames in a video are used to capture more informative frames. Various video key frame visualization techniques for efficient video browsing and navigation purposes are incorporated. A comprehensive evaluation on 100 videos from the Open Video and YouTube databases using both objective and subjective measures demonstrate the superiority of our key frame extraction method.     

基于超图排序算法的视频摘要

视频摘要技术作为一种快速感知视频内容的方式得到了广泛的关注.现有基于图模型的视频摘要方法将视频帧作为顶点,通过边表示两个顶点之间的关系,但并不能很好地捕获视频帧之间的复杂关系.为了克服该缺点,本文提出了一种基于超图排序算法的静态视频摘要方法（Hyper-Graph Ranking based Video Summarization,HGRVS）.HGRVS方法首先通过构建视频超图模型,将任意多个有内在关联的视频帧使用一条超边连接;然后提出一种基于超图排序的视频帧分类算法将视频帧按内容分类;最后通过求解提出的一种优化函数来生成静态视频摘要.在Open Video Project和YouTube两个数据集上的大量主观与客观实验验证了所提HGRVS算法的优良性能.     

Cross-Layer Optimization for Video Summary Transmission over Wireless Networks

Video summarization has gained increased popularity in the emerging multimedia communication applications, however, very limited work has been conducted to address the transmission problem of video summary frames. In this paper, we propose a cross-layer optimization framework for delivering video summaries over wireless networks. Within a rate-distortion theoretical framework, the source coding, allowable retransmission, and adaptive modulation and coding have been jointly optimized, which reflects the joint selection of parameters at physical, data link and application layers. The goal is to achieve the best video quality and content coverage of the received summary frames and to meet the delay constraint. The problem is solved using Lagirangian relaxation and dynamic programming. Experimental results indicate the effectiveness and efficiency of the proposed optimization framework, especially when the delay budget imposed by the upper layer applications is small, where more than 10% distortion gain can be achieved.     

On the use of hierarchical prediction structures for efficient summary generation of H.264/AVC bitstreams

Video summarization refers to an important set of abstraction techniques aimed to provide a compact representation of the video essential to effectively browse and retrieve video content from multimedia repositories. Most of these video summarization techniques, such as image storyboards, video skims and fast previews, are based on selecting some frames or segments. H.264/AVC has become a widely accepted coding standard and is expected that many of the content will be available in this format soon. This paper proposes a generic model of video summarization especially suitable for generating summaries of H.264/AVC bitstreams in a highly efficient manner, using the concept of temporal scalability via hierarchical prediction structures. Along with the model, specific examples of summarization techniques are given to prove the utility of the model.     

Multiview video summarization using video partitioning and clustering

Multiview video summarization plays a crucial role in abstracting essential information form multiple videos of the same location and time. In this paper, we propose a new approach for the multiview summarization. The proposed approach uses the BIRCH clustering algorithm for the first time on the initial set of frames to get rid of the static and redundant. The work presents a new approach for shot boundary detection using frame similarity measures Jaccard and Dice. The algorithm performs effectively synchronized merging of keyframes from all camera-views to obtain the final summary. Extensive experimentation conducted on various datasets suggests that the proposed approach significantly outperforms most of the existing video summarization approaches. To state a few, a 1.5% improvement on video length reduction, 24.28% improvement in compression ratio, and 6.4% improvement in quality assessment ratio is observed on the lobby dataset.     

Multi-video synopsis for video representation

The world is covered with millions of cameras with each recording a huge amount of video. It is a time-consuming task to watch these videos, as most of them are of little interest due to the lack of activity. Video representation is thus an important technology to tackle with this issue. However, conventional video representation methods mainly focus on a single video, aiming at reducing the spatiotemporal redundancy as much as possible. In contrast, this paper describes a novel approach to present the dynamics of multiple videos simultaneously, aiming at a less intrusive viewing experience. Given a main video and multiple supplementary videos, the proposed approach automatically constructs a synthesized multi-video synopsis by integrating the supplementary videos into the most suitable spatiotemporal portions within this main video. The problem of finding suitable integration between the main video and supplementary videos is formulated as the maximum a posterior (MAP) problem, in which the desired properties related to a less intrusive viewing experience, i.e., informativeness, consistency, visual naturalness, and stability, are maximized. This problem is solved by using an efficient Viterbi beam search algorithm. Furthermore, an informative blending algorithm that naturalizes the connecting boundary between different videos is proposed.The proposed method has a wide variety of applications such as visual information representation, surveillance video browsing, video summarization, and video advertising. The effectiveness of multi-video synopsis is demonstrated in extensive experiments over different types of videos with different synopsis cases.     

An integrated approach to summarization and adaptation using H.264/MPEG-4 SVC

The huge amount of multimedia content and the variety of terminals and networks make video summarization and video adaptation two key technologies to provide effective access and browsing. With scalable video coding, the adaptation of video to heterogeneous terminals and networks can be efficiently achieved using together a layered coding hierarchy and bitstream extraction. On the other hand, many video summarization techniques can be seen as a special case of structural adaptation. This paper describes how some of them can be modified and included in the adaptation framework of the scalable extension of H.264/AVC. The advantage of this approach is that summarization and adaptation are integrated into the same efficient framework. The utility of this approach is demonstrated with experimental results for the generation of storyboards and video skims, showing that the proposed framework can generate the adapted bitstream of the summary faster than a conventional transcoding approach.     

新标准H.264的核心技术与应用前景

围绕视频传输要求高效、可靠和网络友好的三个焦点问题,深入研究H.264标准编码层压缩性能的提升、网络适配层的无缝集成和编/解码器的容错性能;以基于包交换的IP视频信息包为例,重点分析H.264对因特网RTP/UDP/IP协议的映射过程;阐述H.264在视频移动通信、视频流服务、数字电视广播与存储领域的典型应用特点;指出H.264对视频源压缩、信道有效利用、缓解视频传输瓶颈的影响和发展前景.     

基于Android手机的无线视频门禁系统应用研究

提出一种基于Android手机的视频门禁系统,具有灵活性好、功能丰富等优点.先从无线局域网通信方式和手机门禁终端两方面提出了系统方案特点,接着设计系统框架,并从视频数据采集与处理,服务器监控软件及Android手机客户端软件设计三方面对系统进行了详细设计,最后对Android客户端软件进行了简单测试.结果证明该方案可行,且运行稳定,有广泛的应用价值.     

基于Liod平台的视频数据采集系统

介绍一种基于嵌入式Linux和Liod平台上开发的视频数据采集系统。在简要介绍了Liod开发平台后,详细论述在该平台上如何实现视频数据采集,及其从系统硬件设计到系统软件设计的实现。通过USB摄像头获取实时视频,使用Video4Linux提供的API函数进行视频数据采集程序的设计。着重介绍视频数据采集程序的具体实现方法和过程,最后完成应用程序向目标平台的移植。     

基于FPGA的视频转换系统的实现

文章分析了视频转换中的关键技术：视频扫描转换和视频图象处理的基本原理，并给出了一种实际的实现方案，构建了以FPGA为控制核心的视频转换硬件系统。利用FPGA对整个系统进行编程配置，灵活地对系统进行控制，实现从非标准视频制式到标准视频制式以及标准制式之间的相互转换。     

一种基于交互信息量的视频摘要生成方法

提出了一种基于交互信息量的视频摘要生成方法。该方法首先使用基于交互信息量的方法进行视频镜头检测,通过对检测到镜头帧的聚类提取镜头候选关键帧。然后对候选关键帧按照相邻帧间交互信息量的比较来提取镜头关键帧,最后将镜头关键帧按时序排列在一起形成视频摘要。试验表明,这种关键帧提取算法是有效的,其建立的视频摘要能较好的反映原视频的内容。     

视频云在有线电视网实现交互运用的定量分析

视频云计算是基于云计算技术的理念,采用视频作为"云端"向"终端"呈现处理结果的一种云计算方案。介绍了视频云计算的主要技术特性,分析了视频云计算和广电行业结合的天然优势和典型的应用服务,并在此基础上定量分析了利用视频云计算技术开展2D/3D应用的投资成本,为广电行业利用视频云计算技术开展增值服务提供了一定的参考。     

HVR系统中多路视频接入和处理的软件设计

针对多路混合型硬盘录像机系统中多路摄像机视频的接入和处理问题,在基于服务器和编码板卡构架的HVR系统基础上提出了可靠高效的多路视频接入和处理软件方案,并实现了此方案.在多路视频接入和处理中,需对多通道视频并行处理及同一视频通道同时执行多个功能,针对此过程中可能存在的资源冲突问题进行了优化,提出了完善的多通道并发操作及视频码流分发机制、功能会话安全结束机制.测试结果表明,应用该软件方案的系统可接入能力强,资源占用率低、系统稳定可靠.     

基于Node-Cell结构的HEVC帧内编码

随着视频传输和广播的发展,高分辨率视频的应用也越来越广泛,为了更好地适应高清视频内容,JCT-VC(Joint Collaborative Team on Video Coding)工作组制定了具有更高压缩效率的新一代视频压缩标准HEVC(High Efficiency Video Coding)。HEVC中的帧内预测包括Angular预测模式、planar模式等。基于Node-Cell结构的帧内预测方法在Angular预测基础上实现了双向预测,提供了更多的模式选择和更高的预测精度。Node-Cell结构中所有像素在当前块被分成两个子集,node像素点和cell像素点,node像素的重建值被用于内插预测cell像素。新增的帧内模式信息被设计为表示下采样率,它由该编码单元的细节及复杂度决定。为了保证重建质量,node像素和cell像素的残差均被发送到解码器。实验结果表明Node-Cell结构会提高预测精度。     

Adaptive key frame extraction for video summarization using an aggregation mechanism

Video summarization is a method to reduce redundancy and generate succinct representation of the video data. One of the mechanisms to generate video summaries is to extract key frames which represent the most important content of the video. In this paper, a new technique for key frame extraction is presented. The scheme uses an aggregation mechanism to combine the visual features extracted from the correlation of RGB color channels, color histogram, and moments of inertia to extract key frames from the video. An adaptive formula is then used to combine the results of the current iteration with those from the previous. The use of the adaptive formula generates a smooth output function and also reduces redundancy. The results are compared to some of the other techniques based on objective criteria. The experimental results show that the proposed technique generates summaries that are closer to the summaries created by humans.     

Recognition of dynamic video contents with global probabilistic models of visual motion.

The exploitation of video data requires methods able to extract high-level information from the images. Video summarization, video retrieval, or video surveillance are examples of applications. In this paper, we tackle the challenging problem of recognizing dynamic video contents from low-level motion features. We adopt a statistical approach involving modeling, (supervised) learning, and classification issues. Because of the diversity of video content (even for a given class of events), we have to design appropriate models of visual motion and learn them from videos. We have defined original parsimonious global probabilistic motion models, both for the dominant image motion (assumed to be due to the camera motion) and the residual image motion (related to scene motion). Motion measurements include affine motion models to capture the camera motion and low-level local motion features to account for scene motion. Motion learning and recognition are solved using maximum likelihood criteria. To validate the interest of the proposed motion modeling and recognition framework, we report dynamic content recognition results on sports videos.     

Deep multi-query video retrieval

Video retrieval methods have been developed for a single query. Multi-query video retrieval problem has not been investigated yet. In this study, an efficient and fast multi-query video retrieval framework is developed. Query videos are assumed to be related to more than one semantic. The framework supports an arbitrary number of video queries. The method is built upon using binary video hash codes. As a result, it is fast and requires a lower storage space. Database and query hash codes are generated by a deep hashing method that not only generates hash codes but also predicts query labels when they are chosen outside the database. The retrieval is based on the Pareto front multi-objective optimization method. Re-ranking performed on the retrieved videos by using non-binary deep features increases the retrieval accuracy considerably. Simulations carried out on two multi-label video databases show that the proposed method is efficient and fast in terms of retrieval accuracy and time.