基于改进BEMD的视频镜头转场检测算法研究

对二维经验模式分解(BEMD)算法进行了改进,采用限定域的Delaunay三角剖分和三次插值得到极大值和极小值包络面,用基于限邻域经验模式分解(NLEMD),即通过设定最大邻域(时宽)和采用邻域内局部自适应均值算法代替包络均值算法进行分解,给出了图像BEMD分解后内蕴模函数(I MF)1和2的Hilbert谱,以I MF2的瞬时振幅作为图像的特征向量,计算镜头转场中图像序列帧特征向量间的欧式距离。采用大量的视频镜头转场的样本进行实验,结果表明,剪切镜头查准率和查全率皆为98%;渐变镜头查准率86.4%,查全率87.6%。     

引入视觉感知的视频镜头分割

视频的大数据时代已经到来,将视频序列分割成镜头来进行视频内容分析和视频检索是十分重要的研究方向。文中提出一种基于帧间一致(Frame Consistency,FC)模型和光流特征的视频镜头分割技术。利用基于视觉感知的"整体到局部"的思想,首先浏览视频,除去视频的冗余信息,以降低计算成本,并通过提取视频的视觉特征构建帧间一致性函数,以此创建可能的镜头分割集合,并结合运动特征进一步优化分割结果。该技术在评估上,其精确度、召回率和F1值,都呈现出较好的效果。     

用非监督式聚类进行视频镜头分割

镜头边界检测是基于内容的视频检索首先要解决的问题。研究人员通常将镜头转换分为突变和渐变,并根据各种转换的特点采用不同的检测算法。在研究中发现,视频镜头的抽象程度与对其进行边界划分的精度相关。为此,提出采用非监督式聚类算法,按照给定的相似尺度对视频数据进行自组织和动态分析,完成层次化的镜头分割。该算法侧重于揭示视频的层次结构,能实现不同精度的视频抽象要求。     

多特征融合的视频镜头分割

针对传统镜头分割算法特征单一、分割效果较差等问题,提出了一种多特征融合的视频镜头分割算法。首先,通过建立一种优化模型,融合空间差异度量和感知哈希度量两种重要特征,构造一种镜头边界区分能力更强的镜头边界检测特征,即像素差异度量;其次,结合直方图差异度量,形成有效的镜头边界检测策略,以便更加准确判断是否发生镜头切换。实验结果表明,与传统算法相比,所提算法取得了较好的分割效果,具有较高的查准率和查全率。     

On the representation of a continuous stochastic intensity by Poisson shot noise

In many applications a Poisson shot noise (PSN) process is said to statistically "represent" its intensity process. In this paper an investigation is made of the relationship between a PSN process and its intensity, when the latter is a sample function of a continuous stochastic process. The difference of the moments and the mean-square difference between the two processes are examined. The continuity assumption on the intensity permits the development of a sequence of moment relationships in which the effect of the PSN parameters can be seen. The results simplify and afford some degree of physical interpretation when the component functions of the PSN are "rectangular," or when the intensity process does not vary appreciably over their time width. An integral equation is derived that defines the component function that minimizes the mean-square difference between the two processes. It is shown that a "degenerate" form of component function induces complete statistical equality of the two processes. The problem has application to optical communication systems using photodetectors.     

Video representation with three-dimensional entities

Very low bit-rate coding requires new paradigms that go well beyond pixel- and frame-based video representations. We introduce a novel content-based video representation using tridimensional entities: textured object models and pose estimates. The multiproperty object models carry stochastic information about the shape and texture of each object present in the scene. The pose estimates define the position and orientation of the objects for each frame. This representation is compact. It provides alternative means for handling video by manipulating and compositing three-dimensional (3-D) entities. We call this representation tridimensional video compositing, or 3DVC for short. We present the 3DVC framework and describe the methods used to construct incrementally the object models and the pose estimates from unregistered noisy depth and texture measurements. We also describe a method for video frame reconstruction based on 3-D scene assembly, and discuss potential applications of 3DVC to video coding and content-based handling. 3DVC assumes that the objects in the scene are rigid and segmented. By assuming segmentation, we do not address the difficult questions of nonrigid segmentation and multiple object segmentation. In our experiments, segmentation is obtained via depth thresholding. It is important to notice that 3DVC is independent of the segmentation technique adopted. Experimental results with synthetic and real video sequences where compression ratios in the range of 1:150-1:2700 are achieved demonstrate the applicability of the proposed representation to very low bit-rate coding     

基于聚类的视频镜头分割和关键帧提取

镜头分割是基于内容的视频检索和浏览首先要解决的关键技术。视频分割为镜头后,下一步的工作就是进行关键帧提取,用以描述镜头的主要内容。提出了一种改进的基于聚类的镜头分割和关键帧提取算法．在无监督聚类中引入一个参考变量,解决了利用无监督聚类进行镜头分割和关键帧提取时可能产生的帧序不连续或分割错误的问题。在关键帧提取阶段,将镜头分割为子镜头后,引入图像熵的概念提取关键帧。实验结果表明了改进算法在镜头分割和关键帧提取方面的有效性。     

时空深度特征AP聚类的稀疏表示视频异常检测算法

针对异常行为检测问题, 提出基于时空深度特征的AP聚类稀疏表示视频异常检测方法。由于视频序列中大量背景信息及有效信息分布不均匀的情况，首先利用光流结合非均匀的细胞分割对视频的运动目标进行提取并得到空间尺寸大小不同的时空兴趣块。其次利用三维卷积神经网络提取不同时空兴趣块的时空深度特征从而对原始视频序列进行三维描述。然后在字典学习时，采用AP聚类方法，将训练样本中具有代表性的特征作为字典，极大降低字典维度以及稀疏表示方法对计算内存的要求。本文将测试样本进行AP聚类后仅对具有代表性的聚类中心进行检测，在减少实验时间的同时削减了阈值对检测效果的敏感度。实验结果表明，与现有的检测方法相比本文方法具有优越性。      

Modified Kolmogorov-Smirnov metric for shot boundary detection

A new metric extended from the existing Kolmogorov-Smirnov measure is proposed for detecting shot boundaries of video. Exploiting more fine-grained differences between the cumulative histograms of two successive frames, the proposed metric improves both the recall and precision of shot boundary detection. Experiments conducted on a comprehensive video set show that the new metric outperforms other widely-used histogram-based metrics.     

两种镜头检测算法的对比

对于基于内容的视频检索(CBVR)、索引、浏览等系统来说,镜头检测是第一步也是关键的一步.本文首先介绍了基于内容的视频检索系统的研究价值、镜头检测的分类,以及镜头检测算法的一般步骤.然后对两种镜头检测算法进行了详细介绍,一种是比较复杂的基于小波变换的算法,另一种是较为简单的基于颜色直方图的算法,按照这两种算法在MATL...     

镜头变换边界复合检测算法研究

将视频图像进行有效时间分段及标定是视频图像内容分析、随机接入、视频数据库浏览与检索的重要研究内容。在对视频图像进行时间分段和标定过程中,镜头变换边界的自动检测十分重要。镜头变换边界的自动检测算法在国际上得到了广泛的研究,在分析了当前国际上的镜头变换边界检测算法的基础上,提出了一种镜头变换边界复合检测算法。理论分析及实验结果表明该复合检测算法不仅具有实现复杂度较低,而且对各类型图像序列检测性能好的特点。     

High accuracy flashlight scene determination for shot boundary detection

Shot boundary detection, or scene change detection, is a technique used in the initial phase of video indexing. One of the problems in the detection is the discrimination of abrupt scene change from flashlight scenes. The usual discriminate method tests the similarity of the frame before and after a suspected flashlight effect. However, the performance of such a technique in discriminating flashlight scene from abrupt scene change can be affected by the scene content. To overcome this, we present a novel method that utilises the edge direction, thereby reducing erroneous matching with increasing dilation radius. This improves the accuracy of similarity testing and reduces the amount of erroneously matched edges by four times. Our experiment in discriminating flashlight effect from abrupt scene change frame pairs shows that our technique produces a perfect detection, which cannot be achieved by normal edge-based detection. Such a contribution is important as it improves the indexing of real life video.     

A motion and illumination resilient framework for automatic shot boundary detection

Detecting and locating a desired information in hefty amount of video data through manual procedure is very cumbersome. This necessitates segregation of large video into shots and finding the boundary between the shots. But shot boundary detection problem is unable to achieve satisfactory performance for video sequences consisting of flash light and complex object/camera motion. The proposed method is intended for recognising abrupt boundary between shots in the presence of motion and illumination change in an automatic way. Typically any scene change detection algorithm assimilates time separation in a shot resemblance metric. In this communication, absolute sum gradient orientation feature difference is matched to automatically generated threshold for sensing a cut. Experimental study on TRECVid 2001 data set and other publicly available data set certifies the potentiality of the proposed scheme that identifies scene boundaries efficiently, in a complex environment while preserving a good trade-off between recall and precision measure.     

Tsallis entropy-based information measures for shot boundary detection and keyframe selection

Automatic shot boundary detection and keyframe selection constitute major goals in video processing. We propose two different information-theoretic approaches to detect the abrupt shot boundaries of a video sequence. These approaches are, respectively, based on two information measures, Tsallis mutual information and Jensen–Tsallis divergence, that are used to quantify the similarity between two frames. Both measures are also used to find out the most representative keyframe of each shot. The representativeness of a frame is basically given by its average similarity with respect to the other frames of the shot. Several experiments analyze the behavior of the proposed measures for different color spaces (RGB, HSV, and Lab), regular binnings, and entropic indices. In particular, the Tsallis mutual information for the HSV and Lab color spaces with only 8 regular bins for each color component and an entropic index between 1.5 and 1.8 substantially improve the performance of previously proposed methods based on mutual information and Jensen–Shannon divergence.     

Video handling with music and speech detection

The audio-based approach to video indexing described by the authors detects music and speech independently even when they occur simultaneously. The indexed video segments, when presented on the Video Sound Browser, let users randomly access the video. The Video in Time system provides different video condensation levels based on video structuring that can link the video segments and the director's intentions     

Video denoising using shape-adaptive sparse representation over similar spatio-temporal patches

We present an effective patch-based video denoising algorithm that exploits both local and nonlocal correlations. The method groups 3D shape-adaptive patches, whose surrounding cubic neighborhoods along spatial and temporal dimensions have been found similar by patch clustering. Such grouping results in 4D data structures with arbitrary shapes. Since the obtained 4D groups are highly correlated along all the dimensions, they can be represented very sparsely with a 4D shape-adaptive DCT. The noise can be effectively attenuated by transform shrinkage. Experimental results on a wide range of videos show that this algorithm provides significant improvement over the state-of-the-art denoising algorithms in terms of both objective metric and subjective visual quality.     

基于自相似性约束的视频稀疏超分辨率重建

通过研究帧间自相似性对图像重建的影响,提出一种自相似性约束的单视频稀疏超分辨率重建算法,以达到保持图像局部结构完整性的同时有效去噪的目的。该算法运用主成分分析PCA训练出适应图像不同局部结构的分类词典;通过帧间光流场的粗略运动估计和帧内帧间的精确块匹配,搜索自相似信息,运用非局部均值NLM滤波,并以此约束稀疏模型。仿真实验表明,提出的算法无论是客观指标,还是主观视觉上都超过了进行比较的几种分辨率提高算法。     

A perceptual scheme for fully automatic video shot boundary detection

In this paper, we propose a novel and robust modus operandi for fast and accurate shot boundary detection where the whole design philosophy is based on human perceptual rules and the well-known “Information Seeking Mantra”. By adopting a top–down approach, redundant video processing is avoided and furthermore elegant shot boundary detection accuracy is obtained under significantly low computational costs. Objects within shots are detected via local image features and used for revealing visual discontinuities among shots. The proposed method can be used for detecting all types of gradual transitions as well as abrupt changes. Another important feature is that the proposed method is fully generic, which can be applied to any video content without requiring any training or tuning in advance. Furthermore, it allows a user interaction to direct the SBD process to the user's “Region of Interest” or to stop it once satisfactory results are obtained. Experimental results demonstrate that the proposed algorithm achieves superior computational times compared to the state-of-art methods without sacrificing performance.     

基于拉格朗日高斯变换的奥运会视频分类机制研究

随着现代体育不断发展,奥运会承办比赛种类越 来越繁杂,对赛事视频分类提出了一 个新的挑战。现有的人工分类方法无法有效地区分团体竞技类比赛(球类)和个人竞技类比 赛(田径类)视频,从而进行大规模自动分类存储。然而,为了有效地重复使用这些视频文 件,需要对其进行分类存储,主要目的在于提高资源的利用率。针对人工分类手段太过于低 效的现状,本文对奥运会运动视屏内容分类问题进行研究,并提出了一种基于关键帧特征提 取和支持向量机(Supported Vector Machine,SVM)的视频分类方法。以第31届奥运会的 体育视频作为数据集,对每个视频进行关键帧提取和总结,并借由拉格朗日-高斯变换来计 算视频对应的特征向量,将特征向量作为SVM分类器的输入进行体育视频分类。实验结果表 明,对于任意奥运视频,提出的方法平均能够取得70%以上的正确分类率,而错误分类的比 例始终低于10%。特别地,对于奥运中的射击类视频,平均正确分类 率接近90%左右,说明了提出方法的有效性。