相似视频片段的检测与定位方法研究

相似视频片段探测可以辅助网络视频检索、内容关联分析等方面的研究,具有重要的意义。重点研究了位置随机的相似视频片段的探测与定位问题,首先在视频结构化分析与关键帧提取的基础上,对不同视频进行相似关键帧探测。为保证探测的精度与效率,针对视频关键帧的特点,采用了FAST检测子和BRIEF描述子相结合的方法,利用关键帧的局部特征进行相似关键帧探测;其次提出了一种相似关键帧距离度量的方法,利用相似关键帧所在源视频的位置来构建相似关键帧距离矩阵,保留矩阵中距离较小的相似关键帧,将寻找相似视频片段的过程转化为寻找矩阵对应的连通图的过程。最后对算法进行了实验,结果表明,该方法可以有效地探测处于各个位置的相似视频片段。     

一种通过视频片段进行视频检索的方法

视频片段检索是基于内容的视频检索的主要方式,它需要解决两个问题:(1) 从视频库里自动分割出与查询片段相似的多个片段;(2) 按照相似度从高到低排列这些相似片段.首次尝试运用图论的匹配理论来解决这两个问题.针对问题(1),把检索过程分为两个阶段:镜头检索和片段检索.在镜头检索阶段,利用相机运动信息,一个变化较大的镜头被划分为几个内容一致的子镜头,两个镜头的相似性通过对应子镜头的相似性计算得到;在片段检索阶段,通过考察相似镜头的连续性初步得到一个个相似片段,再运用最大匹配的Hungarian算法来确定真正的相似片段.针对问题(2),考虑了片段相似性判断的视觉、粒度、顺序和干扰因子,提出用最优匹配的Kuhn-Munkres算法和动态规划算法相结合,来解决片段相似度的度量问题.实验对比结果表明,所提出的方法在片段检索中可以取得更高的检索精度和更快的检索速度.     

基于感知概念和有限状态机的体育视频语义内容分析模型

视频内容自动分析领域中,关键的挑战在干如何识别重要对象和如何建模对象之间的时空关系.本文基于感知概念(Perception Concepts,简称PCs)和有限状态机(Finite State Machines.简称FSMs)提出一种语义内容分析模型自动描述和探测体育视频中有意义的语义内容.根据体育视频中可识别的特征元素,定义PCs来表示视频中重要的语义模式;设计PC-FSM模型来描述PCs间的时空关系;采用一个图匹配方法自动探测视频中的高层语义.本文提出的方法使用户能够根据其自身的兴趣和知识设计体育视频的查询描述,并将语义内容探测问题转换为图匹配问题.实验结果验证了本文提出的方法的有效性.     

利用等价关系理论进行视频片段检索的方法

视频片段检索是基于内容的视频检索的主要方式,可是现有的片段检索方法大多只是对预先分割好的片段进行检索。为了从连续的视频节目中自动分割出多个相似的片段,提出了一种新的有效的视频片段检索方法,并首次尝试将等价关系理论应用于视频片段的检索.该方法首先用等价关系理论定义了片段匹配函数,同时采用滑动镜头窗自动分割出多个真正相似的片段;然后把等价类映射为矩阵表达形式,再通过矩阵的特性来度量影响片段相似度的不同因子,实现了相似片段的排序。实验结果表明,该方法能够一次性快速准确地从连续视频库中自动分割出与查询片段相似的多个片段。     

视频描述中全局-局部联合语义生成网络

针对视频描述中语义特征不能兼顾全局概括信息和局部细节信息,影响视频描述效果的问题,提出一种视频描述中全局-局部联合语义生成网络GLS-Net.首先利用全局与局部信息的互补性设计全局和局部语义提取单元, 2个单元采用残差结构的多层感知机(residualmulti-layerperceptron,r-MLP)来增强特征提取效果;然后联合概括性全局语义和细节性局部语义增强语义特征的表达能力;最后将该语义特征作为视频内容编码,提升视频描述模型性能.在MSR-VTT和MSVD数据集上,以语义辅助视频描述(semantics-assistedvideocaptioningnetwork,SAVC)网络为基础进行的实验的结果表明, GLS-Net优于现有同类算法,与SAVC网络相比,准确率平均提升6.2%.     

基于语义轨迹的视频事件探测

视频事件探测是视频内容自动理解领域的一个重要研究问题.在视频事件探测中,感兴趣对象的运动轨迹常被作为视频中探测事件的一种重要依据.目前基于轨迹的事件探测方法主要集中于根据轨迹几何特征进行视频事件探测,而忽略了与轨迹相关的语义信息.然而我们知道,轨迹的产生往往受到一些与轨迹相关联的语义信息的影响,如轨迹产生时的地理信息等.将轨迹相关联的语义信息整合到轨迹中可以使我们了解更多关于轨迹的信息.语义轨迹为我们提供了一个将语义信息与轨迹信息有效整合的方法.该文将语义轨迹应用到视频事件探测领域,提出了一个基于语义轨迹的视频事件探测方法.该方法将视频中抽取的感兴趣对象的原始轨迹转化为语义轨迹,并根据语义轨迹探测可能的视频事件.同时该方法还提供了一个描述语义轨迹特征以及对语义轨迹与轨迹特征进行匹配的方法.最后我们通过实验分析验证了基于语义轨迹的视频事件探测方法的有效性.     

基于可变步长关键帧提取的网络视频拷贝检索

为快速有效地检测网络中的拷贝视频,针对现有关键帧提取算法时间复杂度高、关键帧不具有代表性的缺点,提出一种可变步长提取关键帧提取方法。根据视频变化连续性特点,用相似的两近邻帧代表它们之间的视频片段;该方法首先选取关键帧中的核心区域与受影响较小的边缘区域,对不同的区域取权值并通过转换距离度量分块灰度顺序特征(OM)来判断两帧间相似度;然后利用滑动窗口来查找最大相似匹配,从而检测出查询视频中的拷贝片段。在网络数据和MUSCLE-VCD-2007数据上的实验结果表明,该方法相对于现有的基于OM特征拷贝检测法而言,其鲁棒性更强,检测效率更高。     

基于局部匹配窗口的动作识别方法

在词袋模型基础上,综合考虑姿态之间的时序约束关系,提出一种基于局部匹配窗口的动作识别方法.首先采用人体姿态差别作为动作序列特征描述.其次,在模型学习阶段,使用局部训练法而非传统的整体训练法来提高特征词汇的表征性;在特征量化阶段,使用自适应局部线性重构策略来给特征基更灵活的权值;在对象描述阶段,分别使用时间金字塔、滑动窗口2种方法将整个动作序列划分成多个局部动作片段,进而通过连接各个局部动作片段的特征来描述整个动作序列.最后使用直方图相交操作来完成特征匹配工作.在MSR Action3D数据库上测试了所提算法的性能并与已有的动作识别方法进行对比,结果表明,该方法的识别效果较优.     

羽毛球视频自动剪辑技术研究

当前针对羽毛球比赛视频的自动剪辑技术尚不完善,该文通过分析完整的羽毛球比赛视频,将其分为比赛片段、回放片段与人物特写片段,并采用图像处理技术实现三类片段的完整剪辑。该文针对比赛片段提出了结构相似度判断法与霍夫直线检验法,针对回放片段采用了SIFT特征匹配法与轮廓检测法,针对人物特写片段则采用了深度学习方法,并对各种方法的实验结果进行评价和分析,为羽毛球视频自动剪辑系统的实现提供了理论基础。     

基于视频指纹的视频片段检索方法

提出一种基于视频指纹的视频片段检索方法,通过视频指纹的相似性匹配对视频片段进行检索。通过改进的Harris检测计算DC图像的角点信息,去掉视频图像边缘提取视频运动特征,根据图像特征相似度和运动特征相似度的调和因子得到视频片段总的相似度。实验结果表明,与现有方法相比,该方法能准确检测视频片段,具有较强的鲁棒性,满足视频检索的实时要求。     

A spatio-temporal pyramid matching for video retrieval

An efficient video retrieval system is essential to search relevant video contents from a large set of video clips, which typically contain several heterogeneous video clips to match with. In this paper, we introduce a content-based video matching system that finds the most relevant video segments from video database for a given query video clip. Finding relevant video clips is not a trivial task, because objects in a video clip can constantly move over time. To perform this task efficiently, we propose a novel video matching called Spatio-Temporal Pyramid Matching (STPM). Considering features of objects in 2D space and time, STPM recursively divides a video clip into a 3D spatio-temporal pyramidal space and compares the features in different resolutions. In order to improve the retrieval performance, we consider both static and dynamic features of objects. We also provide a sufficient condition in which the matching can get the additional benefit from temporal information. The experimental results show that our STPM performs better than the other video matching methods.     

Abnormal visual event detection based on multi-instance learning and autoregressive integrated moving average model in edge-based Smart City surveillance

The abnormal visual event detection is an important subject in Smart City surveillance where a lot of data can be processed locally in edge computing environment. Real-time and detection effectiveness are critical in such an edge environment. In this paper, we propose an abnormal event detection approach based on multi-instance learning and autoregressive integrated moving average model for video surveillance of crowded scenes in urban public places, focusing on real-time and detection effectiveness. We propose an unsupervised method for abnormal event detection by combining multi-instance visual feature selection and the autoregressive integrated moving average model. In the proposed method, each video clip is modeled as a visual feature bag containing several subvideo clips, each of which is regarded as an instance. The time-transform characteristics of the optical flow characteristics within each subvideo clip are considered as a visual feature instance, and time-series modeling is carried out for multiple visual feature instances related to all subvideo clips in a surveillance video clip. The abnormal events in each surveillance video clip are detected using the multi-instance fusion method. This approach is verified on publically available urban surveillance video datasets and compared with state-of-the-art alternatives. Experimental results demonstrate that the proposed method has better abnormal event detection performance for crowded scene of urban public places with an edge environment.     

基于拓扑独立成分分析和高斯混合模型的视频语义概念检测

针对目前词袋模型(BoW)视频语义概念检测方法中的量化误差问题,为了更有效地自动提取视频的底层特征,提出一种基于拓扑独立成分分析(TICA)和高斯混合模型(GMM)的视频语义概念检测算法。首先,通过TICA算法进行视频片段的特征提取,该特征提取算法能够学习到视频片段复杂不变性特征;其次利用GMM方法对视频视觉特征进行建模,描述视频特征的分布情况;最后构造视频片段的GMM超向量,采用支持向量机(SVM)进行视频语义概念检测。GMM是BoW概率框架下的拓展,能够减少量化误差,具有良好的鲁棒性。在TRECVID 2012和OV两个视频库上,将所提方法与传统的BoW、SIFT-GMM方法进行了对比实验,结果表明,基于TICA和GMM的视频语义概念检测方法能够提高视频语义概念检测的准确率。     

基于改进快速鲁棒性特征的导弹视频特征匹配

利用电视制导导弹视频图像确定导弹落点,从而开展精确目标毁伤评估研究,是目前全新的一种评估手段。图像特征匹配是利用视频图像确定导弹落点的关键步骤。针对导弹视频图像的特点及其作战应用,在特征匹配阶段,依据准确性和实时性两个原则,对快速鲁棒性特征算法做了两方面的改进:一是限制特征点提取区域,定义了图像区域限制算子;二是限制特征点数量,利用算法阈值和随机抽样一致性算法对特征点进行限制。实验结果表明,提出的算法对典型视频片段进行处理,较原算法在匹配时间上平均减少12.6%,匹配准确率平均提升13.4%,较尺度不变特征变换算法匹配时间平均提升了74.9%,同时,有效消除伪匹配点。通过对三段视频进行测试仿真,改进算法在整体上较原算法的匹配时间加快14.9%,且通用性较强,适用于视频图像的特征点匹配。     

Pattern classification models for classifying and indexing audio signals

In the age of digital information, audio data has become an important part in many modern computer applications. Audio classification and indexing has been becoming a focus in the research of audio processing and pattern recognition. In this paper, we propose effective algorithms to automatically classify audio clips into one of six classes: music, news, sports, advertisement, cartoon and movie. For these categories a number of acoustic features that include linear predictive coefficients, linear predictive cepstral coefficients and mel-frequency cepstral coefficients are extracted to characterize the audio content. The autoassociative neural network model (AANN) is used to capture the distribution of the acoustic feature vectors. Then the proposed method uses a Gaussian mixture model (GMM)-based classifier where the feature vectors from each class were used to train the GMM models for those classes. During testing, the likelihood of a test sample belonging to each model is computed and the sample is assigned to the class whose model produces the highest likelihood. Audio clip extraction, feature extraction, creation of index, and retrieval of the query clip are the major issues in automatic audio indexing and retrieval. A method for indexing the classified audio using LPCC features and k-means clustering algorithm is proposed.     

Effective and Efficient Query Processing for Video Subsequence Identification

With the growing demand for visual information of rich content, effective and efficient manipulations of large video databases are increasingly desired. Many investigations have been made on content-based video retrieval. However, despite the importance, video subsequence identification, which is to find the similar content to a short query clip from a long video sequence, has not been well addressed. This paper presents a graph transformation and matching approach to this problem, with extension to identify the occurrence of potentially different ordering or length due to content editing. With a novel batch query algorithm to retrieve similar frames, the mapping relationship between the query and database video is first represented by a bipartite graph. The densely matched parts along the long sequence are then extracted, followed by a filter-and-refine search strategy to prune some irrelevant subsequences. During the filtering stage, maximum size matching is deployed for each subgraph constructed by the query and candidate subsequence to obtain a smaller set of candidates. During the refinement stage, sub-maximum similarity matching is devised to identify the subsequence with the highest aggregate score from all candidates, according to a robust video similarity model that incorporates visual content, temporal order, and frame alignment information. The performance studies conducted on a long video recording of 50 hours validate that our approach is promising in terms of both search accuracy and speed.     

基于三维小波变换视频片断检索技术研究

提出了一种基于三维小波的视频片断检索的新方法,该方法克服了传统视频检索方法中未考虑的时空关系.首先利用低频子图小波系数的标准方差均值对目标视频库进行初步筛选,然后对利用八叉树算法提取细节特征,进一步的把细节特征和近似特征组合起来对图像进行精确的检索.实验证明,该方法具有更好的检索精度.