Mining for video production invariants to measure style similarity

This article focuses on video document comparison using audiovisual production invariants (API). API are characterized by invariant segments obtained on a set of low‐level features. We propose an algorithm to detect production invariants throughout a collection of audiovisual documents. The algorithm runs on low‐level features, considered as time series, and extracts invariant segments using a one‐dimensional morphological envelop comparison. Then, based on the extracted results, we define a style similarity measure between two video documents. A derivative pseudo distance is also proposed. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 747–763, 2006.     

Inclusion measures,similarity measures,and the fuzziness of fuzzy sets and their relations

The inclusion measure, the similarity measure, and the fuzziness of fuzzy sets are three important measures in fuzzy set theory. In this article, we investigate the relations among inclusion measures, similarity measures, and the fuzziness of fuzzy sets, prove eight theorems that inclusion measures, similarity measures, and the fuzziness of fuzzy sets can be transformed by each other based on their axiomatic definitions, and propose some new formulas to calculate inclusion measures, similarity measures, and the fuzziness of fuzzy sets. These results can be applied in many fields, such as pattern recognition, image processing, fuzzy neural networks, fuzzy reasoning, and fuzzy control. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 639–653, 2006.     

基于索引结构的高效运动视频检索

视频片断检索是视频领域的研究热点,为了提高查询效率,利用高维索引结构Vector-Approximation File（VA-File）来组织视频子片段,并采用新的相似度模型和基于限定性滑动窗口的高效视频检索算法进行视频片段检索．提出的子片段的分割算法能够较好地区分运动的细节动作,且相似度模型充分考虑了对应子片段之间的视觉相似性以及时间顺序关系,因此对于运动视频的检索十分有效．实验证明,对于运动视频片段检索不仅具有较高的查询效率,而且能够得到较高的查全率和准确率．     

A Formal Model for Video Shot Segmentation and its Application via Animate Vision

The first step in a video indexing process is the segmentation of videos into meaningful parts called shots. In this paper we present a formal model of the video shot segmentation process. Starting from a mathematical characterization of the most common transition effects, a video segmentation algorithm capable to detect both abrupt and gradual transitions is proposed. The proposed algorithm is based on the computation of an arbitrary similarity measure between consecutive frames of a video. The algorithm has been tested adopting a similarity metric based on the Animate Vision theory and results have been reported.     

基于视频页的视频流分割方法

提出了基于视频页的视频流分割方法。该方法通过定义视频帧的色彩相近区、光照变化区和色彩变化区,获得视觉内容改变的局部区域,由此进一步确定两帧的视觉内容相似性,从而实现视频页的分割,经与几种有代表性的视频流分割算法的实验比较,证明该检测算法不仅对光照变化的影响具有很高的鲁棒性,而且使得分割能在较高信噪比上进行,明显提高了视频流分割的准确性。     

On a similarity measure between LR‐type fuzzy numbers and its application to database acquisition

This article presents a new similarity measure for LR‐type fuzzy numbers. The proposed similarity measure is based on a defined metric between LR‐type fuzzy numbers. It is known that an exponential operation is highly useful in dealing with the classical Shannon entropy and cluster analysis. We adopted, therefore, the exponential operation on this metric. Furthermore, we analyze its properties and make numerical comparisons to several similarity measures. The results show that the proposed similarity measure can overcome the drawbacks of the existing similarity measures. We then apply it to compound attributes for handling null queries to database systems. These applications can also be widely used in fuzzy queries to databases. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 1001–1016, 2005.     

一种通过视频片段进行视频检索的方法

视频片段检索是基于内容的视频检索的主要方式,它需要解决两个问题:(1) 从视频库里自动分割出与查询片段相似的多个片段;(2) 按照相似度从高到低排列这些相似片段.首次尝试运用图论的匹配理论来解决这两个问题.针对问题(1),把检索过程分为两个阶段:镜头检索和片段检索.在镜头检索阶段,利用相机运动信息,一个变化较大的镜头被划分为几个内容一致的子镜头,两个镜头的相似性通过对应子镜头的相似性计算得到;在片段检索阶段,通过考察相似镜头的连续性初步得到一个个相似片段,再运用最大匹配的Hungarian算法来确定真正的相似片段.针对问题(2),考虑了片段相似性判断的视觉、粒度、顺序和干扰因子,提出用最优匹配的Kuhn-Munkres算法和动态规划算法相结合,来解决片段相似度的度量问题.实验对比结果表明,所提出的方法在片段检索中可以取得更高的检索精度和更快的检索速度.     

基于运动相似性的运动对象分割算法

由于全局运动参数在一些情况下不能很准确地被估计,因此基于全局运动补偿算法存在一定的局限性。该文提出了一种基于运动相似性的视频运动对象的分割算法。该算法对运动矢量场进行累加和滤波处理,根据运动相似性对运动对象进行初次分割,并利用空间相关性进行二次分割,细化运动对象边缘。实验结果表明该算法避免了全局运动补偿算法的局限性,并使分割的精确性有明显提高。     

Activity based surveillance video content modelling

This paper tackles the problem of surveillance video content modelling. Given a set of surveillance videos, the aims of our work are twofold: firstly a continuous video is segmented according to the activities captured in the video; secondly a model is constructed for the video content, based on which an unseen activity pattern can be recognised and any unusual activities can be detected. To segment a video based on activity, we propose a semantically meaningful video content representation method and two segmentation algorithms, one being offline offering high accuracy in segmentation, and the other being online enabling real-time performance. Our video content representation method is based on automatically detected visual events (i.e. ‘what is happening in the scene’). This is in contrast to most previous approaches which represent video content at the signal level using image features such as colour, motion and texture. Our segmentation algorithms are based on detecting breakpoints on a high-dimensional video content trajectory. This differs from most previous approaches which are based on shot change detection and shot grouping. Having segmented continuous surveillance videos based on activity, the activity patterns contained in the video segments are grouped into activity classes and a composite video content model is constructed which is capable of generalising from a small training set to accommodate variations in unseen activity patterns. A run-time accumulative unusual activity measure is introduced to detect unusual behaviour while usual activity patterns are recognised based on an online likelihood ratio test (LRT) method. This ensures robust and reliable activity recognition and unusual activity detection at the shortest possible time once sufficient visual evidence has become available. Comparative experiments have been carried out using over 10 h of challenging outdoor surveillance video footages to evaluate the proposed segmentation algorithms and modelling approach.     

Space-time behavior-based correlation-Or-how to tell if two underlying motion fields are similar without computing them?

We introduce a behavior-based similarity measure which tells us whether two different space-time intensity patterns of two different video segments could have resulted from a similar underlying motion field. This is done directly from the intensity information, without explicitly computing the underlying motions. Such a measure allows us to detect similarity between video segments of differently dressed people performing the same type of activity. It requires no foreground/background segmentation, no prior learning of activities, and no motion estimation or tracking. Using this behavior-based similarity measure, we extend the notion of 2-dimensional image correlation into the 3-dimensional space-time volume, thus allowing to correlate dynamic behaviors and actions. Small space-time video segments (small video clips) are "correlated" against entire video sequences in all three dimensions (x,y, and t). Peak correlation values correspond to video locations with similar dynamic behaviors. Our approach can detect very complex behaviors in video sequences (e.g., ballet movements, pool dives, running water), even when multiple complex activities occur simultaneously within the field-of-view of the camera. We further show its robustness to small changes in scale and orientation of the correlated behavior.     

Fast level set image and video segmentation using new evolution indicator operators

We propose an effective level set evolution method for robust object segmentation in real images. We construct an effective region indicator and an multiscale edge indicator, and use these two indicators to adaptively guide the evolution of the level set function. The multiscale edge indicator is defined in the gradient domain of the multiscale feature-preserving filtered image. The region indicator is built on the similarity map between image pixels and user specified interest regions, where the similarity map is computed using Gaussian Mixture Models (GMM). Then we combine these two methods to develop a new mixing edge stop function, which makes the level set method more robust to initial active contour setting, and forces the level set to evolve adaptively based on the image content. Furthermore, we apply an acceleration approach to speed up our evolution process, which yields real time segmentation performance. Finally, we extend the proposed approach to video segmentation for achieving effective target tracking results. As the results show, our approach is effective for image and video segmentation and works well to accurately detect the complex object boundaries in real-time.     

视频目标分割中帧间相似性传播的研究

目前视频目标分割算法多是基于匹配和传播策略分割目标,常常以掩模或者光流的方式利用前一帧的信息,探索了新的帧间特征传播方式,利用短时匹配模块提取前一帧信息并传播给当前帧,提出一种面向视频序列数据的目标分割模型。通过长时匹配模块和短时匹配模块分别与第一帧和前一帧做相关操作进行像素级匹配,得到的全局相似性图和局部相似性图,以及前一帧的掩模和当前帧的特征图,经过两个优化网络后通过分割网络得到分割结果。在视频目标分割公开数据集上的实验表明,所提出方法在单目标和多目标上分别取得了86.5%和77.4%的区域相似度和轮廓精度均值,每秒可计算21帧。提出的短时匹配模块比仅使用掩模更有利于提取前一帧的信息,通过长时匹配模块和短时匹配模块的结合,不使用在线微调即可实现高效的视频目标分割,适合应用于移动机器人视觉感知。     

基于深度学习的视频语义分割综述

目前对视频语义分割的研究主要分为两方面,一是如何利用视频帧之间的时序信息提高图像分割的精度;二是如何利用视频帧之间的相似性确定关键帧,减少计算量,提升模型的运行速度.在提升分割精度方面一般设计新的模块,将新模块与现有的CNNs结合;在减少计算量方面,利用帧序列的低层特征相关性选择关键帧,同时减少操作时间.本文首先介绍视频语义分割的发展背景与操作数据集Cityscapes、CamVid;其次,介绍现有的视频语义分割方法;最后总结当前视频语义分割的发展情况,并对未来的发展给出一些展望和建议.     

Fast gradual matching measure for image retrieval based on visual similarity and spatial relations

In this article, we propose a new method to retrieve images containing a request set of regions. The user is asked to specify a set of regions belonging to a single image. Then this request set of regions is compared to the sets of the regions of the images in the database. We propose a comparison measure that not only evaluates the similarity of regions one to the other, but that also takes into account the spatial configuration of the regions. The spatial structure of the regions is represented by means of fuzzy spatial relations, like horizontal and vertical disposal and connexity. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 711–723, 2006.     

时间序列的层次分段及相似性度量

时间序列的相似性度量是时间序列数据挖掘的研究基础,为数据挖掘任务的效率和准确度提供可靠的保障。提出一种时间序列的层次分段及相似性度量方法,方法首先识别时间序列中的极值点,依据极值点的特征对时间序列进行分层次分段,并以此为基础,通过定义新的距离公式来度量时间序列间的相似性。使用新提出的相似性度量方法对时间序列进行聚类计算,实验结果表明,该方法能够有效地度量时间序列间的相似性,聚类效果明显,具有较好的实用性和良好的应用前景。     

Scene Detection in Videos Using Shot Clustering and Sequence Alignment

Video indexing requires the efficient segmentation of video into scenes. The video is first segmented into shots and a set of key-frames is extracted for each shot. Typical scene detection algorithms incorporate time distance in a shot similarity metric. In the method we propose, to overcome the difficulty of having prior knowledge of the scene duration, the shots are clustered into groups based only on their visual similarity and a label is assigned to each shot according to the group that it belongs to. Then, a sequence alignment algorithm is applied to detect when the pattern of shot labels changes, providing the final scene segmentation result. In this way shot similarity is computed based only on visual features, while ordering of shots is taken into account during sequence alignment. To cluster the shots into groups we propose an improved spectral clustering method that both estimates the number of clusters and employs the fast global k-means algorithm in the clustering stage after the eigenvector computation of the similarity matrix. The same spectral clustering method is applied to extract the key-frames of each shot and numerical experiments indicate that the content of each shot is efficiently summarized using the method we propose herein. Experiments on TV-series and movies also indicate that the proposed scene detection method accurately detects most of the scene boundaries while preserving a good tradeoff between recall and precision.     

基于镜头竞争力的多模态视频场景分割算法

为了能快速、有效地进行视频场景分割,论文提出一种基于镜头竞争力的多模态视频场景分割算法,充分考虑视频中多模态之间的时序关联共生特性,通过对视频物理特征的提取、融合计算出镜头间相似度,结合镜头竞争力的判定思想分割出视频场景.实验结果表明,该算法能较为高效地进行视频场景分割,查全率和查准率可达82.1％和86.7％.     

Integration on finite sets

Various types of integrals with respect to signed fuzzy measures on finite sets with cardinality n can be presented as corresponding rules for partitioning the integrand. The partition can be expressed as an n‐dimensional vector, whereas the signed fuzzy measure is also an n‐dimensional vector. Thus, the integration value is the inner product of these two vectors. Two pairs of extremes, the Lebesgue‐like integral versus the Choquet integral and the upper integral versus the lower integral, are discussed in detail. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 1073–1092, 2006.