Selecting salient frames for spatiotemporal video modeling and segmentation.

We propose a new statistical generative model for spatiotemporal video segmentation. The objective is to partition a video sequence into homogeneous segments that can be used as "building blocks" for semantic video segmentation. The baseline framework is a Gaussian mixture model (GMM)-based video modeling approach that involves a six-dimensional spatiotemporal feature space. Specifically, we introduce the concept of frame saliency to quantify the relevancy of a video frame to the GMM-based spatiotemporal video modeling. This helps us use a small set of salient frames to facilitate the model training by reducing data redundancy and irrelevance. A modified expectation maximization algorithm is developed for simultaneous GMM training and frame saliency estimation, and the frames with the highest saliency values are extracted to refine the GMM estimation for video segmentation. Moreover, it is interesting to find that frame saliency can imply some object behaviors. This makes the proposed method also applicable to other frame-related video analysis tasks, such as key-frame extraction, video skimming, etc. Experiments on real videos demonstrate the effectiveness and efficiency of the proposed method.     

CNN spatiotemporal features and fusion for surveillance video forgery detection

Surveillance cameras are widely used to provide protection and security; also their videos are used as strong evidences in the courts. Through the availability of video editing tools, it has become easy to distort these evidences. Sometimes, to hide the traces of forgery, some post-processing operations are performed after editing. Hence, the authenticity and integrity of surveillance videos have become urgent to scientifically validate. In this paper, we propose inter-frame forgeries (frame deletion, frame insertion, and frame duplication) detection system using 2D convolution neural network (2D-CNN) of spatiotemporal information and fusion for deep automatically feature extraction; Gaussian RBF multi-class support vector machine (RBF-MSVM) is used for classification process. The experimental results show that the efficiency of the proposed system for detecting all inter-frame forgeries, even when the forged videos have undergone additional post-processing operations such as Gaussian noise, Gaussian blurring, brightness modifications and compression.     

Saliency model-based face segmentation and tracking in head-and-shoulder video sequences

In this paper, a novel face segmentation algorithm is proposed based on facial saliency map (FSM) for head-and-shoulder type video application. This method consists of three stages. The first stage is to generate the saliency map of input video image by our proposed facial attention model. In the second stage, a geometric model and an eye-map built from chrominance components are employed to localize the face region according to the saliency map. The third stage involves the adaptive boundary correction and the final face contour extraction. Based on the segmented result, an effective boundary saliency map (BSM) is then constructed, and applied for the tracking based segmentation of the successive frames. Experimental evaluation on test sequences shows that the proposed method is capable of segmenting the face area quite effectively.     

基于色彩和纹理特征融合的模糊人脸识别方法

基于纹理特征的方法被广泛应用于人脸识别。然而纹理特征依赖于图像的高频细节信息,当图像出现模糊时,单纯利用纹理特征的识别方法的识别精度会急剧下降。为了克服纹理特征的在模糊人脸识别中的不足,提出了一种基于色彩特征和纹理特征融合的识别方法。首先参照人类的对立色感知机制提取人脸的色彩特征;然后,将该色彩特征和纹理特征分别用于识别分类;最后,将二者的识别相似度进行融合,得到最终的识别结果。该色彩特征描述了图像的低频信息,其对图像模糊不敏感,并且与描述图像高频信息的纹理特征具有良好的互补性。在FERET 和AR 人脸库上的实验表明,融合色彩特征和纹理特征有效地提高了模糊人脸的识别精度。     

Color local texture features for color face recognition

This paper proposes new color local texture features, i.e., color local Gabor wavelets (CLGWs) and color local binary pattern (CLBP), for the purpose of face recognition (FR). The proposed color local texture features are able to exploit the discriminative information derived from spatiochromatic texture patterns of different spectral channels within a certain local face region. Furthermore, in order to maximize a complementary effect taken by using both color and texture information, the opponent color texture features that capture the texture patterns of spatial interactions between spectral channels are also incorporated into the generation of CLGW and CLBP. In addition, to perform the final classification, multiple color local texture features (each corresponding to the associated color band) are combined within a feature-level fusion framework. Extensive and comparative experiments have been conducted to evaluate our color local texture features for FR on five public face databases, i.e., CMU-PIE, Color FERET, XM2VTSDB, SCface, and FRGC 2.0. Experimental results show that FR approaches using color local texture features impressively yield better recognition rates than FR approaches using only color or texture information. Particularly, compared with grayscale texture features, the proposed color local texture features are able to provide excellent recognition rates for face images taken under severe variation in illumination, as well as for small- (low-) resolution face images. In addition, the feasibility of our color local texture features has been successfully demonstrated by making comparisons with other state-of-the-art color FR methods.     

Histogram-based segmentation in a perceptually uniform color space

In this work, we present a segmentation algorithm for color images that uses the watershed algorithm to segment either the two-dimensional (2-D) or the three-dimensional (3-D) color histogram of an image. For compliance with the way humans perceive color, this segmentation has to take place in a perceptually uniform color space like the Luv space. To avoid oversegmentation, the watershed algorithm has to be applied to a smoothed histogram.     

Multi-view video based multiple objects segmentation using graph cut and spatiotemporal projections

In this paper, we present an automatic algorithm to segment multiple objects from multi-view video. The Initial Interested Objects (IIOs) are automatically extracted in the key view of the initial frame based on the saliency model. Multiple objects segmentation is decomposed into several sub-segmentation problems, and solved by minimizing the energy function using binary label graph cut. In the proposed novel energy function, the color and depth cues are integrated with the data term, which is then modified with background penalty with occlusion reasoning. In the smoothness term, foreground contrast enhancement is developed to strengthen the moving objects boundary, and at the same time attenuates the background contrast. To segment the multi-view video, the coarse predictions of the other views and the successive frame are projected by pixel-based disparity and motion compensation, respectively, which exploits the inherent spatiotemporal consistency. Uncertain band along the object boundary is shaped based on activity measure and refined with graph cut, resulting in a more accurate Interested Objects (IOs) layer across all views of the frames. The experiments are implemented on a couple of multi-view videos with real and complex scenes. Excellent subjective results have shown the robustness and efficiency of the proposed algorithm.     

A hybrid color and frequency features method for face recognition

This correspondence presents a novel hybrid Color and Frequency Features (CFF) method for face recognition. The CFF method, which applies an Enhanced Fisher Model (EFM), extracts the complementary frequency features in a new hybrid color space for improving face recognition performance. The new color space, the RIQ color space, which combines the R component image of the RGB color space and the chromatic components I and Q of the YIQ color space, displays prominent capability for improving face recognition performance due to the complementary characteristics of its component images. The EFM then extracts the complementary features from the real part, the imaginary part, and the magnitude of the R image in the frequency domain. The complementary features are then fused by means of concatenation at the feature level to derive similarity scores for classification. The complementary feature extraction and feature level fusion procedure applies to the I and Q component images as well. Experiments on the Face Recognition Grand Challenge (FRGC) version 2 Experiment 4 show that i) the hybrid color space improves face recognition performance significantly, and ii) the complementary color and frequency features further improve face recognition performance.     

Compressed domain video saliency detection using global and local spatiotemporal features

A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.     

基于改进的2-D Otsu方法和YCgCr空间的肤色分割

为了弥补基于固定阈值的肤色分割方法存在的缺陷,在对多种彩色空间和肤色模型进行分析的基础上,提出采用改进的2-D Otsu方法和YCgCr彩色空间进行肤色分割。首先将光照补偿之后的肤色样本图像从RGB彩色空间转换到YCgCr彩色空间,并利用样本图像上的179221个肤色点建立2维高斯模型;进而将待分割的图像进行光照补偿并转换到YCgCr彩色空间,利用已经建立的高斯模型计算图像的肤色相似度,得到肤色相似度图像;最后,结合像素的空间邻域信息,使用改进的2-D Otsu方法对肤色相似度图像进行2值化处理。对这种方法进行了理论分析和实验验证。结果表明,该肤色分割算法有效地克服了使用固定阈值法进行图像分割时缺乏针对性和抗噪性的缺陷,该算法是可行的。     

基于彩色空间的二维OSTU的图像分割算法

图像分割的研究一直是图像处理研究的热点问题,尤其是对彩色图像的分割研究更为重要,虽然对彩色图像分割的研究提出很多分割算法,但是很多算法仍存在缺陷,本文针对解决二维OSTU分割算法分割图像时计算复杂和易受噪声干扰的问题,提出将Lab彩色空间应用到二维OSTU算法中,首先将色彩图像从RGB空间转到Lab空间,然后联合利用L通道、a通道、b通道图像信息进行粗分割,最后针对其中某个通道的图像信息进行二维OSTU细分割.通过试验表明,该方法对彩色图像有较好的分割效果.     

Computer-aided tumor detection in endoscopic video using color wavelet features

We present an approach to the detection of tumors in colonoscopic video. It is based on a new color feature extraction scheme to represent the different regions in the frame sequence. This scheme is built on the wavelet decomposition. The features named as color wavelet covariance (CWC) are based on the covariances of second-order textural measures and an optimum subset of them is proposed after the application of a selection algorithm. The proposed approach is supported by a linear discriminant analysis (LDA) procedure for the characterization of the image regions along the video frames. The whole methodology has been applied on real data sets of color colonoscopic videos. The performance in the detection of abnormal colonic regions corresponding to adenomatous polyps has been estimated high, reaching 97% specificity and 90% sensitivity.     

基于一种动态颜色模型的模糊C-均值图像分割

不同颜色的可见光本质上是具有不同波长范围的电磁波.本文试探性地提出了一种动态颜色模型,它模拟了成像曝光时间内图像平面所接收到的电磁波的动态变化.离散化之后,彩色图像的颜色特征能够被表示成一个K维矢量,称为彩色图像的动态颜色空间表示.然后建立了模糊C-均值分割算法,分别在动态颜色空间和RGB空间分割彩色图像,实验结果表明动态颜色空间的分割结果优于RGB空间的分割,从而验证了动态颜色空间的性能.笔者相信本文所提出的动态颜色模型也能够被用于纹理分析或其它的图像处理领域.     

Modified intelligent scissors and video decomposing for video object segmentation

A semiautomatic video object segmentation is proposed. The initial object contour is obtained by modified intelligent scissors. Video decomposing is performed to avoid errors accumulating during object tracking. Snake-based bidirectional tracking is utilised to interpolate the VOPs of successive frames. Experimental results show the effectiveness of the method.     

动态背景下融合运动线索和颜色信息的视频目标分割算法

针对现有动态背景下目标分割算法存在的局限性,提出了一种融合运动线索和颜色信息的视频序列目标分割算法。首先,设计了一种新的运动轨迹分类方法,利用背景运动的低秩特性,结合累积确认的策略,可以获得准确的运动轨迹分类结果;然后,通过过分割算法获取视频序列的超像素集合,并计算超像素之间颜色信息的相似度;最后,以超像素为节点建立马尔可夫随机场模型,将运动轨迹分类信息以及超像素之间颜色信息统一建模在马尔可夫随机场的能量函数中,并通过能量函数最小化获得每个超像素的最优分类。在多组公开发布的视频序列中进行测试与对比,结果表明,本文方法可以准确分割出动态背景下的运动目标,并且较传统方法具有更高的分割准确率。     

音视频会议在MRF中的设计和实现

本文首先介绍了3GPP 定义的IMS 网络的体系架构,之后阐述了MRF 的组成结构及功能,然后着重讨论了音视频会议在MRF 中的具体设计方案,分别给出了MRFC 和MRFP 的功能模块的设计,最后具体描述了MRF 的音视频会议的信令流程的实现.     

Performance of the color set partitioning in hierarchical tree scheme (CSPIHT) in video coding

This paper describes the implementation of the recently introducedcolor set partitioning in hierarchical tree (CSPIHT)-based scheme for video coding. The intra- and interframe coding performance of a CSPIHT-based video coder (CVC) is compared against that of the H.263 at bit rates lower than 64 kbit/s. The CVC performs comparably or better than the H.263 at lower bit rates, whereas the H.263 performs better than the CVC at higher bit rates. We identify areas that hamper the performance of the CVC and propose an improved scheme that yields better performance in image and video coding in low bit-rate environments.     

基于Haar-like和肤色特征的驾驶员脸部检测融合算法

对驾驶员面部疲劳状态进行视觉监测的前提是脸部区域的准确、快速检测。采用改进的基于Haar-like特征的人脸检测算法检测出可能存在的初始人脸区域,然后适当扩大初始人脸区域范围,并在此基础上利用肤色特征和区域连通算法在YCbCr和rgb颜色空间上对人脸区域进行二次定位,最后根据定义的脸部区域重合度和人脸几何特征,实现脸部区域的融合检测。实验结果验证了该算法的准确性和可靠性。     

Automatic video segmentation and tracking for content-based applications

Advanced multimedia applications have to provide content-related functionalities such as search and retrieval of meaningful objects, detection and analysis of events, and understanding of scenes, which allow the user to access and manipulate the multimedia content with greater flexibility. This greatly depends on automatic techniques for extracting such objects from multimedia data. In this article we intend to provide a tutorial on the state-of-the-art in video segmentation and tracking technology with particular attention paid to the recent developments in attention-based object extraction. Performance results are included to highlight this emerging technology