Intelligent Mobile Video Surveillance System with Multilevel Distillation

This paper proposes a mobile video surveillance system consisting of intelligent video analysis and mobile communication networking. This multilevel distillation approach helps mobile users monitor tremendous surveillance videos on demand through video streaming over mobile communication networks. The intelligent video analysis includes moving object detection/tracking and key frame selection which can browse useful video clips. The communication networking services, comprising video transcoding, multimedia messaging, and mobile video streaming, transmit surveillance information into mobile appliances. Moving object detection is achieved by background subtraction and particle filter tracking. Key frame selection, which aims to deliver an alarm to a mobile client using multimedia messaging service accompanied with an extracted clear frame, is reached by devising a weighted importance criterion considering object clarity and face appearance. Besides, a spatial-domain cascaded transcoder is developed to convert the filtered image sequence of detected objects into the mobile video streaming format. Experimental results show that the system can successfully detect all events of moving objects for a complex surveillance scene, choose very appropriate key frames for users, and transcode the images with a high power signal-to-noise ratio (PSNR).     

Reranking Methods for Visual Search

Most semantic video search methods use text-keyword queries or example video clips and images. But such methods have limitations. To address the problems of example-based video search approaches and avoid the use of specialized models, we conduct semantic video searches using a reranking method that automatically reorders the initial text search results based on visual cues and associated context. We developed two general reranking methods that explore the recurrent visual patterns in many contexts, such as the returned images or video shots from initial text queries, and video stories from multiple channels.     

面向足球视频摘要的叙事性拼图算法

利用叙事性拼图优化足球视频摘要的显示,根据精彩事件分析和不同镜头类型的叙事相关性最大化策略抽取候选关键帧,提取不同感兴趣对象为中心定义和重定向感兴趣区域,根据叙事结构中的时-空逻辑构建拼图排列,比随机拼图或故事板能保持更多精彩事件的叙事内容。实验结果显示,叙事性拼图摘要不仅清晰地对精彩事件中的球员动作和人物表情实现了叙事目的,还能在同等显示区域内比故事板描绘更多的叙事内容,也比直接的视频片段快放节约流量,以及节约浏览时间。      

User-controlled, multimedia-enhanced communication using prior knowledge and experience

This article focuses on enriching the communication experience between human beings interacting through different kinds of devices, including mobile phones and PDAs. The key point is how to effectively exploit multimedia data to enhance the richness of communication without overloading the communication channel. To this end, the system tries to minimize the quantity of information transmitted over the networks and to maximize the usage of locally stored information. The proposed system sends several kinds of data, including continuous video, video clips, still images, and avatars, over the communication channel. The specific data that are sent are based on the parameters set by the transmitting and receiving terminals. Bandwidth-consuming data, such as continuous video, are transmitted only occasionally at the receiving terminal. The system has been partially implemented and incorporated in avatar-enabled cellular phones. The user may then easily create different avatars combining components characterizing faces, hair, and so on, and associate them with different callers. When a call comes through, the avatar corresponding to the caller is retrieved and displayed on the screen and the corresponding information is retrieved from the personal database.     

A Scheduling Policy of Short Video Clips for VoD Systems Based on Optical Jukeboxes

1　IntroductionVideo on Demand (VoD)isapromisinginter activemultimediaapplicationthatiscapableofhand ingbothlongvideoprograms,suchasmovies,andshortvideoclips,e.g .news.MuchofpreviousstudiesofVoDsystemsweremainlyfocusedonlongprogramapplicationssuchasmovie on de mand[1～2 ] .Butshortvideoclipscanbemorepreva lentinapplicationslikedigitallibrary ,news on de mandandelectroniccommercewhereavideocliponlylasts 2 - 3minutes[3～4] .StudiesofVoDsys temsforshortclipsareofsignificanceforthatkindofapplic…     

Location based abstraction of user generated mobile videos

Demand for efficient ways to represent vast amount of video data has grown rapidly in recent years. The advances in positioning services have led to new possibilities in combining location information to video content. In this paper we present an automatic video editing system for geotagged mobile videos. In our solution the system creates automatically a video summary from a set of unedited video clips. Location information and timestamps are used to group video clips with the same context properties. The groups are used to create a video summary where subshots from same context group are represented as scenes. The novelty in our solution lies in combining geotags with low level content analysis tools in video abstraction. We have evaluated the created video summaries with a group of users and the system usability for service creation by building a semi-automatic web-based video editing service. The evaluations prove that our concept is useful as it improves coherence and enjoyability of the automatic video summaries.     

Self-supervised video representation learning by maximizing mutual information

We address the problem of learning representations from the videos without manual annotation. Different video clips sampled from the same video usually have a similar background and consistent motion. A novel self-supervised task is designed to learn such temporal coherence, which is measured by the mutual information in our work. First, we maximize the mutual information between features extracted from the clips which are sampled from the same video. This encourages the network to learn the shared content by these clips. As a result, the network may focus on the background and ignore the motion in videos due to that different clips from the same video normally have the same background. Second, to address this issue, we simultaneously maximize the mutual information between the feature of the video clip and the local regions where salient motion exists. Our approach, which is referred to as Deep Video Infomax (DVIM), strikes a balance between the background and the motion when learning the temporal coherence. We conduct extensive experiments to test the performance of the proposed DVIM on various tasks. Experimental results of fine-tuning for the high-level action recognition problems validate the effectiveness of the learned representations. Additional experiments for the task of action similarity labeling also demonstrate the generalization of the learned representations of the DVIM.     

一种自适应的关键帧选取算法

文章针对传统的视频编码方法中,当预测图像中绝大多数图像块在参考帧中都找不到相似的块时,其预测效果急剧降低的问题,在对具有预测关系的视频图像集进行分析计算的基础上,设计了一种动态选择关键帧的算法.实验结果表明,该方法能有效地提高视频图像的压缩率.     

Refocusing multimedia research on short clips

The multimedia authoring research agenda today, is searching for the proverbial killer application. We believe that multimedia's killer app might already be at hand-and it's focused around audio and video clips. But researchers aren't addressing critical open research issues because of the current focus on commercially produced, feature-length videos as an experimental corpus. The article focuses on the notion of multimedia clips, and most especially their retrieval, as a key enabler for the wide adoption of multimedia.     

Automatic foveation for video compression using a neurobiological model of visual attention

We evaluate the applicability of a biologically-motivated algorithm to select visually-salient regions of interest in video streams for multiply-foveated video compression. Regions are selected based on a nonlinear integration of low-level visual cues, mimicking processing in primate occipital, and posterior parietal cortex. A dynamic foveation filter then blurs every frame, increasingly with distance from salient locations. Sixty-three variants of the algorithm (varying number and shape of virtual foveas, maximum blur, and saliency competition) are evaluated against an outdoor video scene, using MPEG-1 and constant-quality MPEG-4 (DivX) encoding. Additional compression radios of 1.1 to 8.5 are achieved by foveation. Two variants of the algorithm are validated against eye fixations recorded from four to six human observers on a heterogeneous collection of 50 video clips (over 45 000 frames in total). Significantly higher overlap than expected by chance is found between human and algorithmic foveations. With both variants, foveated clips are, on average, approximately half the size of unfoveated clips, for both MPEG-1 and MPEG-4. These results suggest a general-purpose usefulness of the algorithm in improving compression ratios of unconstrained video.     

On-chip traffic modeling and synthesis for MPEG-2 video applications

The objective of this paper is to introduce self-similarity as a fundamental property exhibited by the bursty traffic between on-chip modules in typical MPEG-2 video applications. Statistical tests performed on relevant traces extracted from common video clips establish unequivocally the existence of self-similarity in video traffic. Using a generic tile-based communication architecture, we discuss the implications of our findings on on-chip buffer space allocation and present quantitative evaluations for typical video streams. We also describe a technique for synthetically generating traces having statistical properties similar to those obtained from real video clips. Our proposed technique speeds up buffer simulations, allows media system designers to explore architectures rapidly and use large media data benchmarks more efficiently. We believe that our findings open new directions of research with deep implications on some fundamental issues in on-chip networks design for multimedia applications.     

Do video coding impairments disturb the visual attention deployment?

The visual attention deployment in a visual scene is contingent upon a number of factors. The relationship between the observer's attention and the visual quality of the scene is investigated in this paper: can a video artifact disturb the observer's attention? To answer this question, two experiments have been conducted. First, eye-movements of human observers were recorded, while they watched ten video clips of natural scenes under a free-viewing task. These clips were more or less impaired by a video encoding scheme (H.264/AVC). The second experiment relies on the subjective rating of the quality of the video clips. A quality score was then assigned to each clip, indicating the extent to which the impairments were visible. The standardized method double stimulus impairment scale (DSIS) was used, meaning that each observer viewed the original clip followed by its impaired version. Both experimental results have conjointly been analyzed. Our results suggest that video artifacts have no influence on the deployment of visual attention, even though these artifacts have been judged by observers as at least annoying.     

Key-frame selection for video summarization: an approach of multidimensional time series analysis

This paper presents a novel method of key-frame selection for video summarization based on multidimensional time series analysis. In the proposed scheme, the given video is first segmented into a set of sequential clips containing a number of similar frames. Then the key frames are selected by a clustering procedure as the frames closest to the cluster centres in each resulting video clip. The proposed algorithm is implemented experimentally on a wide range of testing data, and compared with state-of-the-art approaches in the literature, which demonstrates excellent performance and outperforms existing methods on frame selection in terms of fidelity-based metric and subjective perception.     

MMS(多媒体短消息业务)及其实现

MMS是3GPP和WAP论坛制订的移动数据业务。与SMS相比，它除了可以发送简单文本消息外还可以发送图像、音频、视频、动画等多媒体信息内容。文中对MMS的体系结构进行了介绍，对MMS系统两种实现方式：基于WAP的实现和基于IP的实现进行了探讨，最后还对MMS业务的部署进行了讨论。     

Real-time video-shot detection for scene surveillance applications

A surveillance system with automatic video-shot detection and indexing capabilities is presented. The proposed system aims at detecting the presence of abandoned objects in a guarded environment and at automatically performing online semantic video segmentation in order to facilitate the human operator's task of retrieving the cause of an alarm. The former task is performed by operating image segmentation based on temporal rank-order filtering, followed by classification in order to reduce false alarms. The latter task is performed by operating temporal video segmentation when an alarm is detected. In the clips of interest, the key frame is the one depicting a person leaving a dangerous object, and is determined on the basis of a feature indicating the movement around the dangerous region. Experimental results are reported in terms of static region detection, classification, clip and key-frame detection errors versus different levels of complexity of the guarded environment, in order to establish the performance that can be expected from the system in different situations.     

Toward a direct measure of video quality perception using EEG

An approach to the direct measurement of perception of video quality change using electroencephalography (EEG) is presented. Subjects viewed 8-s video clips while their brain activity was registered using EEG. The video signal was either uncompressed at full length or changed from uncompressed to a lower quality level at a random time point. The distortions were introduced by a hybrid video codec. Subjects had to indicate whether they had perceived a quality change. In response to a quality change, a positive voltage change in EEG (the so-called P3 component) was observed at latency of about 400-600 ms for all subjects. The voltage change positively correlated with the magnitude of the video quality change, substantiating the P3 component as a graded neural index of the perception of video quality change within the presented paradigm. By applying machine learning techniques, we could classify on a single-trial basis whether a subject perceived a quality change. Interestingly, some video clips wherein changes were missed (i.e., not reported) by the subject were classified as quality changes, suggesting that the brain detected a change, although the subject did not press a button. In conclusion, abrupt changes of video quality give rise to specific components in the EEG that can be detected on a single-trial basis. Potentially, a neurotechnological approach to video assessment could lead to a more objective quantification of quality change detection, overcoming the limitations of subjective approaches (such as subjective bias and the requirement of an overt response). Furthermore, it allows for real-time applications wherein the brain response to a video clip is monitored while it is being viewed.     

基于公共点提取的多视图像校正

自由视点立体视频既可以给观看者强烈的立体感,又可以为用户同时呈现不同角度的视点画面。但是垂直视差的存在,会影响合成立体图像的质量,出现图像的模糊,严重的垂直视差会使图像出现“跳跃”现象,甚至不能观看。因此,多视点图像的校正是提高3DTV观看质量的重要步骤之一。现阶段关于双视点图像校正技术的研究已较多,但多视点图像校正研究却需深入。其中,基于多相机标定参数的校正算法需要提供预先的标定数据,应用不便。本文提出了一种在无多摄像机标定参数情况下,消除垂直视差的校正方法。首先,使用SIFT算法提取图像特征点。然后,利用多视点图像公共点的匹配关系,选取唯一基准视点图像平面计算各视点校正矩阵。实验结果表明,该方法有效降低了垂直视差,限制了校正误差的传播,从而保持了校正后图像质量的一致性。      

The holy grail of content-based media analysis

Tools and systems for content-based access to multimedia and-image., video, audio, graphics, text, and any number of combinations-has increased in the last decade. We've seen a common theme of developing automatic analysis techniques for deriving metadata (data describing information in the content at both syntactic and semantic levels). Such metadata facilitates developing innovative tools and systems for multimedia information retrieval, summarization, delivery, and manipulation. Many interesting demonstrations of potential applications and services have emerged-finding images visually similar to a chosen picture (or sketch); summarizing videos with thumbnails of keyframes; finding video clips of a specific event, story, or person; and producing a two-minute skim of an hour-long program. In order to evaluate content-based research methodologies, the article considers intended users and whether alternative solutions exist and areas of research     

视频探测器探测算法的研究

对视频图像探测算法进行了研究.运用基本视频帧分析原理并采用Matlab语言设计了算法程序,并对算法进行了仿真验证和分析.通过对视频移动的探测以及火焰的探测经过一系列视频片断检测,该算法具有明显的探测效果.     

基于数字签名的高效视频拷贝检测算法研究

为了解决视频特征提取和大型数据库的快速搜索问题,提出了一种新的视频拷贝检测算法。该算法基于视频剪辑的时空特征计算视频签名,通过视频签名的距离来衡量视频之间的相似性。为视频签名数据库建立聚类索引表,提高搜索速度。在大型数据库中查询测试的实验结果表明该算法对视频拷贝检测有较高查全率和正确率。该算法可以作为一种有效的视频跟踪和检测技术应用于数字广播电视的监控领域。