Video tampering dataset development in temporal domain for video forgery authentication

Videos are tampered by the forgers to modify or remove their content for malicious purpose. Many video authentication algorithms are developed to detect this tampering. At present, very few standard and diversified tampered video dataset is publicly available for reliable verification and authentication of forensic algorithms. In this paper, we propose the development of total 210 videos for Temporal Domain Tampered Video Dataset (TDTVD) using Frame Deletion, Frame Duplication and Frame Insertion. Out of total 210 videos, 120 videos are developed based on Event/Object/Person (EOP) removal or modification and remaining 90 videos are created based on Smart Tampering (ST) or Multiple Tampering. 16 original videos from SULFA and 24 original videos from YouTube (VTD Dataset) are used to develop different tampered videos. EOP based videos include 40 videos for each tampering type of frame deletion, frame insertion and frame duplication. ST based tampered video contains multiple tampering in a single video. Multiple tampering is developed in three categories (1) 10-frames tampered (frame deletion, frame duplication or frame insertion) at 3-different locations (2) 20-frames tampered at 3- different locations and (3) 30-frames tampered at 3-different locations in the video. Proposed TDTVD dataset includes all temporal domain tampering and also includes multiple tampering videos. The resultant tampered videos have video length ranging from 6 s to 18 s with resolution 320X240 or 640X360 pixels. The database is comprised of static and dynamic videos with various activities, like traffic, sports, news, a ball rolling, airport, garden, highways, zoom in zoom out etc. This entire dataset is publicly accessible for researchers, and this will be especially valuable to test their algorithms on this vast dataset. The detailed ground truth information like tampering type, frames tampered, location of tampering is also given for each developed tampered video to support verifying tampering detection algorithms. The dataset is compared with state of the art and validated with two video tampering detection methods.

     

Catching a viral video

The sharing and re-sharing of videos on social sites, blogs e-mail, and other means has given rise to the phenomenon of viral videos—videos that become popular through internet sharing. In this paper we seek to better understand viral videos on YouTube by analyzing sharing and its relationship to video popularity using millions of YouTube videos. The socialness of a video is quantified by classifying the referrer sources for video views as social (e.g. an emailed link, Facebook referral) or non-social (e.g. a link from related videos). We find that viewership patterns of highly social videos are very different from less social videos. For example, the highly social videos rise to, and fall from, their peak popularity more quickly than less social videos. We also find that not all highly social videos become popular, and not all popular videos are highly social. By using our insights on viral videos we are able develop a method for ranking blogs and websites on their ability to spread viral videos.     

Boosting web video categorization with contextual information from social web

Web video categorization is a fundamental task for web video search. In this paper, we explore web video categorization from a new perspective, by integrating the model-based and data-driven approaches to boost the performance. The boosting comes from two aspects: one is the performance improvement for text classifiers through query expansion from related videos and user videos. The model-based classifiers are built based on the text features extracted from title and tags. Related videos and user videos act as external resources for compensating the shortcoming of the limited and noisy text features. Query expansion is adopted to reinforce the classification performance of text features through related videos and user videos. The other improvement is derived from the integration of model-based classification and data-driven majority voting from related videos and user videos. From the data-driven viewpoint, related videos and user videos are treated as sources for majority voting from the perspective of video relevance and user interest, respectively. Semantic meaning from text, video relevance from related videos, and user interest induced from user videos, are combined to robustly determine the video category. Their combination from semantics, relevance and interest further improves the performance of web video categorization. Experiments on YouTube videos demonstrate the significant improvement of the proposed approach compared to the traditional text based classifiers.     

Full-frame video stabilization with motion inpainting

Video stabilization is an important video enhancement technology which aims at removing annoying shaky motion from videos. We propose a practical and robust approach of video stabilization that produces full-frame stabilized videos with good visual quality. While most previous methods end up with producing smaller size stabilized videos, our completion method can produce full-frame videos by naturally filling in missing image parts by locally aligning image data of neighboring frames. To achieve this, motion inpainting is proposed to enforce spatial and temporal consistency of the completion in both static and dynamic image areas. In addition, image quality in the stabilized video is enhanced with a new practical deblurring algorithm. Instead of estimating point spread functions, our method transfers and interpolates sharper image pixels of neighboring frames to increase the sharpness of the frame. The proposed video completion and deblurring methods enabled us to develop a complete video stabilizer which can naturally keep the original image quality in the stabilized videos. The effectiveness of our method is confirmed by extensive experiments over a wide variety of videos.     

基于Web的EAST实时视频点播系统

基于Web的EAST（全超导托卡马克）实时视频点播系统是在原视频点播系统的基础上进行优化和改进实现的,具有实时点播EAST实验中的等离子体放电视频、搜索视频、下载视频以及逐帧分析视频等功能.该系统主要由视频合成应用程序和视频点播网站两个部分组成,视频合成程序采用socket TCP通讯的方式接收炮号信息,通过共享目录转移视频帧文件,最后利用ffmpeg将视频帧文件合成flv视频.视频点播采用B/S（浏览器/服务器）结构框架实现,通过JSP从数据库读取炮号信息实现点播功能,结合XML和JavaScript实现逐帧分析.目前本系统已经在EAST实验中投入使用,为EAST实验人员提供了极大的便利.     

Malicious inter-frame video tampering detection in MPEG videos using time and spatial domain analysis of quantization effects

In this paper, a new algorithm is proposed for forgery detection in MPEG videos using spatial and time domain analysis of quantization effect on DCT coefficients of I and residual errors of P frames. The proposed algorithm consists of three modules, including double compression detection, malicious tampering detection and decision fusion. Double compression detection module employs spatial domain analysis using first significant digit distribution of DCT coefficients in I frames to detect single and double compressed videos using an SVM classifier. Double compression does not necessarily imply the existence of malignant tampering in the video. Therefore, malicious tampering detection module utilizes time domain analysis of quantization effect on residual errors of P frames to identify malicious inter-frame forgery comprising frame insertion or deletion. Finally, decision fusion module is used to classify input videos into three categories, including single compressed videos, double compressed videos without malicious tampering and double compressed videos with malicious tampering. The experimental results and the comparison of the results of the proposed method with those of other methods show the efficiency of the proposed algorithm.     

Moving-Object Detection, Association, and Selection in Home Videos

Due to the prevalence of digital video camcorders, home videos have become an important part of life-logs of personal experiences. To enable efficient video parsing, a critical step is to automatically extract objects, events and scene characteristics present in videos. This paper addresses the problem of extracting objects from home videos. Automatic detection of objects is a classical yet difficult vision problem, particularly for videos with complex scenes and unrestricted domains. Compared with edited and surveillant videos, home videos captured in uncontrolled environment are usually coupled with several notable features such as shaking artifacts, irregular motions, and arbitrary settings. These characteristics have actually prohibited the effective parsing of semantic video content using conventional vision analysis. In this paper, we propose a new approach to automatically locate multiple objects in home videos, by taking into account of how and when to initialize objects. Previous approaches mostly consider the problem of how but not when due to the efficiency or real-time requirements. In home-video indexing, online processing is optional. By considering when, some difficult problems can be alleviated, and most importantly, enlightens the possibility of parsing semantic video objects. In our proposed approach, the how part is formulated as an object detection and association problem, while the when part is a saliency measurement to determine the best few locations to start multiple object initialization     

Ranking,clustering and fusing the normalized LBP temporal facial features for face recognition in video sequences

This paper proposes a novel approach for recognizing faces in videos with high recognition rate. Initially, the feature vector based on Normalized Local Binary Patterns is obtained for the face region. A set of training and testing videos are used in this face recognition procedure. Each frame in the query video is matched with the signature of the faces in the database using Euclidean distance and a rank list is formed. Each ranked list is clustered and its reliability is analyzed for re-ranking. Multiple re-ranked lists of the query video is fused together to form a video signature. This video signature embeds diverse intra-personal variations such as poses, expressions and facilitates in matching two videos with large variations. For matching two videos, their composite ranked lists are compared using a Kendall Tau distance measure. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their novel approach when compared with the existing techniques.

     

A Web video retrieval method using hierarchical structure of Web video groups

In this paper, we propose a Web video retrieval method that uses hierarchical structure of Web video groups. Existing retrieval systems require users to input suitable queries that identify the desired contents in order to accurately retrieve Web videos; however, the proposed method enables retrieval of the desired Web videos even if users cannot input the suitable queries. Specifically, we first select representative Web videos from a target video dataset by using link relationships between Web videos obtained via metadata “related videos” and heterogeneous video features. Furthermore, by using the representative Web videos, we construct a network whose nodes and edges respectively correspond to Web videos and links between these Web videos. Then Web video groups, i.e., Web video sets with similar topics are hierarchically extracted based on strongly connected components, edge betweenness and modularity. By exhibiting the obtained hierarchical structure of Web video groups, users can easily grasp the overview of many Web videos. Consequently, even if users cannot write suitable queries that identify the desired contents, it becomes feasible to accurately retrieve the desired Web videos by selecting Web video groups according to the hierarchical structure. Experimental results on actual Web videos verify the effectiveness of our method.     

Relational large scale multi-label classification method for video categorization

The problem of automated video categorization in large datasets is considered in the paper. A new Iterative Multi-label Propagation (IMP) algorithm for relational learning in multi-label data is proposed. Based on the information of the already categorized videos and their relations to other videos, the system assigns suitable categories—multiple labels to the unknown videos. The MapReduce approach to the IMP algorithm described in the paper enables processing of large datasets in parallel computing. The experiments carried out on 5-million videos dataset revealed the good efficiency of the multi-label classification for videos categorization. They have additionally shown that classification of all unknown videos required only several parallel iterations.     

Watching a Movie Alone yet Together: Understanding Reasons for Watching Danmaku Videos

By overlaying timeline-synchronized user comments on videos, Danmaku commenting creates a unique co-viewing experience of online videos. This study aims to understand the reasons for watching or not watching Danmaku videos. From a review of the literature and a pilot study, an initial pool of motivations and hindrances to Danmaku video viewing was gathered. Then, a survey involving 248 participants to identify the underlying factor structures of motivations and hindrances was conducted. Their influences on users’ attitude and behaviors with Danmaku videos were also examined. The results showed that people viewed Danmaku videos to obtain information, entertainment, and social connectedness. Introverted young men with high openness to new experience are more likely to view Danmaku videos. Infrequent viewers refused to watch Danmaku videos mainly because of the visual clutter that resulted from Danmaku comments.     

基于位置敏感哈希的网络视频重复检测

针对当前网络上存在着大量的重复或近似重复的视频问题,提出了一种基于镜头层比较和位置敏感哈希的快速准确的网络视频重复检测方法。通过视频间匹配的镜头数占查询视频总镜头数的比例来判断视频的相似性。除此之外,还利用著名的近似最近邻查找技术——LSH在镜头层来快速查找相似镜头,从而提高检测速度。通过将镜头作为检索单元,把数据库中所有视频的镜头放到一起构建一个新的数据集,将种子(查询)视频的每一个镜头作为一个查询请求,应用基于LSH的近似近邻检索方法,检索出与查询镜头相匹配的所有镜头,最后融合这些返回的结果,得到查询视频的重复或者近似重复的视频集。通过在包含12 790个视频的CC_WEB_VIDEO数据集上的实验结果表明,该方法取得了相比已有方法更好的检测性能。     

Video archaeology: understanding video manipulation history

Facing the explosive growth of near-duplicate videos, video archaeology is quite desired to investigate the history of the manipulations on these videos. With the determination of derived videos according to the manipulations, a video migration map can be constructed with the pair-wise relationships in a set of near-duplicate videos. In this paper, we propose an improved video archaeology (I-VA) system by extending our previous work (Shen et al. 2010). The extensions include more comprehensive video manipulation detectors and improved techniques for these detectors. Specially, the detectors are used for two categories of manipulations, i.e., semantic-based manipulations and non-semantic-based manipulations. Moreover, the improved detecting algorithms are more stable. The key of I-VA is the construction of a video migration map, which represents the history of how near-duplicate videos have been manipulated. There are various applications based on the proposed I-VA system, such as better understanding of the meaning and context conveyed by the manipulated videos, improving current video search engines by better presentation based on the migration map, and better indexing scheme based on the annotation propagation. The system is tested on a collection of 12,790 videos and 3,481 duplicates. The experimental results show that I-VA can discover the manipulation relation among the near-duplicate videos effectively.     

一种融合颜色和运动特征的镜头探测方法

Video Partitioning serves as the preliminary step to structure the content of videos and is the basis of Content-based Video Retrieval. This paper presents a shot detection scheme combining color and motion features of videos. It first extracts the color and motion information from the compressed domain videos, then detects cut transitions and gradual transitions separately incorporating color features with motion ones. The experiment demonstrats real time performance with high accuracy.     

The crowd as a cameraman: on-stage display of crowdsourced mobile video at large-scale events

Multimedia Tools and Applications - Recording videos with smartphones at large-scale events such as concerts and festivals is very common nowadays. These videos register the atmosphere of the event...     

Extracting and analyzing time-series HCI data from screen-captured task videos

Recent years have witnessed the increasing emphasis on human aspects in software engineering research and practices. Our survey of existing studies on human aspects in software engineering shows that screen-captured videos have been widely used to record developers’ behavior and study software engineering practices. The screen-captured videos provide direct information about which software tools the developers interact with and which content they access or generate during the task. Such Human-Computer Interaction (HCI) data can help researchers and practitioners understand and improve software engineering practices from human perspective. However, extracting time-series HCI data from screen-captured task videos requires manual transcribing and coding of videos, which is tedious and error-prone. In this paper we report a formative study to understand the challenges in manually transcribing screen-captured videos into time-series HCI data. We then present a computer-vision based video scraping technique to automatically extract time-series HCI data from screen-captured videos. We also present a case study of our scvRipper tool that implements the video scraping technique using 29-hours of task videos of 20 developers in two development tasks. The case study not only evaluates the runtime performance and robustness of the tool, but also performs a detailed quantitative analysis of the tool’s ability to extract time-series HCI data from screen-captured task videos. We also study the developer’s micro-level behavior patterns in software development from the quantitative analysis.     

面向视频的细粒度多模态实体链接

随着互联网和大数据的飞速发展,数据规模越来越大,种类也越来越多.视频作为其中重要的一种信息方式,随着近期短视频的发展,占比越来越大.如何对这些大规模视频进行理解分析,成为学界关注的热点.实体链接作为一种背景知识补全方式,可以提供丰富的外部知识.视频上的实体链接可以有效地帮助理解视频内容,从而实现对视频内容的分类、检索、推荐等.但是现有的视频链接数据集和方法的粒度过粗,因此提出面向视频的细粒度实体链接,并立足于直播场景,构建了细粒度视频实体链接数据集.此外,依据细粒度视频链接任务的难点,提出利用大模型抽取视频中的实体及其属性,并利用对比学习得到视频和对应实体的更好表示.实验结果表明,该方法能够有效地处理视频上的细粒度实体链接任务.     

基于帧间差异的人脸篡改视频检测方法

近几年,随着计算机硬件设备的不断更新换代和深度学习技术的不断发展,新出现的多媒体篡改工具可以让人们更容易地对视频中的人脸进行篡改。使用这些新工具制作出的人脸篡改视频几乎无法被肉眼所察觉,因此我们急需有效的手段来对这些人脸篡改视频进行检测。目前流行的视频人脸篡改技术主要包括以自编码器为基础的Deepfake技术和以计算机图形学为基础的Face2face技术。我们注意到人脸篡改视频里人脸区域的帧间差异要明显大于未被篡改的视频中人脸区域的帧间差异,因此视频相邻帧中人脸图像的差异可以作为篡改检测的重要线索。在本文中,我们提出一种新的基于帧间差异的人脸篡改视频检测框架。我们首先使用一种基于传统手工设计特征的检测方法,即基于局部二值模式（Local binary pattern,LBP）/方向梯度直方图（Histogram of oriented gradient,HOG）特征的检测方法来验证该框架的有效性。然后,我们结合一种基于深度学习的检测方法,即基于孪生网络的检测方法进一步增强人脸图像特征表示来提升检测效果。在FaceForensics++数据集上,基于LBP/HOG特征的检测方法有较高的检测准确率,而基于孪生网络的方法可以达到更高的检测准确率,且该方法有较强的鲁棒性;在这里,鲁棒性指一种检测方法可以在三种不同情况下达到较高的检测准确率,这三种情况分别是：对视频相邻帧中人脸图像差异用两种不同方式进行表示、提取三种不同间隔的帧对来计算帧间差异以及训练集与测试集压缩率不同。     

针对近重复视频的FD-means聚类清洗算法

近几年,随着视频数据规模的不断增加,近重复视频数据不断涌现,视频的数据质量问题越来越突出。通过近重复视频清洗方法,有助于提高视频集的数据质量。然而,目前针对近重复视频清洗问题的研究较少,主要集中于近重复视频检索等方面的研究。现有研究方法尽管可以有效识别近重复视频,但较难在保证数据完整性的前提下,自动清洗近重复视频数据,以便改善视频数据质量。为解决上述问题,提出一种融合VGG-16深度网络与FD-means（feature distance-means）聚类的近重复视频清洗方法。该方法借助MOG2模型和中值滤波算法对视频进行背景分割和前景降噪;利用VGG-16深度网络模型提取视频的深度空间特征;构建一种新的FD-means聚类算法模型,通过迭代产生的近重复视频簇,更新簇类中心点,并最终删除簇中中心点之外的近重复视频数据。实验结果表明,该方法能够有效解决近重复视频数据清洗问题,改善视频的数据质量。     

基于弹幕情感分析和主题模型的视频推荐算法

针对互联网上大量自制视频缺少用户评分、推荐准确率不高的问题,提出一种融合弹幕情感分析和主题模型的视频推荐算法（VRDSA）。首先,对视频的弹幕评论进行情感分析,得到视频的情感向量,之后基于情感向量计算视频之间的情感相似度;同时,基于视频的标签建立主题模型来得到视频标签的主题分布,并使用主题分布计算视频之间的主题相似度;接着,对视频的情感相似度和主题相似度进行融合得到视频间的综合相似度;然后,结合视频间的综合相似度和用户的历史记录得到用户对视频的偏好度;同时通过视频的点赞量、弹幕量、收藏数等用户互动指标对视频的大众认可度进行量化,并结合用户历史记录计算出视频的综合认可度;最后,基于用户对视频的偏好度和视频的综合认可度预测用户对视频的认可度,并生成个性化推荐列表来完成视频的推荐。实验结果表明,与融合协同过滤和主题模型的弹幕视频推荐算法（DRCFT）以及嵌入LDA主题模型的协同过滤算法（ULR-itemCF）相比,所提算法推荐的准确率平均提高了17.1%,召回率平均提高了22.9%,F值平均提高了22.2%。所提算法对弹幕进行情感分析,并融合主题模型,以此来完成对视频的推荐,并且充分挖掘了弹幕数据的情感性,使得推荐结果更加准确。