Analogies based video editing

A well-produced video always creates a strong impression on the viewer. However, due to the limitations of the camera, the ambient conditions or the skills of the videographer, the quality of captured videos sometimes falls short of one's expectations. On the other hand, we have a vast amount of superbly captured videos available on the web and in digital libraries. In this paper, we propose the novel approach of video analogies that provides a powerful ability to improve the quality of a video by borrowing features from a higher quality video. We want to improve the given target video in order to obtain a higher quality output video. During the matching phase, we find the correspondence between the pair by using feature matching. Then for the target video, we utilize this correspondence to transfer some desired traits of the source video into the target video in order to obtain a new video. Thus, the new video will obtain the desired features from the source video while retaining the merits of the target video. The video analogies technique provides an intuitive mechanism for automatic editing of videos. We demonstrate the utility of the analogies method by considering three applications – colorizing videos, reducing video blurs, and video rhythm adjustment. We describe each application in detail and provide experimental results to establish the efficacy of the proposed approach.     

真实感视频雨景编辑

用计算机来模拟雨景是影视制作中经常面临的一个课题。目前主要采用的粒子系统方法一般都需要大量的粒子和复杂的物理模型来模拟真实的雨景。提出了一种新的基于图像的方法来实现真实感雨景的绘制。用最小值滤波和改进的Matting算法相结合的方法,从视频中分割出真实的雨幕,然后根据目标场景的光照条件改变雨幕的颜色,最后将其与目标场景融合在一起。实验证明,该方法十分有效。同时该方法是完全基于视频图像的,避免了昂贵的3D场景建模,适合于低成本的影片制作和影视素材重用领域。     

Interactive 3D video editing

We present a generic and versatile framework for interactive editing of 3D video footage. Our framework combines the advantages of conventional 2D video editing with the power of more advanced, depth-enhanced 3D video streams. Our editor takes 3D video as input and writes both 2D or 3D video formats as output. Its underlying core data structure is a novel 4D spatio-temporal representation which we call the video hypervolume. Conceptually, the processing loop comprises three fundamental operators: slicing, selection, and editing. The slicing operator allows users to visualize arbitrary hyperslices from the 4D data set. The selection operator labels subsets of the footage for spatio-temporal editing. This operator includes a 4D graph-cut based algorithm for object selection. The actual editing operators include cut & paste, affine transformations, and compositing with other media, such as images and 2D video. For high-quality rendering, we employ EWA splatting with view-dependent texturing and boundary matting. We demonstrate the applicability of our methods to post-production of 3D video.     

Disk scheduling at compile time

This paper describes a simple algorithm* for allocating program files on a disk with a moving head. The algorithm tries to place a file on consecutive pages (but will scatter them somewhat if necessary). It then rearranges these pages to minimize the rotational delay during a sequential scan of the file. This method combines the best features of consecutive and non-consecutive allocation: fast sequential access and fast allocation. It is used in the Solo operating system to reduce program loading time by a factor of 3.     

无线视频监控中心的多路视频流调度模型

提出了一种用于无线视频监控中心多路视频流调度的通用模型,以实现对多个场景的实时监控.该模型主要包括:远程监控设备的授权管理策略、视频流对象的工作模型、基于连接管理线程的多路视频流调度及抖动缓存自适应累积策略.目前,该模型已经应用于现场视频无线动态传输系统,取得了良好的应用效果.     

Quality adaptive end-to-end packet scheduling to avoid playout interruptions in Internet video streaming systems

In Internet multimedia streaming, the quality of the delivered media can be adapted to the Quality of Service provided by the underlying network, thanks to encoding algorithms. These allow a fine grained enhancement of a low quality base layer at streaming time. The main objective that should be satisfied in such systems is to avoid the starvation of the decoding process and consequent playout interruptions. In this work, we tackle the problem using a control theoretic approach. In particular, we design and implement the novel end-to-end Quality Adaptive Scheduler for properly distributing the network available bandwidth among base and enhancement layers. The developed solution can be adopted in many contexts given that it has been designed without assumptions on the delivered media nor on the protocol stack. Anyway, to test its effectiveness, we have casted it in a H.264/AVC SVC based video streaming architecture for unicast Internet applications. The performance of the scheduler has been experimentally evaluated in both a controlled testbed and several “wild” Internet scenarios, including also UMTS and satellite radio links. Results have clearly demonstrated that our Quality Adaptive Scheduler is able to significantly improve the performance of the video streaming system in all operative conditions.     

Storyboard-based accurate automatic summary video editing system

The recent popularity of smart mobile devices has led to a significant increase in the needs of multimedia services. Finding new more efficient methods for automatic classification and retrieval of a large number of multimedia files will significantly reduce manpower costs. However, most current video content analysis methods adopt low-level features to analyze video frame by frame, and need to improve high-level semantic analysis on a number of issues. Hence, this study presents a storyboard-based accurate automatic summary video editing system that uses storyboard information, such as character dialogue, narration, caption, background music and shot changes, to enable accurate video content retrieval and automatic render summary videos. The proposed system can be applied to the course video trailer and the commercial video trailer for quick preview video content or suitable viewing configuration for smart mobile devices. Consequently, the audience can quickly understand the whole video story and the video editors can substantially reduce the time taken to publish videos.     

Styles in document editing systems

A logical history of document editing mechanisms is presented. The design space for document style mechanisms is analyzed. Six primary design issues and the subsidiary issues they raise are discussed. Some major style issues that are seen as the subject of future research are identified.<>     

Modeling video sources for real-time scheduling

What is the impact of the autocorrelation of variable-bit-rate (VBR) sources on real-time scheduling algorithms? Our results show that the impact of long term, or interframe, autocorrelation is negligible, while the impact of short term, or intraframe, autocorrelation can be significant. Such results are essentially independent of the video coding scheme employed. To derive these results, video sequences are modeled as a collection of stationary subsequences called scenes. Within a scene, a statistical model is derived for both the sequence of frames and of slices. The model captures the distribution and the autocorrelation function of real-time video data. In previous work, the pseudoperiodicity of the slice level auto-correlation function made it difficult to develop a simple yet accurate model. We present a generalization of previous methods that can easily capture this pseudoperiodicity and is suited for modeling a greater variety of autocorrelation functions. By simply tuning a few parameters, the model reproduces the statistic behavior of sources with different types and levels of correlation on both the frame and the slice level.     

Searching and editing MPEG-compressed video in a distributed online environment

WebClip (on-line demo at http://www.ctr.columbia.edu/webclip) is a compressed video searching and editing system operating over the World Wide Web. WebClip uses a distributed client-server model including a server engine for content analysis/editing, and clients for interactive controls of video browsing/editing. It specializes several unique features, including compressed-domain video feature extraction and manipulation, multi-resolution video access, content based video browsing/retrieval, and a distributed network architecture.     

Disk placement for arbitrary-rate playback in an interactive video server

Multimedia data, especially continuous media including video and audio objects, represent a rich and natural stimulus for humans, but require large amount of storage capacity and real-time processing. In this paper, we describe how to organize video data efficiently on multiple disks in order to support arbitrary-rate playback requested by different users independently. Our approach is to segment and decluster video objects and to place the segments in multiple disks using a restricted round-robin scheme, called prime round-robin (PRR). Its placement scheme provides uniform load balance of disks for arbitrary retrieval rate as well as normal playback, since it eliminates hot spots. Moreover, it does not require any additional disk bandwidth to support VCR-like operations such as fast-forward and rewind. We have studied the various effects of placement and retrieval schemes in a storage server by simulation. The results show that PRR offers even disk accesses, and the failure in reading segment by deadline occurs only at the beginning of new operations. In addition, the number of users admitted is not decreased, regardless of arbitrary-rate playback requests.     

A time-code ALU for a microprocessor-based video editing system

An Arithmetic Logic Unit (ALU) was designed for use in a microprocessor-based video editing system. The hardware ALU developed implemented the time-code addition/subtraction required for video editing operating using the European Broadcasting Union standard format for time-code. The ALU was interfaced to an S-100 bus and tested with the North Star Z80A microprocessor development system.     

视频传输中的分组调度方法

数据报丢失控制是在因特网中传输视频时的基本问题,缓冲区管理是防止视频数据报丢失的传统网络技术.为了解决视频通信中数据报丢失的问题,提出了一种新的分组调度方法,并建立视频通信系统环境,进行了编程实验.对运行在路由器内,能够保证视频传过程中视频流内部和视频流之间数据报丢失要求的分组调度方法进行了仿真和分析.结果表明了该分组调度方法在控制数据报丢失和效率方面的优点.     

适用于无线视频传输的动态调度机制

对于无线网络传输实时高清视频时延高的问题,提出了一种增强型混合协调功能(EHCF)的动态调度机制。首先,利用截断和扩展信标间隔以适应视频数据的突发性;然后,根据站点数据缓存量大小来判断各站点传输需求,并由此划分在当前信标间隔中的优先级;其次,利用传输需求门限值和差值门限来评估高优先站点的传输需求权重;最后,根据评估结果为各站点分配信道资源。在基于H.264的视频传输仿真实验中,该机制以22 Mb/s的传输速度且网络站点个数相同时与传统的IEEE 802.11e中混合协调功能(HCF)相比,时延下降幅度最高为87.6%,最低为54.3%。仿真结果表明,EHCF机制在无线视频传输中能有效提高网络的实时性。     

Task scheduling in multiprocessing systems

The complex problem of assigning tasks to processing elements in order to optimize a performance measure has resulted in numerous heuristics aimed at approximating an optimal solution. This article addresses the task scheduling problem in many of its variations and surveys the major solutions. The scheduling techniques we discuss might be used by a compiler writer to optimize the code that comes out of a parallelizing compiler. The compiler would produce grains of sequential code, and the optimizer would schedule these grains such that the program runs in the shortest time     

实时系统中弹性调度策略

弹性调度面向负载可变的实时系统,通过动态调整任务属性以满足系统的灵活性要求,是一种高效的任务调度策略。针对弹性调度研究中的成果及问题,概述了弹性调度的研究背景,从任务模型、调度模型以及调度算法三个方面对弹性调度的国内外研究进展进行综述,探讨当前研究中存在的问题,并对弹性调度未来研究工作进行分析和展望。     

Videoshop: A new framework for spatio-temporal video editing in gradient domain

This paper proposes a new framework for video editing in gradient domain. The spatio-temporal gradient fields of target videos are modified and/or mixed to generate a new gradient field which is usually not integrable. We compare two methods to solve this “mixed gradient problem”, i.e., the variational method and loopy belief propagation. We propose a 3D video integration algorithm, which uses the variational method to find the potential function whose gradient field is closest to the mixed gradient field in the sense of least squares. The video is reconstructed by solving a 3D Poisson equation. The main contributions of our framework lie in three aspects: first, we derive a straightforward extension of current 2D gradient technique to 3D space, thus resulting in a novel video editing framework, which is very different from all current video editing software; secondly, we propose using a fast and accurate 3D discrete Poisson solver which uses diagonal multigrids to solve the 3D Poisson equation, which is up to twice as fast as a simple conventional multigrid algorithm; finally, we introduce a set of new applications, such as face replacement and painting, high dynamic range video compression and graphcut based video compositing. A set of gradient operators is also provided to the user for editing purposes. We evaluate our algorithm using a variety of examples for image/video or video/video pairs. The resulting video can be seamlessly reconstructed.     

Disk load balancing for video-on-demand systems

For a video-on-demand computer system, we propose a scheme which balances the load on the disks, thereby helping to solve a performance problem crucial to achieving maximal video throughput. Our load-balancing scheme consists of two components. The static component determines good assignments of videos to groups of striped disks. The dynamic component uses these assignments, and features a “DASD dancing” algorithm which performs real-time disk scheduling in an effective manner. Our scheme works synergistically with disk striping. We examine the performance of the proposed algorithm via simulation experiments.     

An incentive scheduling mechanism for peer-to-peer video streaming

P2P video streaming networks are found as a scalable solution and an alternative for traditional client–server based video streaming over the Internet. One of the significant issues affecting the success of any P2P streaming network is cooperation between peers. Practical observations have proved the prevalence of free riders in P2P networks that degrade their performance. To solve this problem, using incentive mechanisms, which encourage peers to contribute more in the network, is necessary. In this paper, we designed and proposed a distributed and scalable incentive mechanism for mesh based P2P video streaming networks. In the proposed approach the contribution of the peers is measured and maintained in a distributed fashion. Furthermore, we proposed an incentive sending side scheduler in which peers are served based on their contribution in the network. Our simulation evaluations show the efficiency of the proposed approach in improving the overall perceived video quality by the non-free rider nodes and consequently in the whole network.