Zodiac: A history-based interactive video authoring system

Easy-to-use audio/video authoring tools play a crucial role in moving multimedia software from research curiosity to mainstream applications. However, research in multimedia authoring systems has rarely been documented in the literature. This paper describes the design and implementation of an interactive video authoring system called Zodiac, which employs an innovative edit history abstraction to support several unique editing features not found in existing commercial and research video editing systems. Zodiac provides users a conceptually clean and semantically powerful branching history model of edit operations to organize the authoring process, and to navigate among versions of authored documents. In addition, by analyzing the edit history, Zodiac is able to reliably detect a composed video stream's shot and scene boundaries, which facilitates interactive video browsing. Zodiac also features a video object annotation capability that allows users to associate annotations to moving objects in a video sequence. The annotations themselves could be text, image, audio, or video. Zodiac is built on top of MMFS, a file system specifically designed for interactive multimedia development environments, and implements an internal buffer manager that supports transparent lossless compression/decompression. Shot/scene detection, video object annotation, and buffer management all exploit the edit history information for performance optimization.     

一个面向对象的交互式创作工具OOIAT的设计与实现

在开放交互式创伤工具过程中,常常会遇到这样的难题：如何表示和建立用户模型及操作的对象,并向用户提供可直接操作的交互方式;如何有效地表示和管理对象之间的关系。,为此,我们设计并实现了一个创伤工具OOIAT,它提供了一个可扩充的,用于表示用户模型并集成了事件模型的对象集,使用户只需定义对象,不必了解对象如何传递
动作,如何实现其功能,就能够处理在这些对象上的交互式直接创伤。     

Towards an easy to use authoring tool for interactive non-linear video

With growing bandwidths in the Internet and seemingly unlimited storage capacities on web servers, media became more and more important in the daily use of the World Wide Web. While about ten years ago only text and images with small file sizes (and as a result small resolutions) could be used, it is possible to watch high quality multimedia presentations nowadays. But those rarely exist because of tedious to learn authoring tools. A specialization to one main medium, in our case video, allows creating efficient authoring tools using well known paradigms. This work introduces an authoring tool called SIVA Producer. An iterative process for improving the usability of the authoring tool is described. Furthermore, a distinction of the terms “interactive video”, “annotated video”, “non-linear video” and “hypervideo” is given.     

DanVideo: an MPEG-7 authoring and retrieval system for dance videos

A well-annotated dance media is an essential part of a nation’s identity, transcending cultural and language barriers. Many dance video archives suffer from problems concerning authoring and access, because of the complex spatio-temporal relationships that exist between the dancers in terms of movements of their body parts and the emotions expressed by them in a dance. This paper presents a system named DanVideo for semi-automatic authoring and access to dance archives. DanVideo provides methods of annotation and authoring and retrieval tools for choreographers, dancers, and students. We demonstrate how dance media can be semantically annotated and how this information can be used for the retrieval of the dance video semantics. In particular, DanVideo offers an MPEG-7 based semi-automatic authoring tool that takes dance video annotations generated by dance experts and produces MPEG-7 metadata. DanVideo also has a search engine that takes users’ queries and retrieves dance semantics from metadata arranged using tree-embedding technique and based on spatial, temporal and spatio-temporal features of dancers. The search engine also leverages a domain-specific ontology to process knowledge-based queries. We have assessed the dance-video queries and semantic annotations in terms of precision, recall, and fidelity.     

AI meets authoring: User models for untelligent multimedia

Authoring is a complex, knowledge-intensive activity which until recently has been performed exclusively by humans. New computer-based techniques have added horsepower rather than intelligence to traditional approaches, and have not addressed their principal limitations, chief of which is the inability to tailor presentations to individual users at run-time.We believe a model of the user is needed to support this kind of run-time determination of formand content. We describe our approach to the acquisition, representation and exploitation of user models: themost plausible user model is the result of an abductiverecognition process and it incorporates assumptions about the user which are then used to constrain thedesign by abduction of the best presentation. Both recognition and design processes are performed at run-time. We describe a prototypical implementation designed to demonstrate these ideas in the domain of video authoring.Our approach to authoring is intended to apply across multiple media; we have demonstrated these ideas with video because authoring in the video medium with traditional approaches inherits and exacerbates the problems from traditional media, and because the popularity of video as an authoring medium continues to grow.     

A hypermedia authoring tool for augmented and virtual reality

Most existing hypermedia authoring systems are intended for use on desktop computers. These systems are typically designed for the creation of 2D documents and therefore employ 2D authoring mechanisms. In contrast, authoring systems for nontraditional multimedia/hypermedia experiences for 3D virtual or augmented worlds focus mainly on creating separate media objects and embedding them within the user's surroundings. As a result, linking these media objects to create 3D hypermedia is a tedious manual task. To address this issue, we present an authoring tool for creating and editing linked 3D hypermedia narratives that are interwoven with a wearable computer user's surrounding environment. Our system is designed for use by authors who are not programmers, and allows them to preview their results on a desktop workstation, as well as with an augmented or virtual reality system.     

Content-based management of video in a multimedia authoring environment

This paper describes architecture and implementation of a multimedia authoring environment which features content-based management of video.The goal of our work is to provide multimedia application developers with a useful video database, where they gather video materials from various sources, store them in a organized format, and use them in multimedia applications. The principal design concepts are that (1) it should have a flexible means to store video materials in a organized format so that video clips can be retrieved easily, and that (2) it should have a view to each application so that various applications can share the video materials.To meet these requirements we designed a multimedia authoring system based on a video object model. The video object model features (1) a method to attach annotations not only to scenes but also to things shot in the video, and (2) a method to define video clips independently from physical video data.Our experiences in using the system to develop a multimedia CAI application are also described to show the advantages of our system.     

Local Painting and Deformation of Meshes on the GPU

We present a novel method to adaptively apply modifications to scene data stored in GPU memory. Such modifications may include interactive painting and sculpting operations in an authoring tool, or deformations resulting from collisions between scene objects detected by a physics engine. We only allocate GPU memory for the faces affected by these modifications to store fine‐scale colour or displacement values. This requires dynamic GPU memory management in order to assign and adaptively apply edits to individual faces at runtime. We present such a memory management technique based on a scan‐operation that is efficiently parallelizable. Since our approach runs entirely on the GPU, we avoid costly CPU–GPU memory transfer and eliminate typical bandwidth limitations. This minimizes runtime overhead to under a millisecond and makes our method ideally suited to many real‐time applications such as video games and interactive authoring tools. In addition, our algorithm significantly reduces storage requirements and allows for much higher resolution content compared to traditional global texturing approaches. Our technique can be applied to various mesh representations, including Catmull–Clark subdivision surfaces, as well as standard triangle and quad meshes. In this paper, we demonstrate several scenarios for these mesh types where our algorithm enables adaptive mesh refinement, local surface deformations and interactive on‐mesh painting and sculpting.     

多媒体结构的设计与选择

构造一个上下文敏感,具有可交互性的多媒体表示远比单纯构造视听信息或者单纯构造文本信息复杂得多.多媒体对象之间的交互需要有一个用来同步和异步交互的,包括立即响应的时空关系组织起来.根据构造环境的需要来考虑多媒体的结构构造.先通过介绍影响多媒体结构构造的7个要素来认识多媒体,然后在4种典型的构造模式中进行多媒体环境的选择.这4种构造模式分别是:结构化、时间关系、图关系、脚本形式.通过各种各样的观察和分析发现,结构化的多媒体构造为多媒体的表示提供了最为有效的表示形式.     

Feature set and architecture of a multimedia news system over an ATM network

In this paper we discuss a multimedia news system that we have developed in the Multimedia Information Research Laboratory at the University of Ottawa and beyond. We focus on the feature set—that is, the tools and facilities associated with the system. We explain the functionality of each and give some real examples of the system in action. We then outline the architecture—the system consists of a production server for document authoring, a conferencing system for collaborative news article creation, a content database for authors, a hypernews database for hypermedia news documents, a news database server with aging and archiving, and user sites. The goal was to have all components of the system communicating on OCRInet—an R&D ATM network in the Ottawa region. We then introduce the challenges of representing and browsing large video objects and to this end we introduce a novel solution that we call video-tiles. This video tool is an effective way to browse large news videos that are frequently connected with our multimedia news articles.     

An ODA-like Multimedia Document System

A multimedia document is composed of different media objects. ISO's Open Document Architecture (ODA) proposes a standard multimedia document model. However, the current ODA profile only includes static media, e.g. text, geometric graphics and images. Because the future multimedia documents not only include static media but also continuous media, e.g. video and audio, continuous media document parts should be added to have a complete multimedia document model. In this paper, we propose a multimedia document model, which is derived from ODA's concept. The proposed model is based on the object-oriented approach. Objects in the proposed document model are divided into two types: data objects and pseudo objects. Data objects are data structures of a document; pseudo objects are used to manage data objects. Based on the proposed model, a multimedia document authoring and presenting system (MMDS) is also developed on SUN SPARC workstations using the Solaris 2.X operating system     

Crowdsourcing authoring of sensory effects on videos

Human perception is inherently multi-sensorial involving five traditional senses: sight, hearing, touch, taste, and smell. In contrast to traditional multimedia, based on audio and visual stimuli, mulsemedia seek to stimulate all the human senses. One way to produce multi-sensorial content is authoring videos with sensory effects. These effects are represented as metadata attached to the video content, which are processed and rendered through physical devices into the user’s environment. However, creating sensory effects metadata is not a trivial activity because authors have to identify carefully different details in a scene such as the exact point where each effect starts, finishes, and also its presentation features such as intensity, direction, etc. It is a subjective task that requires accurate human perception and time. In this article, we aim at finding out whether a crowdsourcing approach is suitable for authoring coherent sensory effects associated with video content. Our belief is that the combination of a collective common sense to indicate time intervals of sensory effects with an expert fine-tuning is a viable way to generate sensory effects from the point of view of users. To carry out the experiment, we selected three videos from a public mulsemedia dataset, sent them to the crowd through a cascading microtask approach. The results showed that the crowd can indicate intervals in which users agree that there should be insertions of sensory effects, revealing a way of sharing authoring between the author and the crowd.

     

MediaTouch: A Native Authoring Tool for MHEG-5 Applications

MediaTouch is a visual based authoring tool for creating MHEG-5 applications. It is based on the native approach, which enables the author to operate at the level of MHEG-5 objects, in order to get the most out of the standard's features. Specialized editing functions facilitate the author's work, especially when setting properties such as the hierarchical structure of an application, dynamic behavior and appearance of objects. The paper introduces the problem of authoring multimedia applications, motivating the adoption of a standard solution, and explaining the functionality of MediaTouch, and in so doing, providing an example of use.     

3D C-string: a new spatio-temporal knowledge representation for video database systems

In video database systems, one of the most important methods for discriminating the videos is by using the objects and the perception of spatial and temporal relations that exist between objects in the desired videos. In this paper, we propose a new spatio-temporal knowledge representation called 3D C-string. The knowledge structure of 3D C-string, extended from the 2D C⁺-string, uses the projections of objects to represent spatial and temporal relations between the objects in a video. Moreover, it can keep track of the motions and size changes of the objects in a video. The string generation and video reconstruction algorithms for the 3D C-string representation of video objects are also developed. By introducing the concept of the template objects and nearest former objects, the string generated by the string generation algorithm is unique for a given video and the video reconstructed from a given 3D C-string is unique too. This approach can provide us an easy and efficient way to retrieve, visualize and manipulate video objects in video database systems. Finally, some experiments are performed to show the performance of the proposed algorithms.     

交互式分割视频运动对象的研究与实现

在 MPEG- 4视频编码标准中 ,为了实现基于视频内容的交互功能 ,视频序列的每一帧由视频对象面来表示 ,而生成视频对象面 ,需要对视频序列中运动对象进行有效分割 ,并跟踪运动对象随时间的变化 .在视频分割方法中 ,交互式分割视频对象能满足分割的效率与质量指标要求 ,因此提出了一种交互分割与自动跟踪相结合的方式来分割视频语义对象 ,即在初始分割时 ,依据用户的交互与形态学的分水线分割算法相结合提取视频对象轮廓 ,并用改进的轮廓跟踪方法有效提高视频对象轮廓的精度 ;对后续帧的跟踪 ,采用六参数仿射变换跟踪运动对象轮廓的变化 ,用平移估算的运动矢量作为初始值 ,计算六参数仿射变换的参数 .实验结果表明 ,该方法能有效地分割并跟踪视频运动对象     

基于视频对象的视频内容分级描述模型

本文提出了一种基于视频对象的视频内容分级描述模型.视频序列首先被分成一个个的镜头,在每个镜头内对视频对象进行分割和跟踪.按照镜头、视频对象、视频对象平面和视频对象区域四级框架提取特征,对视频内容进行分级描述.本文对视频内容信息的描述可以用于视频检索、视频注释等应用.     

一种异步视频对象的速率控制算法

MPEG-4提出了视频对象的概念，产生了基于对象的速率控制问题，编码的视频对象可以是同步进行也可以是异步进行，后者在低比特率条件下更具有优越性，实验结果也证实了这点，针对异步视频对象编码，提出 一种速率控制算法，以达到恒定目标比特率，该算法首先根据在一个短的时间段内，一个视频对象的同一类型编码比特数近似相等的假设来估计某时刻总的编码比特数，然后采用全局分配原则，将目标比特数分配到编码的视频对象中，以获得视频对象近似恒定的失真比，另外，还提出了一种后处理技术，并用以克服异步对象的解码合成图象的失真，实验结果表明，该算法能够对异步视频对象进行有效的比特控制，并具有较高的主客观质量。     

Constraint Techniques for Authoring Multimedia Documents

Authoring a multimedia document requires to specify both its spatial layout and its temporal organization, i.e. when and where objects such as pictures, texts and videos appear/disappear on /from the screen and when objects such as audios and videos start/end to play. In this paper, we present some benefits authors can gain when using an authoring tool in which constraints are used to specify these two kinds of information. We describe our experience in building Madeus, a constraint-based environment to design multimedia documents and we point out technical problems that still need to be solved to more completely satisfy author requirements.