MING-I: a distributed interactive multimedia document development mechanism

The next generation of interactive multimedia documents can contain both static media, e.g., text, graph, image, and continuous media, e.g., audio and video, and can provide user interactions in distributed environments. However, the temporal information of multimedia documents cannot be described using traditional document structures, e.g., Open Document Architecture (ODA) and Standard Generalized Mark-up Language (SGML); the continuous transmission of media units also raises some new synchronization problems, which have not been met before, for processing user interactions. Thus, developing a distributed interactive multimedia document system should resolve the issues of document model, presentation control architecture, and control scheme. In this paper, we (i) propose a new multimedia document model that contains the logical structure, the layout structure, and the temporal structure to formally describe multimedia documents, and (ii) point out main interaction-based synchronization problems, and propose a control architecture and a token-based control scheme to solve these interaction-based synchronization problems. Based on the proposed document model, control architecture, and control scheme, a distributed interactive multimedia document development mechanism, which is called MING-I, is developed on SUN workstations.     

A data model and a query languagefor multimedia documents databases

A multimedia application involves information that may be in a form of video, images, audio, text and graphics, need to be stored, retrieved and manipulated in large databases. In this paper, we propose an object-oriented database schema that supports multimedia documents and their temporal, spatial and logical structures. We present a document example and show how the schema can adress all the structures described. We also present a multimedia query specification language that can be used to describe a multimedia content portion to be retrieved from the database. The language provides means by which the user can specify the information on the media as well as the temoral and spatial relationships among these media.     

Automatically generating consistent schedules for multimedia documents

A schedule for a multimedia document indicates when document events should occur. We describe a two-phase algorithm that automatically produces schedules for interactive multimedia documents, which can contain both predictable behavior (such as audio and video) and unpredictable behavior (such as user interaction and programs with unpredictable execution times). The first phase of the algorithm, called the compiletime scheduler, preprocesses high-level temporal specifications before the document is presented and creates as much of the schedule as possible. Our compiletime scheduler is conceptually similar to T_EX's spatial layout algorithm in that it permits time to be stretched or shrunk between events inside media segments to arrive at an optimal presentation for a document. The second phase of the algorithm, called the runtime scheduler, resolves the presentation of media segments that depend upon unpredictable behavior.     

Zodiac: A history-based interactive video authoring system

Easy-to-use audio/video authoring tools play a crucial role in moving multimedia software from research curiosity to mainstream applications. However, research in multimedia authoring systems has rarely been documented in the literature. This paper describes the design and implementation of an interactive video authoring system called Zodiac, which employs an innovative edit history abstraction to support several unique editing features not found in existing commercial and research video editing systems. Zodiac provides users a conceptually clean and semantically powerful branching history model of edit operations to organize the authoring process, and to navigate among versions of authored documents. In addition, by analyzing the edit history, Zodiac is able to reliably detect a composed video stream's shot and scene boundaries, which facilitates interactive video browsing. Zodiac also features a video object annotation capability that allows users to associate annotations to moving objects in a video sequence. The annotations themselves could be text, image, audio, or video. Zodiac is built on top of MMFS, a file system specifically designed for interactive multimedia development environments, and implements an internal buffer manager that supports transparent lossless compression/decompression. Shot/scene detection, video object annotation, and buffer management all exploit the edit history information for performance optimization.     

An ODA-like Multimedia Document System

A multimedia document is composed of different media objects. ISO's Open Document Architecture (ODA) proposes a standard multimedia document model. However, the current ODA profile only includes static media, e.g. text, geometric graphics and images. Because the future multimedia documents not only include static media but also continuous media, e.g. video and audio, continuous media document parts should be added to have a complete multimedia document model. In this paper, we propose a multimedia document model, which is derived from ODA's concept. The proposed model is based on the object-oriented approach. Objects in the proposed document model are divided into two types: data objects and pseudo objects. Data objects are data structures of a document; pseudo objects are used to manage data objects. Based on the proposed model, a multimedia document authoring and presenting system (MMDS) is also developed on SUN SPARC workstations using the Solaris 2.X operating system     

A spatio-temporal semantic model for multimedia database systemsand multimedia information systems

As more information sources become available in multimedia systems, the development of abstract semantic models for video, audio, text, and image data is becoming very important. An abstract semantic model has two requirements: it should be rich enough to provide a friendly interface of multimedia presentation synchronization schedules to the users and it should be a good programming data structure for implementation in order to control multimedia playback. An abstract semantic model based on an augmented transition network (ATN) is presented. The inputs for ATNs are modeled by multimedia input strings. Multimedia input strings provide an efficient means for iconic indexing of the temporal/spatial relations of media streams and semantic objects. An ATN and its subnetworks are used to represent the appearing sequence of media streams and semantic objects. The arc label is a substring of a multimedia input string. In this design, a presentation is driven by a multimedia input string. Each subnetwork has its own multimedia input string. Database queries relative to text, image, and video can be answered via substring matching at subnetworks. Multimedia browsing allows users the flexibility to select any part of the presentation they prefer to see. This means that the ATN and its subnetworks can be included in multimedia database systems which are controlled by a database management system (DBMS). User interactions and loops are also provided in an ATN. Therefore, ATNs provide three major capabilities: multimedia presentations, temporal/spatial multimedia database searching, and multimedia browsing     

Graphical Transformation of Multimedia XML Documents

As a commonly acceptable standard for guiding Web markup documents, XML allows the Internet users to create multimedia documents of their preferred structures and share with other people. The creation of various multimedia document structures, typically as trees, implies that some kinds of conversion mechanisms are needed for people using different structures to understand each other. This paper presents a visual approach to the representation and validation of multimedia document structures specified in XML and transformation of one structure to another. The underlying theory of our approach is a context-sensitive graph grammar formalism. The paper demonstrates the conciseness and expressiveness of the graph grammar formalism. An example XML structure is provided and its graph grammar representation, validation and transformation to a multimedia representation are presented.     

WESONet: Applying semantic web technologies and collaborative tagging to multimedia web information systems

The publication of different media types, like images, audio and video in the World Wide Web is getting more importance each day. However, searching and locating content in multimedia sites is challenging. In this paper, we propose a platform for the development of multimedia web information systems. Our approach is based on the combination between semantic web technologies and collaborative tagging. Producers can add meta-data to multimedia content associating it with different domain-specific ontologies. At the same time, users can tag the content in a collaborative way. The proposed system uses a search engine that combines both kinds of meta-data to locate the desired content. It will also provide browsing capabilities through the ontology concepts and the developed tags.     

SHIATSU: tagging and retrieving videos without worries

The dramatic growth of video content over modern media channels (such as the Internet and mobile phone platforms) directs the interest of media broadcasters towards the topics of video retrieval and content browsing. Several video retrieval systems benefit from the use of semantic indexing based on content, since it allows an intuitive categorization of videos. However, indexing is usually performed through manual annotation, thus introducing potential problems such as ambiguity, lack of information, and non-relevance of index terms. In this paper, we present SHIATSU, a complete system for video retrieval which is based on the (semi-)automatic hierarchical semantic annotation of videos exploiting the analysis of visual content; videos can then be searched by means of attached tags and/or visual features. We experimentally evaluate the performance of SHIATSU on two different real video benchmarks, proving its accuracy and efficiency.     

校园网络教学媒体同步直播系统的设计与实现

介绍了校园网络教学媒体同步直播系统的设计目标、设计方法和实现步骤。采用先进的流媒体传输技术,基于跨平台Web数据直播技术,实时采集直播现场的音视频信号,并通过IP网络实时地将这些现场信息直播出去。通过基于Web的同步媒体技术,把视频/音频信号与课件数据完整同步地集成在一起,实现远程多媒体同步直播和同步录制,生成完整的多媒体课件。     

The perceived utility of standard ontologies in document management for specialized domains

A user-based document management system has been developed for small communities on the Web. The system is based on the free annotation of documents by users. A number of annotation support tools are used to suggest possible annotations, including suggesting terms from external ontologies. This paper outlines some evaluation data on how users actually interact with the system in annotating their document especially on the use of standard ontologies. Results indicate that although an established external taxonomy can be useful in proposing annotation terms, users appear to be very selective in their use of the terms proposed and to have little interest in adhering to the particular hierarchical structure provided.     

Multimedia Clip Generation From Documents for Browsing on Mobile Devices

Small displays on mobile handheld devices, such as personal digital assistants (PDAs) and cellular phones, are the bottlenecks for usability of most content browsing applications. Generally, conventional content such as documents and Web pages need to be modified for effective presentation on mobile devices. This paper proposes a novel visualization for documents, called multimedia thumbnails, which consists of text and image content converted into playable multimedia clips. A multimedia thumbnail utilizes visual and audio channels of small portable devices as well as both spatial and time dimensions to communicate text and image information of a single document. The proposed algorithm for generating multimedia thumbnails includes 1) a semantic document analysis step, where salient content from a source document is extracted; 2) an optimization step, where a subset of this extracted content is selected based on time, display, and application constraints; and 3) a composition step, where the selected visual and audible document content is combined into a multimedia thumbnail. Scalability of MMNails that allows generation of multimedia clips of various lengths is also described. A user study is presented that evaluates the effectiveness of the proposed multimedia thumbnail visualization.     

The Directors’ cut: a solution to collaborative multimedia management

Web 2.0 applications allow rich media contents to be exposed and shared by users. Nevertheless, usually, a multimedia is provided as an unicum, made by synchronized media items. Sound tracks, video sequences, captions, cannot be customized “on-the-fly” by users. Managing multimedia in a deep way would meet the expectations of nowadays Web prosumers (i.e. producers and consumers), and it would widen the audience. Describing and synchronizing each medium, as well as specifying different alternative contents for it, are the keystones of multimedia customization and of audience widening. This paper presents a multimedia collaborative system, which provides support to the arrangement of medium into a multi-views composed multimedia. Each prosumer can add medium by juxtaposition or by defining it as an alternative (audio, video, textual) version of an existing one. The implementation of such a system is based on SMIL 3.0 specification but implements a new and compact syntax to let users manipulate the original multimedia synchronization and their alternatives. The proposed approach has been put to test in two different scenarios.     

Guest Editors' Introduction: 3D Documents

Digital libraries (DLs) in general and technical or cultural preservation applications in particular offer a rich set of multimedia objects like audio, music, images, videos, and also 3D models. However, instead of handling these objects consistently as regular documents - in the same way we treat textual documents - most applications handle them differently. Considering that textual documents are only one media type among many, it's clear that this type of document is handled quite specially. A full-text search engine lets users retrieve a specific document based on its content - that is, one or more words that appear in it. Content-based retrieval of other media types is an active research area, and in the case of 3D documents, only pilot applications exist.     

Adlet: an active document abstraction for multimedia informationfusion

A new approach is described for the fusion of multimedia information based on the concept of active documents advertising on the Internet, whereby the metadata of a document travels in the network to seek out documents of interest to the parent document and, at the same time, advertises its parent document to other interested documents. This abstraction of metadata is called an adlet, which is the core of our approach. Two important features make this approach applicable to multimedia information fusion, information retrieval, data mining, geographic information systems, and medical information systems: 1) any document, including a Web page, database record, video file, audio file, image and even paper documents, can be enhanced by an adlet and become an active document; and 2) any node in a nonactive network can be enhanced by adlet-savvy software and the adlet-enhanced node can coexist with other nonenhanced nodes. An experimental prototype provides a testbed for feasibility studies in a hybrid active network     

基于RTMP的文档在线浏览工具

提出基于RTMP的文档在线浏览实现方案。该方案能够实现各种文档在线浏览,并且可以同步播放,达到各个客户端同步浏览的效果。首先我们在线转换各种文档为Flash文件,然后通过客户端的Flash文件调用文档转换后的Flash文件,实现文档在线浏览,然后通过客户端的操作,实现文档的翻页、拖放、以及缩放,再加上基于RTMP的Red5服务器,利用共享对象来实现文档的同步播放,各个客户端能够同步浏览。本系统关键为：文档转换、文档流传榆和文档同步浏览,该系统最终实现了文档的在线浏览,并增加了同步浏览的功能,提高了工作效率。