A data model and a query languagefor multimedia documents databases

A multimedia application involves information that may be in a form of video, images, audio, text and graphics, need to be stored, retrieved and manipulated in large databases. In this paper, we propose an object-oriented database schema that supports multimedia documents and their temporal, spatial and logical structures. We present a document example and show how the schema can adress all the structures described. We also present a multimedia query specification language that can be used to describe a multimedia content portion to be retrieved from the database. The language provides means by which the user can specify the information on the media as well as the temoral and spatial relationships among these media.     

Synchronized delivery and playout of distributed stored multimedia streams

Multimedia streams such as audio and video impose tight temporal constraints for their presentation. Often, related multimedia streams, such as audio and video, must be presented in a synchronized way. We introduce a novel scheme to ensure the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a new protocol for synchronized playback and compute the buffer required to achieve both, the continuity within a single substream and the synchronization between related substreams. The scheme is very general and does not require synchronized clocks. Using a resynchronization protocol based on buffer level control, the scheme is able to cope with server drop-outs and clock drift. The synchronization scheme has been implemented and the paper concludes with our experimental results.     

Adaptive multimedia synchronization in a teleconference system

In this paper, we present an adaptive buffering scheme for implementing intra-stream and inter-stream synchronization in real-time multimedia applications. The essence of the proposed scheme is to dynamically enforce equalized delays to incoming media streams, in order to piece-wise smooth the network delay variations and to synchronize the streams at the sink. An adaptive control mechanism based on an event-counting algorithm is employed to calibrate the PlayOut Clocks (POCs), which manages the presentations of multimedia data. The algorithm does not rely on globally synchronized clock and makes minimal assumption on underlying network delay distribution. Also, the user defined quality of service (QoS) specifications can be directly incorporated into the design parameters of the synchronization algorithm. The proposed synchronization scheme has been experimentally implemented in a teleconference system which consists of separately controllable audio, video, and data channels. The modular structure of the synchronization control provides the flexibility to maintain an arbitrary synchronization group in conjunction with a distributed conference management scheme. This paper also shows the experimental results of the test implementation and the suitability of the proposed scheme with respect to the multimedia traffic across an FDDI/Ethernet network.     

Synchronization of distributed multimedia systems with user interactions

Without user interactions, multimedia presentations are just fancy slide shows with sound and video supports. User interactions by themselves do not change the temporal relationships among multimedia objects, such as texts, graphics, images, audio, and video, but affect the playback schedules. In this paper, we propose a synchronization mechanism to guarantee the quality of multimedia presentation with user interactions. In our protocol, each presentation site requests media transmission from the required media servers at certain time intervals prior to the playback deadlines, where these time intervals are the response times to cover possible experienced end-to-end delays and packet losses, and waits for an initial setup time to ensure intermedia synchronization before starting the presentation. Users may interact with the presentation. This synchronization mechanism solves the problems incurred by user interactions by determining the new presentation scenario produced by the interactive operation, calculating the corresponding setup time, and then rendering the new playback and retrieval schedules.     

Interactive authoring of multimedia documents in a constraint-based authoring system

As multimedia applications spread widely, it is crucial for programming and design support systems to handle “time” in multimedia documents effectively and flexibly. This paper presents a set of interactive system support tools for designing and maintaining the temporal behavior of multimedia documents. The tool set provides mechanisms for anomaly detection, temporal query processing, and interactive scheduling. It is based on a fast incremental constraint solver we have developed, which can be adapted by any constraint-based system. The incremental constraint solver provides immediate feedback to the user, supporting a highly interactive design process. Combined with existing optimal layout generation mechanisms proposed in the literature, our tools effectively utilize the flexibility provided by constraint-based systems.     

协同多媒体菱工具中演示文档的同步模型

多媒体演示文档的协同菱对文档本身的结构与同步模型提出了一定的要求。为了便于面向对象的实现以及灵活多粒度的共享，提出了一个协同多媒体著作工具中分布交互式多媒体文档的同步模型。在此模型中，文档结构分为三层：页面层、对象组层及单媒体对象层，根据各层的特点，其同步分别采用基于跳转、基于事件及基于时间的策略。     

A proxy architecture for collaborative media streaming

Streaming media from the Internet is a successful application for end-users. With the upcoming success of mobile devices and home networking environments, cooperation among users will become more important in the future. To achieve such cooperation, explicit middleware standards have been defined. On the other hand, Internet conferencing applications do not handle collaborative streaming sessions with individual control for each user. We propose a new concept for cooperation exemplary for collaborative media streaming using IETF multimedia session control protocols together with a proxy architecture. This concept enables both synchronization among clients and flexible control to individual users.     

多媒体对象的组织与结构化检索

多媒体的结构化检索具有广泛的应用前景，但多媒体的检索技术尚不成熟。多媒体数据模型的复杂性，连续媒体基于内容检索的低效，以及缺乏适用的查询语言，都使得多媒体检索困难重重。英国肯特大学新近研制的多媒体检索系统在多媒体检索的相关领域取得了一定的突破。这个系统把用户视图中的多媒体对象组织成具有层次结构的虚拟数据库，使用属性来标识数据库中多媒体对象索引的特征；查询代理机允许用户直观地构造查询过程－包括一个结     

Checking the temporal integrity of interactive multimedia documents

Abstract. When authoring multimedia scenarios, and in particular scenarios with user interaction, where the sequence and time of occurrence of interactions is not predefined, it is difficult to guarantee the consistency of the resulting scenarios. As a consequence, the execution of the scenario may result in unexpected behavior or inconsistent use of media. The present paper proposes a methodology for checking the temporal integrity of interactive multimedia document (IMD) scenarios at authoring time at various levels. The IMD flow is mainly defined by the events occurring during the IMD session. Integrity checking consists of a set of discrete steps, during which we transform the scenario into temporal constraint networks representing the constraints linking the different possible events in the scenario. Temporal constraint verification techniques are applied to verify the integrity of the scenario, deriving a minimal network, showing possible temporal relationships between events given a set of constraints. Received June 9, 1998 / Accepted November 10, 1999     

View management in multimedia databases

Abstract. Though there has been extensive work on multimedia databases in the last few years, there is no prevailing notion of a multimedia view, nor there are techniques to create, manage, and maintain such views. Visualizing the results of a dynamic multimedia query or materializing a dynamic multimedia view corresponds to assembling and delivering an interactive multimedia presentation in accordance with the visualization specifications. In this paper, we suggest that a non-interactive multimedia presentation is a set of virtual objects with associated spatial and temporal presentation constraints. A virtual object is either an object, or the result of a query. As queries may have different answers at different points in time, scheduling the presentation of such objects is nontrivial. We then develop a probabilistic model of interactive multimedia presentations, extending the non-interactive model described earlier. We also develop a probabilistic model of interactive visualization where the probabilities reflect the user profiles, or the likelihood of certain user interactions. Based on this probabilistic model, we develop three utility-theoretic based types of prefetching algorithms that anticipate how users will interact with the presentation. These prefetching algorithms allow efficient visualization of the query results in accordance with the underlying specification. We have built a prototype system that incorporates these algorithms. We report on the results of experiments conducted on top of this implementation. Received June 10, 1998 / Accepted November 10, 1999     

Scheduling and data layout policies for a near-line multimedia storage architecture

Recent advances in computer technologies have made it feasible to provide multimedia services, such as news distribution and entertainment, via high-bandwidth networks. The storage and retrieval of large multimedia objects (e.g., video) becomes a major design issue of the multimedia information system. While most other works on multimedia storage servers assume an on-line disk storage system, we consider a two-tier storage architecture with a robotic tape library as the vast near-line storage and an on-line disk system as the front-line storage. Magnetic tapes are cheaper, more robust, and have a larger capacity; hence, they are more cost effective for large scale storage systems (e.g., video-on-demand (VOD) systems may store tens of thousands of videos). We study in detail the design issues of the tape subsystem and propose some novel tape-scheduling algorithms which give faster response and require less disk buffer space. We also study the disk-striping policy and the data layout on the tape cartridge in order to fully utilize the throughput of the robotic tape system and to minimize the on-line disk storage space.     

A model for secure multimedia document database system in a distributed environment

The Internet provides a universal platform for large-scale distribution of information and supports inter-organizational services, system integration, and collaboration. Use of multimedia documents for dissemination and sharing of massive amounts of information is becoming a common practice for Internet-based applications and enterprises. With the rapid proliferation of multimedia data management technologies over the Internet, there is growing concern about security and privacy of information. Composing multimedia documents in a distributed heterogeneous environment involves integrating media objects from multiple security domains that may employ different access control policies for media objects. In this paper, we present a security model for distributed document management system that allows creation, storage, indexing, and presentation of secure multimedia documents. The model is based on a time augmented Petri-net and provides a flexible, multilevel access control mechanism that allows clearance-based access to different levels of information in a document. In addition, the model provides detailed multimedia synchronization requirements including deterministic and non-deterministic temporal relations and incomplete timing information among media objects.     

Modeling, authoring and formatting hypermedia documents in the HyperProp system

This paper discusses multimedia and hypermedia modeling, authoring and formatting tools, presenting the proposals of the HyperProp system and comparing them to related work. It also highlights several research challenges that still need to be addressed. Moreover, it stresses the importance of document logical structuring and considers the use of compositions in order to represent context relations, synchronization relations, derivation relations and task relations in hypermedia systems. It discusses temporal and spatial synchronization among multimedia objects and briefly presents the HyperProp graphical authoring and formatting tools. Integration between the proposed system and the WWW is also addressed.     

Transporting QoS adaptive flows

Distributed audio and video applications need to adapt to fluctuations in delivered quality of service (QoS). By trading off temporal and spatial quality to available bandwidth, or manipulating the playout time of continuous media in response to variation in delay, audio and video flows can be made to adapt to fluctuating QoS with minimal perceptual distortion. In this paper, we extend our previous work on a QoS Architecture (QoS-A) by populating the QoS management planes of our architecture with a framework for the control and management of multilayer coded flows operating in heterogeneous multimedia networking environments. Two key techniques are proposed: i) an end-to-end rate-shaping scheme which adapts the rate of MPEG-coded flows to the available network resources while minimizing the distortion observed at the receiver; and ii) an adaptive network service, which offers “hard” guarantees to the base layer of multilayer coded flows and “fairness” guarantees to the enhancement layers based on a bandwidth allocation technique called Weighted Fair Sharing.     

Design and evaluation of a multimedia storage server for mixed traffic

The relative simplicity of access to digital communications nowadays and the simultaneous increase in the available bandwidth are leading to the definition of new telematic services, mainly oriented towards multimedia applications and interactivity with the user. In the near future, a decisive role will be played in this scenario by the providers of interactive multimedia services of the on-demand type, which will guarantee the end user a high degree of flexibility, speed and efficiency. In this paper, some of the technical aspects regarding these service providers are dealt with, paying particular attention to the problems of storing information and managing service requests. More specifically, the paper presents and evaluates a new storage technique based on the use of disk array technology, which can manage both typical multimedia connections and traditional requests. The proposed architecture is based on the joint use of the partial dynamic declustering and the information dispersal algorithm, which are employed for the allocation and retrieval of the data stored on the disk array. We also define efficient strategies for request management in such a way as to meet the time constraints imposed by multimedia sessions and guarantee good response times for the rest of the traffic. The system proposed is then analyzed using a simulation approach.     

A study on multimedia documents behavior: a notion of equivalence

In this paper we address the problem of comparing multimedia documents, which can be described according to different reference models. If we consider pre- sentations as collections of media items and constraints among them, expressed ac- cording to their reference model, they must be translated to a common formalism in order to compare their temporal behavior and detect if they have a common component (i.e., intersection), if one of them is included in another one (i.e., inclusion), or if they have the same temporal evolution along time (i.e., equivalence). In this paper, we propose the use of automata, to describe the temporal evolution of a document, and the SMIL language as a case study, since this standard allows to describe the same behavior with different sets of tags. In case of behaviorally equivalent SMIL documents, we propose an algorithm to extract the canonical form that represents this behavior.

A hierarchical architecture for buffer management in integrated services networks

Due to their large bandwidth demand and synchronization requirements, multimedia applications, in general, consume buffers of huge size, which prevents potential customers from using multimedia services. We recognize the problem and propose a hierarchical architecture to reduce the buffer size. The architecture can be applied to both 1- and - applications. We establish the architecture by first determining neighbor sets and then applying a grouping algorithm and a renegotiation process. This architecture can also meet the synchronization requirements of multimedia applications. We evaluate the performance of the architecture through simulations and compare it with that of a direct connection architecture. The result shows that the hierarchical architecture reduces the buffer size significantly without serious penalty to the total bandwidth and without introducing extra hot spots.     

The multimedia multicasting problem

This paper explores the problems associated with the multicasting of continuous media to support multimedia group applications. The interaction between multicasting and the delivery of multiple time-correlated continuous-media streams with real-time delay requirements poses various new and interesting problems in research on communication protocols and architectures. We describe these problems, and identify where the opportunities are for effective solutions, all in the context of providing an overview of the current state of research in multimedia multicasting. The issues we discuss include quality of service, resource reservations, routing, error and traffic control, heterogeneity, and the use of hierarchical coding and open-loop control techniques.