MING-I: a distributed interactive multimedia document development mechanism

The next generation of interactive multimedia documents can contain both static media, e.g., text, graph, image, and continuous media, e.g., audio and video, and can provide user interactions in distributed environments. However, the temporal information of multimedia documents cannot be described using traditional document structures, e.g., Open Document Architecture (ODA) and Standard Generalized Mark-up Language (SGML); the continuous transmission of media units also raises some new synchronization problems, which have not been met before, for processing user interactions. Thus, developing a distributed interactive multimedia document system should resolve the issues of document model, presentation control architecture, and control scheme. In this paper, we (i) propose a new multimedia document model that contains the logical structure, the layout structure, and the temporal structure to formally describe multimedia documents, and (ii) point out main interaction-based synchronization problems, and propose a control architecture and a token-based control scheme to solve these interaction-based synchronization problems. Based on the proposed document model, control architecture, and control scheme, a distributed interactive multimedia document development mechanism, which is called MING-I, is developed on SUN workstations.     

Transformation and exchange of multimedia objects in distributed multimedia systems

One of the challenges in the design of a distributed multimedia system is devising suitable specification models for various schemas in different levels of the system. Another important research issue is the integration and synchronization of heterogeneous multimedia objects. In this paper, we present our models for multimedia schemas and transformation algorithms. They transform high-level multimedia objects into schemas that can be used to support the presentation and communication of the multimedia objects. A key module in the system is the Object Exchange Manager (OEM). In this paper, we present the design and implementation of the OEM module, and discuss in detail the interaction between the OEM and other modules in a distributed multimedia system.     

HeiRAT - Quality-of-service management for distributed multimedia systems

Multimedia systems must be able to support a certain quality of service (QoS) to satisfy the stringent real-time performance requirements of their applications. HeiRAT, the Heidelberg Resource Administration Technique, is a comprehensive QoS management system that was designed and implemented in connection with a distributed multimedia platform for networked PCs and workstations. HeiRAT includes techniques for QoS negotiation, QoS calculation, resource reservation, and resource scheduling for local and network resources.     

Synchronization support and group-membership services for reliable distributed multimedia applications

We describe a collection of algorithms designed to support reliable synchronization and group membership services for distributed multimedia applications. In particular, we consider those applications that require interactivity, isochronous rendering of multimedia data, and high reliability. We show that the algorithms we propose (i) provide reliable support for the synchronization of multimedia data streams, despite the occurrence of possible communication failures, (ii) maintain a consistent view of the relative group membership of all the nonfaulty application components, (iii) guarantee time-bounded delay of component failure detection and join, and (iv) meet effectively possible scalability requirements of the applications.     

Analysis,management and indexing of distributed multimedia performances based on audience feedback

Interactive performance systems allow multiple, networked users to take part in a performance either as players or audience members and influence the performance's development in real time. Among the most prominent research issues concerning the development of these systems are the provision of effective interaction capabilities and adequate means of expression to the audience. This article describes a set of methods for the detection, analysis, synchronization and rendering of audience behavior in real-time during such an event. Furthermore, it describes ways in which audience feedback is utilized to dynamically index the contents of a performance so as to embed this event in large-scale multimedia services. All these methods have been applied in the creation of MISSION, a multi-user game involving both players and audience. This article presents and analyzes the design of the system and the results of user trials with it.     

Dynamic end-to-end QoS management middleware for distributed multimedia systems

In this paper, we present a separable, reusable middleware solution that provides coordinated, end-to-end QoS management over any type of service component, and can use existing (legacy) QoS management solutions (by using wrappers) in a distributed multimedia system. Our middleware solution incorporates strategic and tactical QoS managers, and supports protocols and messages between tactical managers and managed application components, and between QoS managers in the management hierarchy. Strategic QoS managers take a global view of QoS provided by a set of application components within the manager's policy domain. Tactical QoS managers provide local control over application components. We introduce the concept of QoS policy domains to scope the authority of a strategic QoS manager. We describe how the management hierarchy is dynamically configured and reconfigured based on runtime needs of the application.     

Modeling Distributed Multimedia Synchronization with DSPN

Multimedia synchronization is the essential technology for the integration of multimedia in distributed multimedia systems.The multimedia synchronization model has been recognized by many researchers as a premise of the implementation of multimedia synchronization.In distributed multimedia systems,the characteristic of multimedia synchronization is dynamic,and the key medium has the priority in multimedia synchronization.The previously proposed multimedia synchronization models cannot meet these requirements.So a new multimedia dynamic synchronization model-DSPN,based on the timed Petri-net has been designed in this paper.This model can not only let the distributed multimedia system keep multimedia synchronization in a more precise and effective manner according to the runtime situation of the system,but also allow the user to interact with the presentation of multimedia.     

A synchronization algorithm for distributed multimedia environments

Network synchronization plays a significant role in transmitting multimedia objects over computer networks. Even packets from a single channel must be synchronized due to the problems in a packet switching environment, such as network jitter, frequency, and time offsets. We present an algorithm that determines the set of packets generated periodically by various participants arriving at a node. The basic advantage of the proposed algorithm is that the receiver estimates the reference times (expected arrival times of the packets) and achieves synchronization, without knowledge of the packet delays. The accuracy is improved and the complexity is reduced by predicting the time/frequency offsets between the clocks at the source and the mixer. The error is calculated by the Chernoff bound, demonstrated by simulation, and shown to be acceptable in practical applications.     

Detecting global predicates in distributed systems with clocks

Summary. This paper proposes a framework for detecting global state predicates in systems of processes with approximately-synchronized real-time clocks. Timestamps from these clocks are used to define two orderings on events: “definitely occurred before” and “possibly occurred before”. These orderings lead naturally to definitions of 3 distinct detection modalities, i.e., 3 meanings of “predicate held during a computation”, namely: (“ possibly held”), (“ definitely held”), and (“ definitely held in a specific global state”). This paper defines these modalities and gives efficient algorithms for detecting them. The algorithms are based on algorithms of Garg and Waldecker, Alagar and Venkatesan, Cooper and Marzullo, and Fromentin and Raynal. Complexity analysis shows that under reasonable assumptions, these real-time-clock-based detection algorithms are less expensive than detection algorithms based on Lamport's happened-before ordering. Sample applications are given to illustrate the benefits of this approach. Received: January 1999 / Accepted: November 1999     

Quality-of-service adaptation in distributed multimedia applications

High-speed networks and powerful end-systems enable new types of applications, such as video-on-demand and teleconferencing. Such applications are very demanding on quality of service (QoS) because of the isochronous nature of the media they are using. To support these applications, QoS guarantees are required. However, even with service guarantees, violations may occur because of resources shortage, e.g., network congestion. In this paper we propose new adaptation approaches, which allow the system to recover automatically, if possible, from QoS violations (1) by identifying a new configuration of system components that might support the initially agreed QoS and by performing a user-transparent transition from the original configuration to the new one, (2) by redistributing the levels of QoS that should be supported, in the future, by the components, or (3) by redistributing the levels of QoS that should be supported immediately to meet end-to-end requirements based on the principle that (local) QoS violation at one component may be recovered immediately by the other components participating in the support of the requested service. The proposed approaches, together with suitable negotiation mechanisms, allow us (1) to reduce the probability of QoS violations which may be noticed by the user, and thus, to increase the user confidence in the service provider, and (2) to improve the utilization of the system resources, and thus to increase the system availability.     

A standard model for multimedia synchronization: PREMO synchronization objects

This paper describes an event-based synchronization mechanism, which is at the core of the inter-media synchronization in the upcoming standard for multimedia presentation, PREMO. The synchronization mechanism of PREMO is a powerful tool, based on a small number of concepts, and on cooperation among active objects, and represents a synthesis of various synchronization models described in the literature. This model can serve as a basis for the implementation of complex synchronization patterns in multimedia presentations, both purely event-based, as well as time-based.     

Plausible clocks: constant size logical clocks for distributed systems

Summary. In a Distributed System with N sites, the precise detection of causal relationships between events can only be done with vector clocks of size N. This gives rise to scalability and efficiency problems for logical clocks that can be used to order events accurately. In this paper we propose a class of logical clocks called plausible clocks that can be implemented with a number of components not affected by the size of the system and yet they provide good ordering accuracy. We develop rules to combine plausible clocks to produce more accurate clocks. Several examples of plausible clocks and their combination are presented. Using a simulation model, we evaluate the performance of these clocks. We also present examples of applications where constant size clocks can be used. Received: January 1997 / Accepted: January 1999     

Automatic verification for a class of distributed systems

Summary. The paper presents a new analysis method for a class of concurrent systems which are formed of several interacting components with the same structure. The model for these systems is composed of a control process and a set of homogeneous user processes. The control and user processes are modeled by finite labeled state transition systems which interact by means of enabling functions and triggering mechanisms. Based on this structure, an analysis method is presented which allows system properties, derived by reachability analysis for a finite number of user processes, to be generalized to an arbitrary number of user processes. A procedure for the automatic verification of properties such as mutual exclusion and absence of deadlocks is presented and is then used to provide for the first time a fully automated verification of the Lamport's fast mutual exclusion algorithm. Received: October 1998/Accepted January 2000     

Synchronized delivery and playout of distributed stored multimedia streams

Multimedia streams such as audio and video impose tight temporal constraints for their presentation. Often, related multimedia streams, such as audio and video, must be presented in a synchronized way. We introduce a novel scheme to ensure the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a new protocol for synchronized playback and compute the buffer required to achieve both, the continuity within a single substream and the synchronization between related substreams. The scheme is very general and does not require synchronized clocks. Using a resynchronization protocol based on buffer level control, the scheme is able to cope with server drop-outs and clock drift. The synchronization scheme has been implemented and the paper concludes with our experimental results.     

A multimedia social interaction service for inclusive community living: Initial user trials

The move from institution to community care has resulted in many people receiving care at home. For some, disability or frailty restricts their involvement in social activities outside the home, resulting in unacceptable social isolation. This problem is compounded if the person has a speech or language impairment. In this paper, we will describe a communication service designed to provide nonspeaking people with a means to interact socially when living independently, based on the sharing of stories using pictures and other media. Initial exploration on the usability of the system by a pair of representative users will be described. Published online: 6 November 2002     

Floor control for multimedia conferencing and collaboration

Floor control allows users of networked multimedia applications to utilize and share resources such as remote devices, distributed data sets, telepointers, or continuous media such as video and audio without access conflicts. Floors are temporary permissions granted dynamically to collaborating users in order to mitigate race conditions and guarantee mutually exclusive resource usage. A general framework for floor control is presented. Collaborative environments are characterized and the requirements for realization of floor control will be identified. The differences to session control, as well as concurrency control and access control are elicited. Based upon a brief taxonomy of collaboration-relevant parameters, system design issues for floor control are discussed. Floor control mechanisms are discerned from service policies and principal architectures of collaborative systems are compared. The structure of control packets and an application programmer's interface are proposed and further implementation aspects are elaborated. User-related aspects such as floor presentation, assignment, and the timely stages of floor-controlled interaction in relation to user-interface design are also presented.     

Adaptive multimedia synchronization in a teleconference system

In this paper, we present an adaptive buffering scheme for implementing intra-stream and inter-stream synchronization in real-time multimedia applications. The essence of the proposed scheme is to dynamically enforce equalized delays to incoming media streams, in order to piece-wise smooth the network delay variations and to synchronize the streams at the sink. An adaptive control mechanism based on an event-counting algorithm is employed to calibrate the PlayOut Clocks (POCs), which manages the presentations of multimedia data. The algorithm does not rely on globally synchronized clock and makes minimal assumption on underlying network delay distribution. Also, the user defined quality of service (QoS) specifications can be directly incorporated into the design parameters of the synchronization algorithm. The proposed synchronization scheme has been experimentally implemented in a teleconference system which consists of separately controllable audio, video, and data channels. The modular structure of the synchronization control provides the flexibility to maintain an arbitrary synchronization group in conjunction with a distributed conference management scheme. This paper also shows the experimental results of the test implementation and the suitability of the proposed scheme with respect to the multimedia traffic across an FDDI/Ethernet network.     

Checking the temporal integrity of interactive multimedia documents

Abstract. When authoring multimedia scenarios, and in particular scenarios with user interaction, where the sequence and time of occurrence of interactions is not predefined, it is difficult to guarantee the consistency of the resulting scenarios. As a consequence, the execution of the scenario may result in unexpected behavior or inconsistent use of media. The present paper proposes a methodology for checking the temporal integrity of interactive multimedia document (IMD) scenarios at authoring time at various levels. The IMD flow is mainly defined by the events occurring during the IMD session. Integrity checking consists of a set of discrete steps, during which we transform the scenario into temporal constraint networks representing the constraints linking the different possible events in the scenario. Temporal constraint verification techniques are applied to verify the integrity of the scenario, deriving a minimal network, showing possible temporal relationships between events given a set of constraints. Received June 9, 1998 / Accepted November 10, 1999     

Causality and atomicity in distributed computations

Summary. In a distributed system, high-level actions can be modeled by nonatomic events. This paper proposes causality relations between distributed nonatomic events and provides efficient testing conditions for the relations. The relations provide a fine-grained granularity to specify causality relations between distributed nonatomic events. The set of relations between nonatomic events is complete in first-order predicate logic, using only the causality relation between atomic events. For a pair of distributed nonatomic events X and Y, the evaluation of any of the causality relations requires integer comparisons, where and , respectively, are the number of nodes on which the two nonatomic events X and Y occur. In this paper, we show that this polynomial complexity of evaluation can by simplified to a linear complexity using properties of partial orders. Specifically, we show that most relations can be evaluated in integer comparisons, some in integer comparisons, and the others in integer comparisons. During the derivation of the efficient testing conditions, we also define special system execution prefixes associated with distributed nonatomic events and examine their knowledge-theoretic significance. Received: July 1997 / Accepted: May 1998     

The BG distributed simulation algorithm

We present a shared memory algorithm that allows a set of f+1 processes to wait-free “simulate” a larger system of n processes, that may also exhibit up to f stopping failures. Applying this simulation algorithm to the k-set-agreement problem enables conversion of an arbitrary k-fault-tolerant{\it n}-process solution for the k-set-agreement problem into a wait-free k+1-process solution for the same problem. Since the k+1-processk-set-agreement problem has been shown to have no wait-free solution [5,18,26], this transformation implies that there is no k-fault-tolerant solution to the n-process k-set-agreement problem, for any n. More generally, the algorithm satisfies the requirements of a fault-tolerant distributed simulation.\/ The distributed simulation implements a notion of fault-tolerant reducibility\/ between decision problems. This paper defines these notions and gives examples of their application to fundamental distributed computing problems. The algorithm is presented and verified in terms of I/O automata. The presentation has a great deal of interesting modularity, expressed by I/O automaton composition and both forward and backward simulation relations. Composition is used to include a safe agreement\/ module as a subroutine. Forward and backward simulation relations are used to view the algorithm as implementing a multi-try snapshot\/ strategy. The main algorithm works in snapshot shared memory systems; a simple modification of the algorithm that works in read/write shared memory systems is also presented. Received: February 2001 / Accepted: February 2001