A graphical model for audiovisual object tracking

We present a new approach to modeling and processing multimedia data. This approach is based on graphical models that combine audio and video variables. We demonstrate it by developing a new algorithm for tracking a moving object in a cluttered, noisy scene using two microphones and a camera. Our model uses unobserved variables to describe the data in terms of the process that generates them. It is therefore able to capture and exploit the statistical structure of the audio and video data separately, as well as their mutual dependencies. Model parameters are learned from data via an EM algorithm, and automatic calibration is performed as part of this procedure. Tracking is done by Bayesian inference of the object location from data. We demonstrate successful performance on multimedia clips captured in real world scenarios using off-the-shelf equipment.     

Towards a theory of multimedia metacomputing

Multimedia metacomputing is a new approach to the management and processing of multimedia data in web-based information systems. It offers high flexibility and openness while shielding the applications from any system internals. Starting with the vision of a completely open and globally distributed multimedia information system, we consider abstraction concepts required, especially transformation independence, and an appropriate semantic model.Thus, the major focus of this paper is on the abstract data and processing model called VirtualMedia,¹ which provides a transformation independence framework for multimedia processing. In particular, we describe how transformation requests are represented and processed, exploiting semantic equivalence relations on filter graphs and redundant materialization, finally yielding instantiatable plans for materializing the requested media object(s) at the client.     

Optimizing download time of embedded multimedia objects for Web browsing

Notoriously high delays in accessing Web pages loaded with massive multimedia objects are highly undesirable. Inspired by the requirements of news agencies and other information providers to include multimedia content in their pages, this paper proposes a new solution to the problem of minimizing the Web response time. We consider an environment that consists of a central multimedia repository and various sites physically dispersed. Our approach is based on simultaneous downloading of some of the embedded multimedia objects from the repository, and the rest from the regional servers. We propose a cost model to formalize the relative benefits of the proposed scheme, and design an algorithm that replicates multimedia objects so as to take advantage of concurrency in data transferring. An extensive simulation study evaluates the performance of the proposed replication policy under storage and processing capacity constraints, as well as with various network transfer rates. Comparisons are carried out with alternative schemes.     

Striped-texture image segmentation with application to multimedia security

In today’s rapid growth of volume of multimedia data, security is important yet challenging problem in multimedia applications. Image, which covers the highest percentage of the multimedia data, it is very important for multimedia security. Image segmentation is utilized as a fundamental preprocessing of various multimedia applications such as surveillance for security by breaking a given image into multiple salient regions. In this paper, we present a new image segmentation approach based on frequency-domain filtering for images with stripe texture, and generalize it to lattice fence images. Our method significantly reduces the impact of stripes on segmentation performance. The approach proposed in this paper consists of three phases. Given the images, we weaken the effect of stripe texture by filtering in the frequency domain automatically. Then, structure-preserving image smoothing is employed to remove texture details and extract the main image structures. Last, we use an effective threshold method to produce segmentation results. Our method achieves very promising results for the test image dataset and could benefit a number of new multimedia applications such as public security.

     

弱同步及其在WWW上的应用

在当今世界,WWW已经成为新颖而强大的媒体,但是,现在运行于Web上的应用仍然不是完全多媒体化的.这主要是因为Internet网络是不可靠、尽力型的,无法提供数据实时传输的保证,尤其对那些大数据量的连续媒体(如视频与音频数据).因此,在Web环境中多媒体对象间的同步就变得格外复杂.该文提出了一种新颖的同步模型：弱同步.它使多媒体文档即使在多变的环境下也能获得最佳效果.在弱同步中,不仅可以定义对象间确定的时间关系,也可以定义非确定的时间关系.此外,媒体间的同步关系,如严格同步和松散同步,也可以在文档的同步描述中定义.     

An Extended Relational Data Model for Multimedia DBMSs

We have extended the canonical relational data model to enable the management of multimedia objects. In an attempt to provide a smooth paradigm shift to multimedia information system development, we have enhanced the relational data model framework with techniques for modeling, storing and manipulating multimedia data. In particular, we have provided a graphical conceptual model for structuring a multimedia document and mapping rules for translating it into an extended relational data schema. Extensions have regarded the management of foreign keys, active components, mechanisms for the management of spatial and temporal relations, and finally functions for handling multimedia presentations. As a consequence, we have also provided extensions to the SQL language to handle these new mechanisms.     

一个多媒体数据同步模型的建立和实现

文章对多媒体数据流间存在的时序问题和时序关系进行介绍，对现有研究情况进行分析.结合一个多媒体数据模型，基于PETRI网，提出了一个同步模型，以表达媒体数据之间存在的同步关系.根据这两个模型对多媒体数据建立了相应的数据结构，并采用该数据结构介绍了对多媒体数据存取播放的实现算法.     

Multimedia summarization using social media content

In this work, we propose a novel multimedia summarization technique from Online Social Networks (OSNs). In particular, we model each Multimedia Social Network (MSN)—i.e. an OSN focusing on the management and sharing of multimedia information—using an hypergraph based approach and exploit influence analysis methodologies to determine the most important multimedia objects with respect to one or more topics of interest. Successively, we obtain from the list of candidate objects a multimedia summary using a summarization model together with an heuristics that aims to generate summaries with priority (with respect to some user keywords), continuity, variety and not receptiveness features. The performed experiments on Flickr shows the effectiveness of proposed approach.     

支持实时多媒体传输的应用层组播系统

由于IP组播并未取得预期的成功，研究人员又提出了由终端主机来代替路由器实现群组通信功能。针对实时多媒体传输的特点以及目前单组播网络混合存在的现状，该文提出了一利，新的支持实时多媒体传输的应用层组播系统。在系统中，多个网关形成一层覆盖网，由网关完成数据的复制、分发以及组的成员管理，从而在应用层实现了群组通信的功能。     

多媒体数据集中的数据挖掘:系统框架和方法

分析了目前信息环境对多媒体挖掘技术提出的需求,提出了一种适合多媒体挖掘的系统框架、一般结构和挖掘过程;描述了各种媒体可用于挖掘的特征,阐述相应的适合多媒体挖掘的基本方法;讨论了多媒体挖掘技术今后发展可能遇到的问题和面临的挑战。     

基于分层向量距离的实时多媒体数据挖掘模型

论文构造了一个实时多媒体数据挖掘模型,提出了一种原始视频数据进行数据挖掘的新机制,主要采用分层向量距离来进行动态可控序列分析分割、段内特征提取,使用粒子群K均值进行段间聚集,在一定程度上,解决了多媒体数据挖掘各方面的特殊要求。挖掘模型中各个部分与提出的技术相结合,基本上能满足实时情况下处理原始视频数据的要求。     

Continuous retrieval of multimedia data using parallelism

Most implementations of workstation-based multimedia information systems cannot support a continuous display of high resolution audio and video data and suffer from frequent disruptions and delays termed hiccups. This is due to the low I/O bandwidth of the current disk technology, the high bandwidth requirement of multimedia objects, and the large size of these objects, which requires them to be almost always disk resident. A parallel multimedia information system and the key technical ideas that enable it to support a real-time display of multimedia objects are described. In this system, a multimedia object across several disk drives is declustered, enabling the system to utilize the aggregate bandwidth of multiple disks to retrieve an object in real-time. Then, the workload of an application is distributed evenly across the disk drives to maximize the processing capability of the system. To support simultaneous display of several multimedia objects for different users, two alternative approaches are described. The first approach multitasks a disk drive among several requests while the second replicates the data and dedicates resources to each individual request. The trade-offs associated with each approach are investigated using a simulation model     

An ODA-like Multimedia Document System

A multimedia document is composed of different media objects. ISO's Open Document Architecture (ODA) proposes a standard multimedia document model. However, the current ODA profile only includes static media, e.g. text, geometric graphics and images. Because the future multimedia documents not only include static media but also continuous media, e.g. video and audio, continuous media document parts should be added to have a complete multimedia document model. In this paper, we propose a multimedia document model, which is derived from ODA's concept. The proposed model is based on the object-oriented approach. Objects in the proposed document model are divided into two types: data objects and pseudo objects. Data objects are data structures of a document; pseudo objects are used to manage data objects. Based on the proposed model, a multimedia document authoring and presenting system (MMDS) is also developed on SUN SPARC workstations using the Solaris 2.X operating system     

Intelligent multimedia data: data + indices + inference

In this paper, we present a novel approach for multimedia data indexing and retrieval that is machine independent and highly flexible for sharing multimedia data across applications. Traditional multimedia data indexing and retrieval problems have been attacked using the central data server as the main focus, and most of the indexing and query-processing for retrieval are highly application dependent. This precludes the use of created indices and query processing mechanisms for multimedia data which, in general, have a wide variety of uses across applications. The approach proposed in this paper addresses three issues: 1. multimedia data indexing; 2. inference or query processing; and 3. combining indices and inference or query mechanism with the data to facilitate machine independence in retrieval and query processing. We emphasize the third issue, as typically multimedia data are huge in size and requires intra-data indexing. We describe how the proposed approach addresses various problems faced by the application developers in indexing and retrieval of multimedia data. Finally, we present two applications developed based on the proposed approach: video indexing; and video content authorization for presentation.     

0/1背包问题在动态自适应多媒体处理方法中的应用研究

针对Snort网络入侵检测系统在大网络流量下丢包率高的问题,曾提出使用动态自适应多媒体处理方法来降低其丢包率,收到良好效果。本文在前期研究基础上,将0/1背包问题的最优化思想用于动态自适应多媒体处理方法的主要决策步骤中,分别利用动态规划的向前处理法和向后处理法从正反两个方向做出决策并求解。通过这种方法,使Snort在一段网络流量区间内,能够将有限的处理能力集中在更具危险性的多媒体数据包上。实验证明,该方法能有效提高易带危险信息的多媒体数据包的检测率。     

Heterogeneous and Multifaceted Multimedia Objects in DOOR/MM: A Role-Based Approach with Views

Although the distributed multimedia information system is getting popular nowadays, existing database systems do not offer adequate support to structural nor behavioral extension of multimedia objects. Moreover, the multifaceted nature of multimedia data has seldom been investigated. Besides the flexibility for multimedia authoring tools, dynamic extension and multifacet modeling can in general ease the task on integrating the multimedia information of heterogeneous formats over a distributed network. This paper presents the support of dynamic extension and multifacet modeling of multimedia objects in a client-server, object-role database system called DOOR. The resultant system, with multimedia functionalities, is named as DOOR/MM (DOOR with multimedia extensions). First, the underlying data model for DOOR is summarized. We then present the extension of the data model, by means of meta-object protocol (MOP), to implement the metaclasses for multimedia objects. In DOOR, objects are structured as a tree of roles. Each role may carry different properties and interact with other roles cooperatively. An object may acquire or drop multiple roles during its lifetime. Structural and behavioral extensions of multimedia objects are achieved by dynamic role playing. Multifaceted interfaces of an multimedia object are modeled by means of views on roles. In particular, we discuss different ways of using views to model different abstraction levels of the heterogeneous multimedia resources. Throughout the paper, examples such as text, bitmap, MIDI, and 3D graphic objects are used to demonstrate the significance of our approach.     

ZYX-a multimedia document model for reuse and adaptationof multimedia content

Advanced multimedia applications require adequate support for the modeling of multimedia content by multimedia document models. More and more this support calls for not only the adequate modeling of the temporal and spatial course of a multimedia presentation and its interactions, but also for the partial reuse of multimedia documents and adaptation to a given user context. However, our thorough investigation of existing standards for multimedia document models such as HTML, MHEG, SMIL, and HyTime leads to us the conclusion that these standard models do not provide sufficient modeling support for reuse and adaptation. Therefore, we propose a new approach for the modeling of adaptable and reusable multimedia content, the Z_YX model. The model offers primitives that provide-beyond the more or less common primitives for temporal, spatial, and interaction modeling-a variform support for reuse of structure and layout of document fragments and for the adaptation of the content and its presentation to the user context. We present the model in detail and illustrate the application and effectiveness of these concepts by samples taken from our Cardio-OP application in the domain of cardiac surgery. With the Z_YX model, we developed a comprehensive means for advanced multimedia content creation: support for template-driven authoring of multimedia content and support for flexible, dynamic composition of multimedia documents customized to the user's local context and needs. The approach significantly impacts and supports the authoring process in terms of methodology and economic aspects     

基于服务质量需求的存储系统动态优先权调度策略

多媒体应用要求存储系统提供保证服务质量的数据访问服务,响应时间超过延迟上限的访问请求必须小于一定的比例．基于服务质量需求的动态优先权调度策略（QADPS）根据多媒体应用接受存储系统服务的历史信息,计算多媒体应用的服务质量失败距离;并为失败距离较小的多媒体应用分配较高的调度优先权．QADPS根据多媒体应用的不同服务质量需求,实现了有差别的调度：多媒体应用的服务质量需求越严格,它的访问请求被调度的机会越高．QADPS能够在保证服务质量的前提下,支持更多的并发多媒体应用．