A mediator-based approach for integrating heterogeneous multimedia sources

In many applications, the information required by the user cannot be found in just one source, but has to be retrieved from many varying sources. This is true not only of formatted data in database management systems, but also of textual documents and multimedia data, such as images and videos. We propose a mediator system that provides the end-user with a single query interface to an integrated view of multiple heterogeneous data sources. We exploit the capabilities of the MOMIS integration system and the MILOS multimedia data management system. Each multimedia source is managed by an instance of MILOS, in which a collection of multimedia records is made accessible by means of similarity searches employing the query-by-example paradigm. MOMIS provides an integrated virtual view of the underlying multimedia sources, thus offering unified multimedia access services. Two features are that MILOS is flexible—it is not tied to any particular similarity function—and the MOMIS’s mediator query processor only exploits the ranks of the local answers.     

基于概念的多媒体数据语义检索模型*

基于概念的检索是多媒体数据语义检索的解决途径之一。对概念进行了语义扩展,提出了基于概念的多媒体数据语义检索模型,包括人机接口、知识获取、数据获取、概念检索以及语义获取等模块,并对各模块所涉及到的问题进行了分析和探讨。     

Demonstration of an interactive multimedia environment

The images in this article tell the story of one user's experience with a virtual reality environment that promotes collaboration and learning     

Personalized multimedia content delivery on an interactive table by passive observation of museum visitors

The amount of multimedia data collected in museum databases is growing fast, while the capacity of museums to display information to visitors is acutely limited by physical space. Museums must seek the perfect balance of information given on individual pieces in order to provide sufficient information to aid visitor understanding while maintaining sparse usage of the walls and guaranteeing high appreciation of the exhibit. Moreover, museums often target the interests of average visitors instead of the entire spectrum of different interests each individual visitor might have. Finally, visiting a museum should not be an experience contained in the physical space of the museum but a door opened onto a broader context of related artworks, authors, artistic trends, etc. In this paper we describe the MNEMOSYNE system that attempts to address these issues through a new multimedia museum experience. Based on passive observation, the system builds a profile of the artworks of interest for each visitor. These profiles of interest are then used to drive an interactive table that personalizes multimedia content delivery. The natural user interface on the interactive table uses the visitor’s profile, an ontology of museum content and a recommendation system to personalize exploration of multimedia content. At the end of their visit, the visitor can take home a personalized summary of their visit on a custom mobile application. In this article we describe in detail each component of our approach as well as the first field trials of our prototype system built and deployed at our permanent exhibition space at LeMurate (http://www.lemurate.comune.fi.it/lemurate/) in Florence together with the first results of the evaluation process during the official installation in the National Museum of Bargello (http://www.uffizi.firenze.it/musei/?m=bargello).     

The study on content based multimedia data retrieval system

Of late, advance in hardware and communications technology has been rapidly increasing the demand for diverse multimedia information, which, including all image, audio, video, text, numerical data, etc., should be designed to excel the existing information processing system in the functions of data storage, search, transmission, display, etc. The newest image retrieval system is gradually being converted from text-based into content-based retrieval, which uses the image content itself as features. In content-based retrieval, how to combine the color, shape, layout, texture, etc. used for describing each image or object is considered an important element. The existing method has chiefly used histogram out of the content-based image method using color information, but this has a drawback in being sensitive to brightness of light and the object size in the image. Thus, the present methods is intended to design and implement a system that can retrieve images similar to the query image from image database without losing image information in the use of color features.     

A biologically inspired multimedia content repurposing system in heterogeneous environments

The application of biologically inspired algorithms to multimedia repurposing systems in heterogeneous network environments is gaining importance due to its ability to adapt and customize multimedia content to frequently changing network environments. This paper presents a biologically inspired proxy-based multimedia content adaptation system, which is used to repurpose multimedia content dynamically for the transmission over heterogeneous networks and end devices. We use a series of repurposing proxies’ services in a chain fashion between the sender and multiple end devices. In order to find the appropriate chain of repurposing services that satisfy the Quality of Experience (QoE) requirements, the proposed service selection algorithm uses the ant colony metaphor. During the communication session, the algorithm uses biological foraging behavior inspired from ant agents to find optimal service paths between the media sender and the destination. Experimental results demonstrate that the proposed algorithm provides significant performance gain over traditional, state of the art selection algorithms and saves the average delay.     

Topic correlation model for cross-modal multimedia information retrieval

In this paper, we present a simple and effective topic correlation model (TCM) for cross-modal multimedia retrieval by jointly modeling the text and image components in multimedia documents. In this model, the image component is represented by the bag-of-features model based on local scale-invariant feature transform features, meanwhile the text component is described by a topic distribution learned from a latent topic model. Statistical correlations between these two mid-level features are investigated by mapping them into a semantic space. These cross-modality correlations are used to calculate the conditional probabilities of answers in one modality while given query in the other modality. The model is tested on three cross-modal retrieval benchmark problems including Wikipedia documents in both English and Chinese. Experimental results have demonstrated that the new TCM model achieves the best performance compared to recent state-of-the-art cross-modal retrieval models on the given benchmarks.     

Large-scale multimedia content delivery over optical networks for interactive TV services

iTVP is a system built for delivery of live TV programming, video-on-demand and audio-on-demand with interactive access over IP networks. It has a nationwide range and is designed to provide service to a high number of concurrent users. Hence, scalability is one of the most important issues. In this paper we described how scalability is achieved in a two-level hierarchical architecture of the delivery system. We present the principles of content caching mechanisms and the management of the resulting distributed content repository. The system is implemented and currently operates in a test installation. Although the range of the test system operations is limited, as far as the number of users and the size of the content repository is concerned, the experience gained with iTVP so far provides an insight into the expected prototype performance. We present a system performance evaluation and describe the experience gained during the iGrid demonstration.     

Extracting semantics from audio-visual content: the final frontier in multimedia retrieval

Multimedia understanding is a fast emerging interdisciplinary research area. There is tremendous potential for effective use of multimedia content through intelligent analysis. Diverse application areas are increasingly relying on multimedia understanding systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, pattern recognition, multimedia databases, and smart sensors. We review the state-of-the-art techniques in multimedia retrieval. In particular, we discuss how multimedia retrieval can be viewed as a pattern recognition problem. We discuss how reliance on powerful pattern recognition and machine learning techniques is increasing in the field of multimedia retrieval. We review the state-of-the-art multimedia understanding systems with particular emphasis on a system for semantic video indexing centered around multijects and multinets. We discuss how semantic retrieval is centered around concepts and context and the various mechanisms for modeling concepts and context.     

HAMS: Scalable peer-to-peer multimedia streaming model in heterogeneous networks

In a peer-to-peer (P2P) overlay network, a large number and various types of peer processes are interconnected in networks and are cooperating by using multimedia contents like movies and music. Here, multimedia contents are in nature distributed to peers in various ways like downloading and caching to the peers. Multimedia streaming is a key technology to realize multimedia applications in networks. In multimedia streaming applications, multimedia contents are required to be reliable and continuously delivered to processes in a real-time manner. Some contents peer may not send packets of a content at a required rate due to limited computation resource and a communication channel may not support enough Quality of Service (QoS) due to congestions and faults. Thus, P2P overlay networks are in nature heterogeneous. In this paper, we newly discuss a heterogeneous asynchronous multi-source streaming (HAMS) model where multiple contents peers transmit packets of a multimedia content to a requesting leaf peer to increase the throughput, reliability, and scalability in P2P overlay networks. Here, some pair of channels between contents and leaf peers may support different QoS. Peers may be faulty and some pair of contents peers may have different transmission rates. Finally, we show the HAMS model can support higher throughput and shorter transmission time than the other models in the evaluation.     

Combining spatial and colour information for content based image retrieval

Colour is one of the most important features in content based image retrieval. However, colour is rarely used as a feature that codes local spatial information, except for colour texture. This paper presents an approach to represent spatial colour distributions using local principal component analysis (PCA). The representation is based on image windows which are selected by two complementary data driven attentive mechanisms: a symmetry based saliency map and an edge and corner detector. The eigenvectors obtained from local PCA of the selected windows form colour patterns that capture both low and high spatial frequencies, so they are well suited for shape as well as texture representation. Projections of the windows selected from the image database to the local PCs serve as a compact representation for the search database. Queries are formulated by specifying windows within query images. System feedback makes both the search process and the results comprehensible for the user.     

Eventor: an authoring system for interactive multimedia applications

Programming, testing, and maintaining interactive multimedia applications (IMAs) are still difficult and expensive, while substantial progress has been made to reduce the burden on authors. As IMAs get larger and more complex the difficulties will increase. To overcome the complexity of such IMAs, we argue that authoring systems should provide such facilities as (1) a traditional and intuitivedivide-and-conquer paradigm for solving large and complex problems in various fields, (2)formal specification of the behaviors of IMAs for checking the syntactic correctness of visual expressions or semantic anomalies, and (3)automatic aids like validation of temporal constraints and verification of visual expressions. In this paper, we investigate the properties of IMAs for recognizing the inherent interactivity and concurrency. We propose a specification method based on Milner'sCalculus of Communicating Systems (CCS), which is a well-known formal mechanism for specifying the concurrency in various distributed applications. We also design and implement an authoring system calledEventor (Event Editor), which is based on CCS and composed of three tools: a Temporal Synchronizer, a Spatial Synchronizer, and a User Interaction Builder. They focus on describing the temporal and spatial synchronizations and user interactions while they rely on existing tools in Intel's Digital Video Interactive (DVI) for supporting other functionalities. By editing a simple computer aided instruction (CAI) application, we illustrate that our specification mechanism is well-suited for handling the interactivity of multimedia applications, and Eventor is a simple, efficient, and powerful enough tool to handle practical applications. Especially the incremental refinement and the formal specification based on the CCS allow Eventor to be extended with formal verifications to cope with large and complex applications.     

A computation-oriented multimedia data streams model for content-based information retrieval

Multimedia applications nowadays are becoming prevalent. In the past the relational database model was generalized to the multimedia database model. More recently the relational database model was generalized to the data streams model, as the technology advanced and data became bulky and unbounded in size due to the utilization of sensor networks. In this paper we take one more step of generalization by providing a multimedia data streams model. The objective is to furnish a formal framework to design multimedia data streams (MMDS) schema for efficient content based information retrieval. We also extend the functional dependency theory and the normalization framework to handle multimedia data streams. Finally we present algorithmic methods of generating continuous multimedia queries along with examples for illustration.     

Advances in interactive digital multimedia systems

This paper introduces basic concepts in digital multimedia systems and surveys recent literature. Background is provided regarding developments in interactive videodiscs, which first made images and video accessible through computer systems. Digital storage media, including optical, magnetic, and network options, are addressed. The characteristics of audio and video and their digital representations are discussed. Because these media are so demanding of space and channel bandwidth, compression methods are reviewed. Standards for digital multimedia are considered. Current multimedia systems are described, and future prospects are indicated     

Embedding interactive sounds in multimedia applications

We describe a system for generating and controlling sound effects from within applications. We discuss performance demands and current technology constraints on sound synthesis methods, highlight several distinct interactive control strategies, and demonstrate a development environment for making a large database of heterogeneous sound models manageable by developers who are not sound synthesis experts.     

ZYX-a multimedia document model for reuse and adaptationof multimedia content

Advanced multimedia applications require adequate support for the modeling of multimedia content by multimedia document models. More and more this support calls for not only the adequate modeling of the temporal and spatial course of a multimedia presentation and its interactions, but also for the partial reuse of multimedia documents and adaptation to a given user context. However, our thorough investigation of existing standards for multimedia document models such as HTML, MHEG, SMIL, and HyTime leads to us the conclusion that these standard models do not provide sufficient modeling support for reuse and adaptation. Therefore, we propose a new approach for the modeling of adaptable and reusable multimedia content, the Z_YX model. The model offers primitives that provide-beyond the more or less common primitives for temporal, spatial, and interaction modeling-a variform support for reuse of structure and layout of document fragments and for the adaptation of the content and its presentation to the user context. We present the model in detail and illustrate the application and effectiveness of these concepts by samples taken from our Cardio-OP application in the domain of cardiac surgery. With the Z_YX model, we developed a comprehensive means for advanced multimedia content creation: support for template-driven authoring of multimedia content and support for flexible, dynamic composition of multimedia documents customized to the user's local context and needs. The approach significantly impacts and supports the authoring process in terms of methodology and economic aspects     

A layered structure for a design space dedicated to rich interactive multimedia content

Multimedia Tools and Applications - In this paper we propose and discuss a layered organization for a design space suitable for the visualization of and interaction with rich information...     

RETRACTED ARTICLE: Community guidance model based on interactive multimedia system

The rapid development of current wisdom education has become the mainstream that leads the development of education informatization. The development of wisdom education relies on big data network resources to adapt to online teaching resources and online learning environment. Wisdom education can meet the needs of teachers, students, researchers, and the public areas under the support of the Internet of Things, cloud computing, big data, and ubiquitous networks. As a very important part of wisdom education, the wisdom education environment plays a fundamental role in wisdom education. In an intelligent learning environment, personalized learners need to build an efficient community intelligence education model, which acquires and stores the data produced by the learners touching the related physical elements in the intelligent situation through the sensing device. An intelligent learning environment is able to analyze and infer the data and adapt the present educational research according to the individual needs of learners’ program providing “intelligent” learning services. In addition, guided by the system model and design principles of the wisdom education environment, this paper put forward the initial design ideas of the wisdom education inquiry base in order to provide some guidance to the construction of the wisdom education environment. This paper introduces XML language to standardize the representation of learning situation model and illustrates it with a concrete example.