Heterogeneous and Multifaceted Multimedia Objects in DOOR/MM: A Role-Based Approach with Views

Although the distributed multimedia information system is getting popular nowadays, existing database systems do not offer adequate support to structural nor behavioral extension of multimedia objects. Moreover, the multifaceted nature of multimedia data has seldom been investigated. Besides the flexibility for multimedia authoring tools, dynamic extension and multifacet modeling can in general ease the task on integrating the multimedia information of heterogeneous formats over a distributed network. This paper presents the support of dynamic extension and multifacet modeling of multimedia objects in a client-server, object-role database system called DOOR. The resultant system, with multimedia functionalities, is named as DOOR/MM (DOOR with multimedia extensions). First, the underlying data model for DOOR is summarized. We then present the extension of the data model, by means of meta-object protocol (MOP), to implement the metaclasses for multimedia objects. In DOOR, objects are structured as a tree of roles. Each role may carry different properties and interact with other roles cooperatively. An object may acquire or drop multiple roles during its lifetime. Structural and behavioral extensions of multimedia objects are achieved by dynamic role playing. Multifaceted interfaces of an multimedia object are modeled by means of views on roles. In particular, we discuss different ways of using views to model different abstraction levels of the heterogeneous multimedia resources. Throughout the paper, examples such as text, bitmap, MIDI, and 3D graphic objects are used to demonstrate the significance of our approach.     

An ODA-like Multimedia Document System

A multimedia document is composed of different media objects. ISO's Open Document Architecture (ODA) proposes a standard multimedia document model. However, the current ODA profile only includes static media, e.g. text, geometric graphics and images. Because the future multimedia documents not only include static media but also continuous media, e.g. video and audio, continuous media document parts should be added to have a complete multimedia document model. In this paper, we propose a multimedia document model, which is derived from ODA's concept. The proposed model is based on the object-oriented approach. Objects in the proposed document model are divided into two types: data objects and pseudo objects. Data objects are data structures of a document; pseudo objects are used to manage data objects. Based on the proposed model, a multimedia document authoring and presenting system (MMDS) is also developed on SUN SPARC workstations using the Solaris 2.X operating system     

An Object Model for Multimedia Programming

The development of multimedia applications is a complex task. Much of this complexity stems from requirements associated with programming multiple media objects and the control of dependences and inter-relationships between these media objects and the user(s). It is clearly necessary to have a basic framework on which to build multimedia applications in the face of such complexities. Such a conceptual model is what we have called an object model and it is the main subject of this paper. The MADE object model represents a novel approach to multimedia application programming that is founded on the two principal concepts of active objects and delegation. Although these concepts are not novel in themselves, we believe that their combined use in a multimedia development environment represents a substantial enhancement to more traditional approaches to programming in this area.     

Retrieval schedules based on resource availability and flexible presentation specifications

A distributed multimedia document presentation involves retrieval of objects from one or more document servers and their presentation at the client system. The presentation of the multimedia objects has to be carried out in accordance with the specification of temporal relationships between the objects. The retrieval of multimedia objects from the document server(s) is influenced by factors such as temporal specification of objects presentations, throughput offered by the network service provider, and the buffer resources on the client system. Flexibility in the temporal specification of the multimedia document may help in deriving an object retrieval schedule that can handle variations in network throughput and buffer resource availability. In this paper, we develop techniques for deriving a flexible object retrieval schedule for a distributed multimedia document presentation. The schedule is based on flexible temporal specification of the multimedia document using the difference constrai nts approach. We show how the derived retrieval schedule can be validated and modified to ensure that it can work with the offered network throughput and the available buffer resources.     

Exploring context and content links in social media: a latent space method

Social media networks contain both content and context-specific information. Most existing methods work with either of the two for the purpose of multimedia mining and retrieval. In reality, both content and context information are rich sources of information for mining, and the full power of mining and processing algorithms can be realized only with the use of a combination of the two. This paper proposes a new algorithm which mines both context and content links in social media networks to discover the underlying latent semantic space. This mapping of the multimedia objects into latent feature vectors enables the use of any off-the-shelf multimedia retrieval algorithms. Compared to the state-of-the-art latent methods in multimedia analysis, this algorithm effectively solves the problem of sparse context links by mining the geometric structure underlying the content links between multimedia objects. Specifically for multimedia annotation, we show that an effective algorithm can be developed to directly construct annotation models by simultaneously leveraging both context and content information based on latent structure between correlated semantic concepts. We conduct experiments on the Flickr data set, which contains user tags linked with images. We illustrate the advantages of our approach over the state-of-the-art multimedia retrieval techniques.     

Multimedia access to mobile environments using indoor semantic maps

In this paper, we present a semantic map model in order to represent multimedia environments using mobile devices. Our approach makes it possible defining indoor maps and locating the objects of interest. This model is based on an Ontology, which defines both semantic qualities and graphic forms of the objects that exist in the multimedia environment (rooms, doors, cameras, ...). The semantic of the maps enables to query and to reason the map content automatically. For example, in this work, we solve the indoor routing adding a heuristic that exploits the topology of the interior spaces, which has been evaluated by obtaining a significant improvement in indoor spaces. In addition, the proposed Ontology includes the location and the remote services of the multimedia resources, such as ambient cameras and microphones. Thus, users can access in real time to multimedia resources through the maps displayed on mobile devices. The mobile applications have been developed, tested and measured both in light devices and hand-held computers.     

Continuous retrieval of multimedia data using parallelism

Most implementations of workstation-based multimedia information systems cannot support a continuous display of high resolution audio and video data and suffer from frequent disruptions and delays termed hiccups. This is due to the low I/O bandwidth of the current disk technology, the high bandwidth requirement of multimedia objects, and the large size of these objects, which requires them to be almost always disk resident. A parallel multimedia information system and the key technical ideas that enable it to support a real-time display of multimedia objects are described. In this system, a multimedia object across several disk drives is declustered, enabling the system to utilize the aggregate bandwidth of multiple disks to retrieve an object in real-time. Then, the workload of an application is distributed evenly across the disk drives to maximize the processing capability of the system. To support simultaneous display of several multimedia objects for different users, two alternative approaches are described. The first approach multitasks a disk drive among several requests while the second replicates the data and dedicates resources to each individual request. The trade-offs associated with each approach are investigated using a simulation model     

Feature pattern based representation of multimedia documents for efficient knowledge discovery

The rapid growth of multimedia documents has raised huge demand for sophisticated multimedia knowledge discovery systems. The knowledge extraction of the documents mainly relies on the data representation model and the document representation model. As the multimedia document comprised of multimodal multimedia objects, the data representation depends on modality of the objects. The multimodal objects require distinct processing and feature extraction methods resulting in different features with different dimensionalities. Managing multiple types of features is challenging for knowledge extraction tasks. The unified representation of multimedia document benefits the knowledge extraction process, as they are represented by same type of features. The appropriate document representation will benefit the overall decision making process by reducing the search time and memory requirements. In this paper, we propose a domain converting method known as Multimedia to Signal converter (MSC) to represent the multimodal multimedia document in an unified representation by converting multimodal objects as signal objects. A tree based approach known as Multimedia Feature Pattern (MFP) tree is proposed for the compact representation of multimedia documents in terms of features of multimedia objects. The effectiveness of the proposed framework is evaluated by performing the experiments on four multimodal datasets. Experimental results show that the unified representation of multimedia documents helped in improving the classification accuracy for the documents. The MFP tree based representation of multimedia documents not only reduces the search time and memory requirements, also outperforms the competitive approaches for search and retrieval of multimedia documents.     

Enforcing role based access control model with multimedia signatures

Recently ubiquitous technology has invaded almost every aspect of the modern life. Several application domains, have integrated ubiquitous technology to make the management of resources a dynamic task. However, the need for adequate and enforced authentication and access control models to provide safe access to sensitive information remains a critical matter to address in such environments. Many security models were proposed in the literature thus few were able to provide adaptive access decisions based on the environmental changes. In this paper, we propose an approach based on our previous work [B.A. Bouna, R. Chbeir, S. Marrara, A multimedia access control language for virtual and ambient intelligence environments, In Secure Web Services (2007) 111–120] to enforce current role based access control models [M.J. Moyer, M. Ahama, Generalized role-based access control, in: Proceedings of International Conference on Distributed Computing Systems (ICDCS), Phoenix, Arizona, USA, 2001, pp. 391–398] using multimedia objects in a dynamic environment. In essence, multimedia objects tend to be complex, memory and time consuming nevertheless they provide interesting information about users and their context (user surrounding, his moves and gesture, people nearby, etc.). The idea behind our approach is to attribute to roles and permissions, multimedia signatures in which we integrate conditions based on users’ context information described using multimedia objects in order to limit role activation and the abuse of permissions in a given environment. We also describe our architecture which extends the known XACML [XACML, XACML Profile for Role Based Access Control (RBAC), <http://docs.oasis-open.org/xacml/cd-xacml-rbac-profile-01.pdf>, 2008] terminology to incorporate multimedia signatures. We provide an overview of a possible implementation of the model to illustrate how it could be valuable once integrated in an intelligent environment.     

多媒体对象的Agent展示集成模型*

根据多媒体对象的面向对象特点,给出了一种基于“事件-条件-动作”的Agent展示集成模型.该模型集对象的内容、时序、空间关系于一体,通过消息传递与对象展示状态事件控制对象展示集成的行为,保证了对象的封装性,并对用户的交互提供有力的支持,是一种动态集成模型.“事件-条件-动作”机制是主动数据库中广泛采用的方法,已有成熟的技术支持.通过对Allen所定义的时序关系的表示,说明了模型的表达能力,并通过实例加以验证.指出了展示集成模型在多媒体数据库中的作用.     

Extending multimedia languages to support multimodal user interactions

Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces (MUI). In this paper, aiming to assist the development of multimedia applications with MUIs, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the NCL (Nested Context Language) multimedia language. NCL is the declarative language for developing interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. To help evaluate our approach, we discuss a usage scenario and implement it as an NCL application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal NCL against existing multimedia and multimodal languages, for both input and output modalities.

     

Modeling and Management of Fuzzy Information in Multimedia Database Applications

In this paper, we firstly present a conceptual data model for multimedia database applications based on ExIFO₂ model. The ExIFO₂ data model is chosen as the conceptual model since it handles complex objects along with their uncertain and imprecise properties. We enhanced this conceptual model in order to meet the multimedia data requirements. In addition to uncertain and imprecise information, we present a way of handling relationships among objects of multimedia database applications. Events that might be extracted from video or audio are also considered in this study. Secondly, the conceptual model is mapped to a logical model, which the fuzzy object-oriented data (FOOD) model is chosen, for storing and manipulating the multimedia objects. This mapping is done in a way that it preserves most of the information represented at the conceptual level. Finally, in this study videos of football (soccer) games is selected as the multimedia database application to show how we handle crisp and fuzzy querying and retrieval of fuzzy and crisp data from the database. A program has been developed to draw ExIFO₂ schemas and to map the schema to FOOD code automatically.     

Optimizing top-k selection queries over multimedia repositories

Repositories of multimedia objects having multiple types of attributes (e.g., image, text) are becoming increasingly common. A query on these attributes will typically, request not just a set of objects, as in the traditional relational query model (filtering), but also a grade of match associated with each object, which indicates how well the object matches the selection condition (ranking). Furthermore, unlike in the relational model, users may just want the k top-ranked objects for their selection queries for a relatively small k. In addition to the differences in the query model, another peculiarity of multimedia repositories is that they may allow access to the attributes of each object only through indexes. We investigate how to optimize the processing of top-k selection queries over multimedia repositories. The access characteristics of the repositories and the above query model lead to novel issues in query optimization. In particular, the choice of the indexes used to search the repository strongly influences the cost of processing the filtering condition. We define an execution space that is search-minimal, i.e., the set of indexes searched is minimal. Although the general problem of picking an optimal plan in the search-minimal execution space is NP-hard, we present an efficient algorithm that solves the problem optimally with respect to our cost model and execution space when the predicates in the query are independent. We also show that the problem of optimizing top-k selection queries can be viewed, in many cases, as that of evaluating more traditional selection conditions. Thus, both problems can be viewed together as an extended filtering problem to which techniques of query processing and optimization may be adapted.     

A novel server-side proxy caching strategy for large-scale multimedia applications

Nowadays, server-side Web caching becomes an important technique used to reduce the User Perceived Latency (UPL). In large-scale multimedia systems, there are many Web proxies, connected with a multimedia server, that can cache some most popular multimedia objects and respond to the requests for them. Multimedia objects have some particular characteristic, e.g., strict QoS requirements. Hence, even some efficient conventional caching strategies based on cache hit ratio, meant for non-multimedia objects, will confront some problems in dealing with the multimedia objects. If we consider additional resources of proxy besides cache space, say bandwidth, we can readily observe that high hit ratios may deteriorate the entire system performance. In this paper, we propose a novel placement model for networked multimedia systems, referred to as the H^k/T model, which considers the combined influence of arrival rate, size, and playback time to select the objects to be cached. Based on this model, we propose an innovative Web caching algorithm, named as the ART-Greedy algorithm, which can balance the load among the proxies and achieve a minimum Average Response Time (ART) of the requests. Our experimental results conclusively demonstrate that the ART-Greedy algorithm outperforms the most popular and commonly used LFU (Least Frequently Used) algorithm significantly, and can achieve a better performance than the byte-hit algorithm when the system utilization is medium and high.     

A model-based authorware for the construction of distributed multimedia systems

Distributed multimedia systems typically involve a sophisticated user interaction. Further, objects are allocated on physically distributed computing systems, and multimedia data must be transferred across heterogeneous networks in a timely manner. These systems often have complex requirements on a user interaction, quality of service and temporal order among media streams. The design and implementation of these requirements are inherently complex and present an extraordinary design and programming challenge. Generally, these complex requirements cannot be adequately captured using a single model or a design notation. The challenge amounts to (i) identification of multiple, often orthogonal models, each capturing a specific aspect of the requirements, and (ii) provision of an authorware that supports the composition of these models. In this paper, we propose to capture the multimedia requirements in three different models: configuration, user control and presentation, and demonstrate how the composition of these models can be supported by an authorware using the Java and CORBA technologies. The concepts are illustrated using a real-life example based on a virtual city tour application that features distributed controls, collaborative work and multimedia presentations. Various distributed multimedia applications like video phone, video conferencing and distributed presentation have been successfully constructed using the proposed multiple models and authorware. The results are encouraging and the approach can shorten the development of multimedia applications considerably.     

Optimizing download time of embedded multimedia objects for Web browsing

Notoriously high delays in accessing Web pages loaded with massive multimedia objects are highly undesirable. Inspired by the requirements of news agencies and other information providers to include multimedia content in their pages, this paper proposes a new solution to the problem of minimizing the Web response time. We consider an environment that consists of a central multimedia repository and various sites physically dispersed. Our approach is based on simultaneous downloading of some of the embedded multimedia objects from the repository, and the rest from the regional servers. We propose a cost model to formalize the relative benefits of the proposed scheme, and design an algorithm that replicates multimedia objects so as to take advantage of concurrency in data transferring. An extensive simulation study evaluates the performance of the proposed replication policy under storage and processing capacity constraints, as well as with various network transfer rates. Comparisons are carried out with alternative schemes.     

Shape Morphing-Based Control of Robotic Visual Servoing

We present an approach for controlling robotic interactions with objects, using synthetic images generated by morphing shapes. In particular, we attempt the problem of positioning an eye-in-hand robotic system with respect to objects in the workspace for grasping and manipulation. In our formulation, the grasp position (and consequently the approach trajectory of the manipulator), varies with each object. The proposed solution to the problem consists of two parts. First, based on a model-based object recognition framework, images of the objects taken at the desired grasp pose are stored in a database. The recognition and identification of the grasp position for an unknown input object (selected from the family of recognizable objects) occurs by morphing its contour to the templates in the database and using the virtual energy spent during the morph as a dissimilarity measure. In the second step, the images synthesized during the morph are used to guide the eye-in-hand system and execute the grasp. The proposed method requires minimal calibration of the system. Furthermore, it conjoins techniques from shape recognition, computer graphics, and vision-based robot control in a unified engineering amework. Potential applications range from recognition and positioning with respect to partially-occluded or deformable objects to planning robotic grasping based on human demonstration.     

On the Design of Distributed Object Placement and Load Balancing Strategies in Large-Scale Networked Multimedia Storage Systems

In a large-scale multimedia storage system (LMSS) where client requests for different multimedia objects may have different demands, the placement and replication of the objects is an important factor, as it may result in an imbalance in server loading across the system. Since replication management and load balancing are all the more crucial issues in multimedia systems, in the literature, these problems are handled by centralized servers. Each object storage server (OSS) responds to the requests that come from the centralized servers independently and has no communication with other OSSs in the system. In this paper, we design a novel distributed load balancing strategy for LMSS, in which OSSs can cooperate to achieve higher performance. Such OSS modeled as an M/G/m system can replicate the objects to and balance the requests among other servers to achieve a near-optimal average waiting time (AWT) of the requests in the system. We validate the performance of the system via rigorous simulations with respect to several influencing factors and prove that our proposed strategy is scalable, flexible, and efficient for real-life applications.     

Personalized Tag Recommendation Using Social Influence

Tag recommendation encourages users to add more tags in bridging the semantic gap between human concept and the features of media object,which provides a feasible solution for content-based multimedia information retrieval.In this paper,we study personalized tag recommendation in a popular online photo sharing site - Flickr.Social relationship information of users is collected to generate an online social network.From the perspective of network topology,we propose node topological potential to characterize user’s social influence.With this metric,we distinguish different social relations between users and find out those who really have influence on the target users.Tag recommendations are based on tagging history and the latent personalized preference learned from those who have most influence in user’s social network.We evaluate our method on large scale real-world data.The experimental results demonstrate that our method can outperform the non-personalized global co-occurrence method and other two state-of-the-art personalized approaches using social networks.We also analyze the further usage of our approach for the cold-start problem of tag recommendation.     

TrendsSummary: a platform for retrieving and summarizing trendy multimedia contents

With the flood and popularity of various multimedia contents on the Internet, searching for appropriate contents and representing them effectively has become an essential part for user satisfaction. So far, many contents recommendation systems have been proposed for this purpose. A popular approach is to select hot or popular contents for recommendation using some popularity metric. Recently, various social network services (SNSs) such as Facebook and Twitter have become a widespread social phenomenon owing to the smartphone boom. Considering the popularity and user participation, SNS can be a good source for finding social interests or trends. In this study, we propose a platform called TrendsSummary for retrieving trendy multimedia contents and summarizing them. To identify trendy multimedia contents, we select candidate keywords from raw data collected from Twitter using a syntactic feature-based filtering method. Then, we merge various keyword variants based on several heuristics. Next, we select trend keywords and their related keywords from the merged candidate keywords based on term frequency and expand them semantically by referencing portal sites such as Wikipedia and Google. Based on the expanded trend keywords, we collect four types of relevant multimedia contents—TV programs, videos, news articles, and images—from various websites. The most appropriate media type for the trend keywords is determined based on a naïve Bayes classifier. After classification, appropriate contents are selected from among the contents of the selected media type. Finally, both trend keywords and their related multimedia contents are displayed for effective browsing. We implemented a prototype system and experimentally demonstrated that our scheme provides satisfactory results.