A lightweight framework for authoring XML multimedia content on the web

This paper addresses the issue of authoring XML multimedia content on the web. It focuses on methods that apply to different kinds of contents, including structured documents, factual data, and multimedia objects. It argues in favor of a template-based approach that enhances the ability for multiple applications to use the produced content. This approach is illustrated by AXEL, an innovative multipurpose client-side authoring framework (previously described in Sire et al. (2010)), intended for web users with limited skills. The versatility of the tool is illustrated through a series of use cases that demonstrate the flexibility of the approach for creating various kinds of web content.     

Boosting web video categorization with contextual information from social web

Web video categorization is a fundamental task for web video search. In this paper, we explore web video categorization from a new perspective, by integrating the model-based and data-driven approaches to boost the performance. The boosting comes from two aspects: one is the performance improvement for text classifiers through query expansion from related videos and user videos. The model-based classifiers are built based on the text features extracted from title and tags. Related videos and user videos act as external resources for compensating the shortcoming of the limited and noisy text features. Query expansion is adopted to reinforce the classification performance of text features through related videos and user videos. The other improvement is derived from the integration of model-based classification and data-driven majority voting from related videos and user videos. From the data-driven viewpoint, related videos and user videos are treated as sources for majority voting from the perspective of video relevance and user interest, respectively. Semantic meaning from text, video relevance from related videos, and user interest induced from user videos, are combined to robustly determine the video category. Their combination from semantics, relevance and interest further improves the performance of web video categorization. Experiments on YouTube videos demonstrate the significant improvement of the proposed approach compared to the traditional text based classifiers.     

An open framework supporting multimedia web services

With the rapid emergence of Web services, more and more Web services are published on the Internet as resources for Web application development. There may exist some relationships among different Web services, such as exact match, plug-in match, and irrelevant. In this paper, we discuss a set of requirements related to multimedia Web services, and propose a three-tier framework to establish an open environment supporting multimedia Web services, while partially implementing the requirements. This paper focuses on the design of the service broker tier that is essential for future Web services-oriented system design and integration and enabling Web services more transparent, interoperable, and fault-tolerate.     

A geometric framework for data fusion in information retrieval

Data fusion in information retrieval has been investigated by many researchers and a number of data fusion methods have been proposed. However, problems such as why data fusion can increase effectiveness and favorable conditions for the use of data fusion methods are poorly resolved at best. In this paper, we formally describe data fusion under a geometric framework, in which each component result returned from an information retrieval system for a given query is represented as a point in a multi-dimensional space. The Euclidean distance is the measure by which the effectiveness and similarity of search results are judged. This allows us to explain all component results and fused results using geometrical principles. In such a framework, score-based data fusion becomes a deterministic problem. Several interesting features of the centroid-based data fusion method and the linear combination method are discussed. Nevertheless, in retrieval evaluation, ranking-based measures are the most popular. Therefore, this paper investigates the relation and correlation between the Euclidean distance and several typical ranking-based measures. We indeed find that a very strong correlation exists between these. It means that the theorems and observations obtained using the Euclidean distance remain valid when ranking-based measures are used. The proposed framework enables us to have a better understanding of score-based data fusion and use score-based data fusion methods more precisely and effectively in various ways.     

A central multimodal fusion framework for outdoor scene image segmentation

Multimedia Tools and Applications - Robust multimodal fusion is one of the challenging research problems in semantic scene understanding. In real-world applications, the fusion system can overcome...     

Towards an ontology based framework for searching multimedia contents on the web

Multimedia Tools and Applications - We live in a world where there are huge number of consumers and producers of multimedia content. In this sea of information, finding the right content is like...     

多媒体情报系统体系结构研究

针对多媒体情报处理的现状,指出了多媒体情报处理过程中需要解决的主要问题,并在此基础上将多媒体情报的处理流程划分为多媒体情报收集、情报处理、情报服务三个阶段。以新闻视频为例对多媒体情报系统的体系结构进行了详细设计,重点设计了其中的多媒体情报处理与情报服务子系统,并对其中涉及的部分关键技术进行了研究。     

An information fusion framework for robust shape tracking

Existing methods for incorporating subspace model constraints in shape tracking use only partial information from the measurements and model distribution. We propose a unified framework for robust shape tracking, optimally fusing heteroscedastic uncertainties or noise from measurement, system dynamics, and a subspace model. The resulting nonorthogonal subspace projection and fusion are natural extensions of the traditional model constraint using orthogonal projection. We present two motion measurement algorithms and introduce alternative solutions for measurement uncertainty estimation. We build shape models offline from training data and exploit information from the ground truth initialization online through a strong model adaptation. Our framework is applied for tracking in echocardiograms where the motion estimation errors are heteroscedastic in nature, each heart has a distinct shape, and the relative motions of epicardial and endocardial borders reveal crucial diagnostic features. The proposed method significantly outperforms the existing shape-space-constrained tracking algorithm. Due to the complete treatment of heteroscedastic uncertainties, the strong model adaptation, and the coupled tracking of double-contours, robust performance is observed even on the most challenging cases.     

A general framework for multiresolution image fusion: from pixels to regions

This paper presents an overview on image fusion techniques using multiresolution decompositions. The aim is twofold: (i) to reframe the multiresolution-based fusion methodology into a common formalism and, within this framework, (ii) to develop a new region-based approach which combines aspects of both object and pixel-level fusion. To this end, we first present a general framework which encompasses most of the existing multiresolution-based fusion schemes and provides freedom to create new ones. Then, we extend this framework to allow a region-based fusion approach. The basic idea is to make a multiresolution segmentation based on all different input images and to use this segmentation to guide the fusion process. Performance assessment is also addressed and future directions and open problems are discussed as well.     

A general metric and parallel framework for adaptive image fusion in clusters

This article is dedicated to techniques and theories of image fusion in automatic ways and addresses two issues—the parameter setting and quality assessment. Optimal parameters are in demand for specific applications or comparison between fusion methods because, as basic evidence, different parameters bring different fusion effects varying over a large range. In this paper, we propose a general framework of online parameter training to search optimal values that best suit input images. Furthermore, we optimized the compute‐intensive training process using parallelization and genetic algorithm, as well as patches extraction. We also propose a metric—spatial and spectral distortion—as the learning target. The spatial and spectral distortion is a fuzzy combination of mean potential energy measuring spatial distortion and Q4 measuring spectral distortion. Optimization validation on weighted Gram–Schmidt fusion indicated linear or superlinear acceleration ability, which proved that the proposed learning framework can speed up the learning process of image fusion to an acceptable time, and can thus be applied to high‐performance platforms to process large volumes of data. Copyright © 2013 John Wiley & Sons, Ltd.     

The LEMO annotation framework: weaving multimedia annotations with the web

Cultural institutions and museums have realized that annotations contribute valuable metadata for search and retrieval, which in turn can increase the visibility of the digital items they expose via their digital library systems. By exploiting annotations created by others, visitors can discover content they would not have found otherwise, which implies that annotations must be accessible and processable for humans and machines. Currently, however, there exists no widely adopted annotation standard that goes beyond specific media types. Most institutions build their own in-house annotation solution and employ proprietary annotation models, which are not interoperable with those of other systems. As a result, annotation data are usually stored in closed data silos and visible and processable only within the scope of a certain annotation system. As the main contribution of this paper, we present the LEMO Annotation Framework. It (1) provides a uniform annotation model for multimedia contents and various types of annotations, (2) can address fragments of various content-types in a uniform, interoperable manner and (3) pulls annotations out of closed data silos and makes them available as interoperable, dereferencable Web resources. With the LEMO Annotation Framework annotations become part of the Web and can be processed, linked, and referenced by other services. This in turn leads to even higher visibility and increases the potential value of annotations.     

Web warehouse – a new web information fusion tool for web mining

In this study, we introduce a web information fusion tool – web warehouse, which is suitable for web mining and knowledge discovery. To formulate a web warehouse, a four-layer web warehouse architecture for decision support is firstly proposed. According to the layered web warehouse framework architecture, an extraction–fusion–mapping–loading (EFML) process model for web warehouse construction is then constructed. In the web warehouse process model, a series of web services including wrapper service, mediation service, ontology service and mapping service are used. Particularly, two kinds of mediators are introduced to fuse the heterogeneous web information. Finally, a simple case study is presented to illustrate the construction process of web warehouse.     

WESONet: Applying semantic web technologies and collaborative tagging to multimedia web information systems

The publication of different media types, like images, audio and video in the World Wide Web is getting more importance each day. However, searching and locating content in multimedia sites is challenging. In this paper, we propose a platform for the development of multimedia web information systems. Our approach is based on the combination between semantic web technologies and collaborative tagging. Producers can add meta-data to multimedia content associating it with different domain-specific ontologies. At the same time, users can tag the content in a collaborative way. The proposed system uses a search engine that combines both kinds of meta-data to locate the desired content. It will also provide browsing capabilities through the ontology concepts and the developed tags.     

A novel multiscale transform decomposition based multi-focus image fusion framework

Multimedia Tools and Applications - In this work, we propose a novel multiscale transform decomposition model for multi-focus image fusion to get a better fused performance. The motivation of the...     

Multi-scale depth information fusion network for image dehazing

Applied Intelligence - According to the atmospheric physical model, we can use accurate transmittance and atmospheric light information to convert a hazy image into a clean one. The scene-depth...     

A non-reference image fusion metric based on mutual information of image features

The widespread usage of image fusion causes an increase in the importance of assessing the performance of different fusion algorithms. The problem of introducing a suitable quality measure for image fusion lies in the difficulty of defining an ideal fused image. In this paper, we propose a non-reference objective image fusion metric based on mutual information which calculates the amount of information conducted from the source images to the fused image. The considered information is represented by image features like gradients or edges, which are often in the form of two-dimensional signals. In this paper, a method of estimating the joint probability distribution from marginal distributions is also presented which is employed in calculation of mutual information. The proposed method is compared with the most popular existing algorithms. Various experiments, performed on several databases, certify the efficiency of our proposed method which is more consistent with the subjective criteria.     

A general framework for image fusion based on multi-scale transform and sparse representation

In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the MST is firstly performed on each of the pre-registered source images to obtain their low-pass and high-pass coefficients. Then, the low-pass bands are merged with a SR-based fusion approach while the high-pass bands are fused using the absolute values of coefficients as activity level measurement. The fused image is finally obtained by performing the inverse MST on the merged coefficients. The advantages of the proposed fusion framework over individual MST- or SR-based method are first exhibited in detail from a theoretical point of view, and then experimentally verified with multi-focus, visible-infrared and medical image fusion. In particular, six popular multi-scale transforms, which are Laplacian pyramid (LP), ratio of low-pass pyramid (RP), discrete wavelet transform (DWT), dual-tree complex wavelet transform (DTCWT), curvelet transform (CVT) and nonsubsampled contourlet transform (NSCT), with different decomposition levels ranging from one to four are tested in our experiments. By comparing the fused results subjectively and objectively, we give the best-performed fusion method under the proposed framework for each category of image fusion. The effect of the sliding window’s step length is also investigated. Furthermore, experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance, especially for the fusion of multimodal images.