多媒体技术研究:2014——深度学习与媒体计算

目的 海量数据的快速增长给多媒体计算带来了深刻挑战。与传统以手工构造为核心的媒体计算模式不同,数据驱动下的深度学习(特征学习)方法成为当前媒体计算主流。方法 重点分析了深度学习在检索排序与标注、多模态检索与语义理解、视频分析与理解等媒体计算方面的最新进展和所面临的挑战,并对未来的发展趋势进行展望。结果 在检索排序与标注方面, 基于深度学习的神经编码等方法取得了很好的效果;在多模态检索与语义理解方面,深度学习被用于弥补不同模态间的“异构鸿沟“以及底层特征与高层语义间的”语义鸿沟“,基于深度学习的组合语义学习成为研究热点;在视频分析与理解方面, 深度神经网络被用于学习视频的有效表示方式及动作识别,并取得了很好的效果。然而,深度学习是一种数据驱动的方法,易受数据噪声影响, 对于在线增量学习方面还不成熟,如何将深度学习与众包计算相结合是一个值得期待的问题。结论 该综述在深入分析现有方法的基础上,对深度学习框架下为解决异构鸿沟和语义鸿沟给出新的思路。     

物联网发展关键技术研究

物联网发展的关键在于解决核心技术中存在的难点。重点研究了RFID、无线传感器网络和语义网格等物联网关键技术,采用融合多种关键技术解决了物联网应用难的问题。语义网格融合了语义Web机器可阅读和网格技术强大的计算能力;语义P2P技术解决了信息过载通信瓶颈问题,大大方便了传感器节点的部署以及传感器网络的扩展;采用本体技术实现了物联网中的信息语义的交互和共享,为物联网中信息的及时、无歧义理解提供了便利。     

Multimedia technology for pervasive computing environment

With the emerging multimedia techniques which can be used for various pervasive and ubiquitous applications such as healthcare, environmental monitoring, security and surveillance, there are a number of important challenges that arise when multimedia transmits in this kind of pervasive computing environments. This special issue is called for high-quality, up-to-date related multimedia technology for pervasive computing environment. In this special issue, we have accepted only nine high-quality papers. We expect that this special issue will be a trigger for further related research and technology improvements in multimedia technology for pervasive computing environment.     

Enabling decentralised management through federation

Cross-domain management is an increasingly important concern in network management and such management capability is a key-enabler of many emerging computing environments. This paper analyses the requirements for management systems that aim to support flexible and general capability sharing between autonomously managed domains. It introduces a novel Layered Federation Model (LFM) to structure this requirements analysis and describes the Federal Relationship Manager (FRM) which instantiates several layers of this model. The FRM combines semantic mapping management and authority management technologies to help solve several of the general management problems that are encountered whenever organisations enter into capability sharing agreements. An overview of related work on federation and the technical underpinnings of our approach are discussed and our work’s particular relevance to real world problems is explained through two service-centric use cases which involve the end-to-end delivery of a multimedia stream to a user’s home across several independent operators. Finally, experimental results are presented to highlight the practical advantages of our approach.     

多媒体信息检索中的查询与反馈技术

历经几十年的发展,多媒体检索取得了长足的进步,然而检索性能的提升依然受到“意图鸿沟”与“语义鸿沟”的制约.针对此问题,学术界提出了一系列查询技术帮助用户清楚地表达检索意图以及反馈技术帮助系统准确地理解用户意图与媒体数据,有效提升了检索性能.对多媒体检索中的查询与反馈技术进行了分析与讨论.分析了查询方式的演变与反馈技术的发展,综述了面向PC机、移动智能终端、触屏设备的查询技术,介绍了不同时期的反馈技术,探讨了探索式搜索中的交互问题,最后分析了该领域的未来研究趋势.     

An intelligent multimedia information system for multimodal content extraction and querying

Multimedia Tools and Applications - This paper introduces an intelligent multimedia information system, which exploits machine learning and database technologies. The system extracts semantic...     

面向企业本体复用的语义贴近度算法

摘要：为了实现企业本体的重用、整合、演化和通信,研究了本体的语义内涵和语义距离,包括语义的差异性、共性和综合性;提出一种语义贴近度算法,并通过定量计算获得不同本体的语义贴近度,提供了面向本体应用的决策依据;最后通过一个实例给出企业本体中两个概念的贴近度计算方法,表明了该语义贴近度算法的实用性和有效性。     

音频信息检索综述

随着多媒体和Internet技术的广泛应用和深入普及,多媒体数据的数量正在飞速增长,音频数据作为多媒体数据的重要组成部分,其信息量也在不断膨胀。如何有效的对音频信息进行检索成为现代信息检索的一个重要的研究领域。然而,音频数据同传统的文本数据不同,文本检索中的相关技术不能简单地应用到音频信息检索中,因为后者往往是基于语义的,所以不可避免地要用到音频数据的特征提取和模式匹配等技术。本文将对音频数据检索相关技术和系统做一综述性的介绍。     

State-of-the-art on spatio-temporal information-based video retrieval

Video retrieval is increasingly based on image content. A number of studies on video retrieval have used low-level pixel content related to statistical moments, shape, colour and texture. However, it is well recognised that such information is not enough for uniquely discriminating across different multimedia content. The use of semantic information, especially which derived from spatio-temporal analysis is of great value in multimedia annotation, archiving and retrieval. In this review paper, we detail how the use of spatiotemporal semantic knowledge is changing the way in which modern research the conducted. In this paper we review a number of studies and concepts related to such analysis, and draw important conclusions on where future research is headed.     

A semantic interoperability framework for software as a service systems in cloud computing environments

In cloud computing environments in software as a service (SaaS) level, interoperability refers to the ability of SaaS systems on one cloud provider to communicate with SaaS systems on another cloud provider. One of the most important barriers to the adoption of SaaS systems in cloud computing environments is interoperability. A common tactic for enabling interoperability is the use of an interoperability framework or model. During the past few years, in cloud SaaS level, various interoperability frameworks and models have been developed to provide interoperability between systems. The syntactic interoperability of SaaS systems have already been intensively researched. However, not enough consideration has been given to semantic interoperability issues. Achieving semantic interoperability is a challenge within the world of SaaS in cloud computing environments. Therefore, a semantic interoperability framework for SaaS systems in cloud computing environments is needed. We develop a semantic interoperability framework for cloud SaaS systems. The capabilities and value of service oriented architecture for semantic interoperability within cloud SaaS systems have been studied and demonstrated. This paper is accomplished through a number of steps (research methodology). It begins with a study on related works in the literature. Then, problem statement and research objectives are explained. In the next step, semantic interoperability requirements for SaaS systems in cloud computing environments that are needed to support are analyzed. The details of the proposed semantic interoperability framework for SaaS systems in cloud computing environments are presented. It includes the design of the proposed semantic interoperability framework. Finally, the evaluation methods of the semantic interoperability framework are elaborated. In order to evaluate the effectiveness of the proposed semantic interoperability framework for SaaS systems in cloud computing environments, extensive experimentation and statistical analysis have been performed. The experiments and statistical analysis specify that the proposed semantic interoperability framework for cloud SaaS systems is able to establish semantic interoperability between cloud SaaS systems in a more efficient way. It is concluded that using the proposed framework, there is a significant improvement in the effectiveness of semantic interoperability of SaaS systems in cloud computing environments.     

基于云计算的多媒体检索系统设计

现今普遍使用的多媒体检索系统采用的是B/S单节点架构,存在速度慢、并发性差,不能处理大规模数据的问题。在云计算技术迅速发展的今天,提出了一种基于云计算的多媒体检索方法,将基于内容的多媒体检索技术与MapReduce并行计算框架相结合,把多媒体数据库和媒体特征库存储于HDFS。针对基于内容的多媒体检索存在的＂语义鸿沟＂的问题,采用相关反馈技术来提高检索的准确性。     

Synchronization-Oriented Placement and Retrieval Strategies for Delay-Sensitive Media Streams

Multimedia data include text,image,audio and video,etc.Recent developments and advances in the areas of mass storage technology and high speed networks make it feasible for multimedia computing systems to offer services such as multimedia e-mial,teleconferencing and various vivid games.The storage and retrieval of multimedia data are two of the most important technologies in the futre multimedia systems.This paper analyzes the synchronization requirements of the delay-sensitive media streams,classifies the synchronization hierarchically and proposes several effective strategies for the intra-media and inter-media data placement problems.     

Extracting semantics from audio-visual content: the final frontier in multimedia retrieval

Multimedia understanding is a fast emerging interdisciplinary research area. There is tremendous potential for effective use of multimedia content through intelligent analysis. Diverse application areas are increasingly relying on multimedia understanding systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, pattern recognition, multimedia databases, and smart sensors. We review the state-of-the-art techniques in multimedia retrieval. In particular, we discuss how multimedia retrieval can be viewed as a pattern recognition problem. We discuss how reliance on powerful pattern recognition and machine learning techniques is increasing in the field of multimedia retrieval. We review the state-of-the-art multimedia understanding systems with particular emphasis on a system for semantic video indexing centered around multijects and multinets. We discuss how semantic retrieval is centered around concepts and context and the various mechanisms for modeling concepts and context.     

Integrating multimedia archives: the architecture and the content layer

In the last few years, numerous multimedia archives have made extensive use of digitized storage and annotation technologies. Still, the development of single points of access, providing common and uniform access to their data, despite the efforts and accomplishments of standardization organizations, has remained an open issue as it involves the integration of various large-scale heterogeneous and heterolingual systems. This paper describes a mediator system that achieves architectural integration through an extended three-tier architecture and content integration through semantic modeling. The described system has successfully integrated five multimedia archives, quite different in nature and content from each other, while also providing easy and scalable inclusion of more archives in the future.     

A support platform for distributed multimedia applications

Abstract

Considerable progress has been made in the development of technologies to support distributed multimedia computing, such that there is now a wide range of pilot applications under development. The design of support systems which help in the development of multimedia applications is therefore an emerging area of research. This paper discusses the requirements of distributed multimedia applications with respect to their support environments. A number of experimental systems are then described which intend to provide the necessary facilities for multimedia applications. Particular focus is placed on the Lancaster system which provides multimedia support for open distributed systems.     

Towards integrating semantics of multi-media resources and processes in e-Learning

Internet-based e-Learning has experienced a boom and bust situation in the past 10 years [32]. To bring in new forces to knowledge-oriented e-Learning, this paper addresses the semantic integration issue of multi-media resources and learning processes with theoretical learning supports in an integrated framework. This paper proposes a context-mediated approach that aims to enable semantic-based inter-operations across knowledge domains, even across the WWW and the Semantic Web [8]. The proposed semantic e-Learning framework enables intelligent operations of heterogeneous multi-media contents based on a generic semantic context intermediation model. This framework supports intelligent e-Learning with a knowledge network for knowledge object visualization, an enhanced Kolb's learning cycle [31] to guide learning practices, and a learning health care framework for personalized learning. W. Huang received his PhD in Computer Science from Nanjing University in 2001, MEng in Pattern Recognition and Intelligent Control and BEng in Automatic Control from Southeast University in 1998 and 1995, respectively. Dr. Huang is currently a senior lecturer with the Faculty of Computing, Information Systems and Mathematics at Kingston University London. Prior to this, he was a lecturer with the Centre for Internet Computing, The University of Hull, United Kingdom. Between October 2001 and September 2002, Dr. Huang was a post-doctoral research fellow at the University Lyon 1, France. His research interests include knowledge engineering and management, adaptive multimedia service, and pragmatic Semantic Web supporting technologies. His recent research focuses on semantic context aware computing and its applications in intelligent e-Services such as e-Learning and e-Enterprises. Dr. Huang is a member of ACM and IEEE Computer Society. E. Eze received his BS Degree in Computer Science from University of Nigeria in 1999. He is now a PhD student with the Centre for Internet Computing, The University of Hull, United Kingdom. His research interests include multimedia semantic modelling and representation and contextual knowledge engineering. D. Webster is currently studying for a PhD in Computer Science on the topic of trusted agents in the Semantic Web. He holds a 2-1 honours degree in Internet Computing from the University of Hull, UK and is a member of the British Computer Society. In addition to Web-based research, he also has an interest in graphics and has been involved in the development of graphics engines for video game projects on embedded and personal computing platforms.     

Semantic Space: an infrastructure for smart spaces

Semantic Space is a pervasive computing infrastructure that exploits semantic Web technologies to support explicit representation, expressive querying, and flexible reasoning of contexts in smart spaces.     

基于概念的多媒体数据语义检索模型*

基于概念的检索是多媒体数据语义检索的解决途径之一。对概念进行了语义扩展,提出了基于概念的多媒体数据语义检索模型,包括人机接口、知识获取、数据获取、概念检索以及语义获取等模块,并对各模块所涉及到的问题进行了分析和探讨。     

多媒体语义模型研究进展

多媒体语义研究是多媒体数据处理与多媒体信息服务领域的核心和关键问题。多媒体数据的语义问题源于多媒体的数据获取方式,在多媒体数据的应用阶段,这一问题成为制约多谋体数据使用和创作的重要瓶颈。语义模型研究是多媒体语义研究的重点,是多媒体数据处理过程的总结和抽象,其实质就是研究多媒体数据整个生命周期的语义问题。介绍了近几年多媒体语义模型在内容描述、语义表示、数据检索三个方面的研究进展情况。