压缩域多媒体数据处理技术研究

多媒体数据压缩技术是多媒体、网络通讯以及计算机等应用领域的一项关键技术。多媒体数据经过压缩后再进行存储或通过网络进行传输,这已经逐渐成为多媒体应用中的标准模式;同时,多媒体应用中所涉及到的各种媒体数据,如图象、视频和音频数据,往往需要进行各种灵活的操作与处理,因此,压缩域多媒体数据直接处理技术成为一项很有意义的研究领域。文中首先就压缩数据处理的基本概念进行讨论,然后集中分析了这一研究领域的重要研究     

Content-based retrieval in digital libraries

With the recent developments in multimedia and telecommunication technologies, content-based information is becoming increasingly important for various areas such as digital libraries, interactive video and multimedia publishing. Multimedia data refers to simple structured data (such as numbers and short strings), large unstructured data (such as text documents, images, audio and video data) and complex structured data (such as maps, graphs, charts and tables). In this article, we briefly address content-based retrieval and the issues of representation, storage and retrieval of multimedia objects in digital libraries. We then very briefly identify some open areas of research     

Feature pattern based representation of multimedia documents for efficient knowledge discovery

The rapid growth of multimedia documents has raised huge demand for sophisticated multimedia knowledge discovery systems. The knowledge extraction of the documents mainly relies on the data representation model and the document representation model. As the multimedia document comprised of multimodal multimedia objects, the data representation depends on modality of the objects. The multimodal objects require distinct processing and feature extraction methods resulting in different features with different dimensionalities. Managing multiple types of features is challenging for knowledge extraction tasks. The unified representation of multimedia document benefits the knowledge extraction process, as they are represented by same type of features. The appropriate document representation will benefit the overall decision making process by reducing the search time and memory requirements. In this paper, we propose a domain converting method known as Multimedia to Signal converter (MSC) to represent the multimodal multimedia document in an unified representation by converting multimodal objects as signal objects. A tree based approach known as Multimedia Feature Pattern (MFP) tree is proposed for the compact representation of multimedia documents in terms of features of multimedia objects. The effectiveness of the proposed framework is evaluated by performing the experiments on four multimodal datasets. Experimental results show that the unified representation of multimedia documents helped in improving the classification accuracy for the documents. The MFP tree based representation of multimedia documents not only reduces the search time and memory requirements, also outperforms the competitive approaches for search and retrieval of multimedia documents.     

Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework

In this paper, a subspace-based multimedia data mining framework is proposed for video semantic analysis, specifically video event/concept detection, by addressing two basic issues, i.e., semantic gap and rare event/concept detection. The proposed framework achieves full automation via multimodal content analysis and intelligent integration of distance-based and rule-based data mining techniques. The content analysis process facilitates the comprehensive video analysis by extracting low-level and middle-level features from audio/visual channels. The integrated data mining techniques effectively address these two basic issues by alleviating the class imbalance issue along the process and by reconstructing and refining the feature dimension automatically. The promising experimental performance on goal/corner event detection and sports/commercials/building concepts extraction from soccer videos and TRECVID news collections demonstrates the effectiveness of the proposed framework. Furthermore, its unique domain-free characteristic indicates the great potential of extending the proposed multimedia data mining framework to a wide range of different application domains.     

基于隐马尔可夫模型的复杂数据挖掘实现

利用隐马尔可夫模型(HMM)对多媒体数据仓库进行复杂数据挖掘，复杂数据挖掘要解决的难题是音频和视频识别。建立音频和视频的识别模型及其相关的算法，在视频识别算法上构造出符合HMM的识别方法。根据模型建立系统，实验证明声音的识别率最高达到96．67％，视频中特征值的检测率可达87．81％。研究结果可以应用在多媒体的识别和数据挖掘领域，提供一个比较完整的复杂数据挖掘的模型和算法。     

Learning Based Neural Similarity Metrics for Multimedia Data Mining

Multimedia data mining refers to pattern discovery, rule extraction and knowledge acquisition from multimedia database. Two typical tasks in multimedia data mining are of visual data classification and clustering in terms of semantics. Usually performance of such classification or clustering systems may not be favorable due to the use of low-level features for image representation, and also some improper similarity metrics for measuring the closeness between multimedia objects as well. This paper considers a problem of modeling similarity for semantic image clustering. A collection of semantic images and feed-forward neural networks are used to approximate a characteristic function of equivalence classes, which is termed as a learning pseudo metric (LPM). Empirical criteria on evaluating the goodness of the LPM are established. A LPM based k-Mean rule is then employed for the semantic image clustering practice, where two impurity indices, classification performance and robustness are used for performance evaluation. An artificial image database with 11 semantics is employed for our simulation studies. Results demonstrate the merits and usefulness of our proposed techniques for multimedia data mining.     

Reconfigurable media processing

Multimedia processing is becoming increasingly important with wide variety of applications ranging from multimedia cell phones to high definition interactive television. Media processing techniques typically involve the capture, storage, manipulation and transmission of multimedia objects such as text, handwritten data, audio objects, still images, 2D/3D graphics, animation and full-motion video. A number of implementation strategies have been proposed for processing multimedia data. These approaches can be broadly classified into two major categories, namely (i) general purpose processors with programmable media processing capabilities, and (ii) dedicated implementations (ASICs). We have performed a detailed complexity analysis of the recent multimedia standard (MPEG-4) which has shown the potential for reconfigurable computing, that adapts the underlying hardware dynamically in response to changes in the input data or processing environment. We therefore propose a methodology for designing a reconfigurable media processor. This involves hardware–software co-design implemented in the form of a parser, profiler, recurring pattern analyzer, spatial and temporal partitioner. The proposed methodology enables efficient partitioning of resources for complex and time critical multimedia applications.     

多媒体数据挖掘的体系结构和方法

提出了一个多媒体数据挖掘系统的一般结构(M3)，包括多媒体数据库(MD)、多媒体挖掘引擎(MME)和多媒体挖掘界面(MMI)，重点分析了几种挖掘方法(分类、关联和聚类)在多媒体挖掘中的应用。针对不同的媒体，如图像、音频、视频，讨论了各自的挖掘特点和主要挖掘内容。     

A survey on wireless multimedia sensor networks

The availability of low-cost hardware such as CMOS cameras and microphones has fostered the development of Wireless Multimedia Sensor Networks (WMSNs), i.e., networks of wirelessly interconnected devices that are able to ubiquitously retrieve multimedia content such as video and audio streams, still images, and scalar sensor data from the environment. In this paper, the state of the art in algorithms, protocols, and hardware for wireless multimedia sensor networks is surveyed, and open research issues are discussed in detail. Architectures for WMSNs are explored, along with their advantages and drawbacks. Currently off-the-shelf hardware as well as available research prototypes for WMSNs are listed and classified. Existing solutions and open research issues at the application, transport, network, link, and physical layers of the communication protocol stack are investigated, along with possible cross-layer synergies and optimizations.     

面向声音监测的多媒体传感器节点硬件设计与实现

多媒体传感器网络能够采集和传输信息丰富的音频、视频、图像等多媒体信息,具有十分广泛的应用前景,是近年来无线传感器网络的研究热点。目前,国外多媒体传感器节点主要针对图像传输;国内使用的节点大多都难以满足多媒体信息处理和传输等方面的应用要求。本文针对鄱阳湖鸟类声音监测的应用,设计实现了一种新型的高性能多媒体传感器节点。实验结果表明,由该节点组成的多媒体传感器网络,能有效建立路由,实时地感知和采集网络覆盖区域内的多媒体信息。     

无线多媒体传感器网络的QoS问题研究

无线多媒体传感器网络（WMSNs）是在传统无线传感器网络（WSNs）基础上发展起来的具有音频、视频、图像等多媒体信息感知功能的新型传感器网络．具有广阔的发展前景。WMSNs感知媒体丰富、数据量大和处理任务复杂等显著特点,使其QoS问题的研究极具挑战性。该文分析了WMSNs的QoS需求,并从MAC层和网络层讨论了这一领域的国内外研究现状。最后对QoS亟待解决的问题作了总结。     

DCT域图象处理和特征提取技术

现今 ,大量的图象与视频信息都是以压缩数据格式进行存储和传输的 .DCT(Discrete Cosine Transform离散余弦变换 )是目前应用最为广泛的多媒体数据压缩技术之一 .直接在 DCT域实现如视频编辑、特征提取等传统空域处理技术 ,能够避免繁琐的压缩数据编、解码操作 ,减少处理时间和数据处理量 ,节省内存空间 .这一技术对于高速海量的数据处理场合 ,如 Internet信息检索、视频编辑和检索、远程监视图象的理解等 ,是很有吸引力的 ,因此其是近年来国际上有关领域的研究热点之一 .本文对近年来文献中所见的 DCT域图象处理和特征提取技术进行了回顾和综述 ,并在此基础上对其发展方向进行了探讨     

多媒体传感器网络及其研究进展

作为一种全新的信息获取和处理技术,多媒体传感器网络较之传统传感器网络更多地关注于音频、视频、图像等大数据量、大信息量媒体的采集与处理,在军事、民用及商业领域中具有广阔的应用前景.介绍了多媒体传感器网络的概念与特点,着重探讨了多媒体传感器网络所面临的挑战与国内外的研究进展,最后分析了当前亟待解决的问题,并展望了其未来的发展趋势.多媒体传感器网络是一种新的概念系统但也存在较多的问题需要解决,其研究具有很强的理论意义和实用价值.     

基于VFW的远程视频捕获技术研究

多媒体技术在人们日常生活中的应用越来越广泛,图像、视频、音频等多媒体数据逐渐成为信息处理领域中主要的信息媒体形式。视频捕获技术是信息处理中的重要环节,研究该项技术具有重要的实用价值。文章提出一种基于VFW的远程视频捕获方法。该方法利用VFW捕获视频数据,采用H.263编码标准压缩视频数据,利用面向连接协议的流式套接字实现实时视频流的传输,结合多线程技术实现视频文件播放。然后,基于Windows操作系统设计实现了远程视频捕获系统。实验结果表明,该方法CPU占用率低、内存占用小,可靠性强,具有较好的应用价值。     

Evaluation of a CPU Scheduling Mechanism for Multimedia Systems

Multimedia applications handling audio and video data have to obey time characteristics of these media types. Besides a basic functionality to express time relations, correctness with respect to time constraints requires mechanisms which lead to favoured processing of multimedia operations. CPU scheduling techniques based on the experience from real-time operating systems offer a solution and provide multimedia applications with the ability to meet time-related quality of service requirements. This paper discusses mechanisms to express time in multimedia systems and describes an implementation of a CPU scheduler designed to run under IBM's UNIX derivate AIX. The evaluation of the implementation based on measurements shows that the scheduler is able to support the time requirements of multimedia applications and that such mechanisms are indeed necessary since otherwise deadline violations occur.     

CMMB标准MFS流的音视频解复用

CMMB（中国移动多媒体广播）是我国自主研发的第一套面向多种移动终端的系统。本文根据移动多媒体广播行业标准,对复用结构进行分析,详细介绍了解析音视频数据的方法,并使用离线MFS进行仿真,获得音视频数据流,存入文件。     

Video data mining: semantic indexing and event detection from the association perspective

Advances in the media and entertainment industries, including streaming audio and digital TV, present new challenges for managing and accessing large audio-visual collections. Current content management systems support retrieval using low-level features, such as motion, color, and texture. However, low-level features often have little meaning for naive users, who much prefer to identify content using high-level semantics or concepts. This creates a gap between systems and their users that must be bridged for these systems to be used effectively. To this end, in this paper, we first present a knowledge-based video indexing and content management framework for domain specific videos (using basketball video as an example). We will provide a solution to explore video knowledge by mining associations from video data. The explicit definitions and evaluation measures (e.g., temporal support and confidence) for video associations are proposed by integrating the distinct feature of video data. Our approach uses video processing techniques to find visual and audio cues (e.g., court field, camera motion activities, and applause), introduces multilevel sequential association mining to explore associations among the audio and visual cues, classifies the associations by assigning each of them with a class label, and uses their appearances in the video to construct video indices. Our experimental results demonstrate the performance of the proposed approach.     

多媒体语义模型研究进展

多媒体语义研究是多媒体数据处理与多媒体信息服务领域的核心和关键问题。多媒体数据的语义问题源于多媒体的数据获取方式,在多媒体数据的应用阶段,这一问题成为制约多谋体数据使用和创作的重要瓶颈。语义模型研究是多媒体语义研究的重点,是多媒体数据处理过程的总结和抽象,其实质就是研究多媒体数据整个生命周期的语义问题。介绍了近几年多媒体语义模型在内容描述、语义表示、数据检索三个方面的研究进展情况。