基于内容的表示及其在远程教育系统中的应用

文章首先简要介绍了MPEG－4的基本思想和特点，基于内容的两种结构；然后介绍了三个使用基于内容表示的远程教育系统的实例，其中第三个例子是一个新颖的多媒体课件制作系统，这些例子说明了基于内容的表示特别适用于多媒体信息的描述，具有很好的应用前景。     

MPEG-4: a multimedia compression standard for interactiveapplications and services

The Moving Pictures Experts Group (MPEG), which produced the MPEG-1 and MPEG-2 video and audio compression standards, is developing the MPEG-4 standard. MPEG-4 targets interactive multimedia applications and will become a standard in 1999. As well as an increased compression efficiency, MPEG-4 will also offer content-based functionality, i.e. the possibility of accessing and manipulating individual objects in the picture. Furthermore, MPEG-4 will offer possibilities for efficient video storage and for transmission over poor audio and video channels at bit rates between 5 kbit/s and 4 Mbit/s. This paper gives an overview of the state of the art of MPEG-4 development, concentrating especially on video content-based functionality, which is so important for interactive applications     

MPEG-4视频对象分割技术

随着MPEG-4,MPEG-7的研究发展,其基于内容的编码和面向对象的存取和操纵技术日益得到人们的重视。基于对象的视频图像分割是实现MPEG-4基于内容的编码和交互功能的关键。视频图像分割方法分为自动分割法和半自动分割法两种。结合视频分割的发展趋势,深入介绍了基于对象的视频分割的主要技术及国内外的最新研究算法,包括数学形态学算法以及活动轮廓模型(蛇模型)在该领域的应用,并分析了当前视频分割技术尚存在的问题和研究前景。     

基于内容视频信息检索系统的分析研究

介绍了基于内容视频检索系统所涉及的主要技术和工作原理,对目前国外几个典型视频检索系统进行了分析,指出了存在的问题及相关的解决方案,并就今后如何进行基于内容视频检索系统的研究提出了一些新的看法.     

视频分割算法的研究

视频对象分割是视频处理中的难点问题,它在基于内容的检索、对象识别和交互操作的多媒体中有重要应用。视频对象分割技术具有重要的研究意义和应用价值。本文主要分析了2种视频分割算法,包括基于时空联合的视频分割算法、基于运动一致性的视频分割算法。基于时空联合的视频分割算法是充分利用时间和空间信息,以获得满意的分割效果;基于运动一致性的视频分割算法是选用稳健的多分辨率平均位移帧差平方最小方法来估计视频对象的参数化运动。     

The role of analysis in content-based video coding and indexing

The increasing spread of digital technology in many areas, notably telecommunications, and entertainment (TV/cinema), is nowadays changing the production, delivery, and consumption paradigms for multimedia information. New applications with critical requirements in terms of content-based interactivity are imminent, motivating the evolution of the models used for data representation, notably for coding and indexing. The emerging MPEG-4 and MPEG-7 standards are the recognition, by the industry, of these upcoming needs. This paper addresses the problem of video analysis for content-based coding and indexing in the context of a changing technological landscape. The main video analysis objectives and constraints are identified, the role of user interaction is studied, and some application examples are described.     

Content-based retrieval of dynamic PET functional images

The recent information explosion has led to a massively increased demand for multimedia data storage in integrated database systems. Content-based retrieval is an important alternative and complement to traditional keyword-based searching for multimedia data and can greatly enhance information management. However, current content-based image retrieval techniques have some deficiencies when applied in the biomedical functional imaging domain. In this paper, we presented a prototype design for a content-based functional image retrieval database system for dynamic positron emission tomography (PET). The system supports efficient content-based retrieval based on physiological kinetic features and reduces image storage requirements. This design makes it possible to maintain a large number of patient data sets online and to rapidly retrieve dynamic functional image sequences for the interpretation and generation of physiological parametric images, and offers potential advantages in medical image data management and telemedicine, as well as providing possible opportunities in the statistical and comparative analysis of functional image data     

Personal content management system: A semantic approach

The amount of multimedia resources that is created and needs to be managed is increasing considerably. Additionally, a significant increase of metadata, either structured (metadata fields of standardized metadata formats) or unstructured (free tagging or annotations) is noticed. This increasing amount of data and metadata, combined with the substantial diversity in terms of used metadata fields and constructs, results in severe problems to manage and retrieve these multimedia resources. Standardized metadata schemes can be used but the plethora of these schemes results in interoperability issues. In this paper, we propose a metadata model suited for personal content management systems. We create a layered metadata service that implements the presented model as an upper layer and combines different metadata schemes in the lower layers. Semantic web technologies are used to define and link formal representations of these schemes. Specifically, we create an ontology for the DIG35 metadata standard and elaborate on how it is used within this metadata service. To illustrate the service, we present a representative use case scenario consisting of the upload, annotation, and retrieval of multimedia content within a personal content management system.     

Multimedia Retrieval and Delivery: Essential Metadata Challenges and Standards

Multimedia information retrieval (MIR) and delivery plays an important role in many application domains due to the increasing need to identify, filter, and manage growing amounts of data, notably multimedia information. To efficiently manage and exchange multimedia information, interoperability between coded data and metadata is required and standardization is central to achieving the necessary level of interoperability. In the context of this paper, the term retrieval refers to the process by which a user, human or machine, identifies the content it needs, and the term delivery refers to the adaptive transport and consumption of the identified content in a particular context or usage environment. Both the retrieval and delivery processes may require content and context metadata. This paper will argue that maximum quality of experience depends not only on the content itself (and thus content metadata) but also on the consumption conditions (thus context metadata). Additionally, the rights and protection conditions have become critically important in recent years, especially with the explosion of electronic music commerce and different ldquoshoppingrdquo conditions. This paper will review existing multimedia standards related to information retrieval and adaptive delivery of multimedia content, emphasizing the need for such standards, and will show how these standards can help the development, dissemination, and valorization of MIR research results. Moreover, it will also discuss limitations of the current standards and anticipate what future standardization activities are relevant and needed. Due to space limitations, the paper will mainly concentrate on MPEG standards although many other relevant standards are also reviewed and discussed.     

基于内容的视频检索系统的比较和研究

分析了数字视频的特点和优势,阐述了基于内容的视频检索的迫切性和重要性。比较分析了4个典型检索系统,归纳了其系统结构、功能和应用领域等,指出存在的主要问题,并提出了解决方案。分析了研究的热点和方向。     

Visual Computing Laboratory

It would be difficult to overestimate the importance of visual information in current computer systems. Visual computing, which embraces processing, interpreting, modeling, assimilating, storing, retrieving, and synthesizing visual information, now plays a crucial role in many fields. These include multimedia, virtual reality, robotics, scientific visualization, and communications systems. And the demand for further integration of visual information into these areas shows every sign of continuing unabated. Under the direction of Ramesh Jain, the Visual Computing Laboratory at the University of California, San Diego, was established as a center for innovative visual computing research to address the requirements of these applications in next-generation computer technologies. As such, the Visual Computing Lab hosts a group of researchers working in a variety of areas, notably multimedia databases, information assimilation, interactive video, and visual interaction through gesture recognition. This article presents a high-level overview of activities in the Visual Computing Laboratory and provides some details on prototype systems that we are currently developing     

面向跨模态检索的音频数据库内容匹配方法研究

跨模态检索旨在通过以某一模态的数据为查询词，使人们能够得到与之相关的其他不同模态数据的检索结果的新型检索方法，这已成为多媒体和信息检索领域中一个有趣的研究问题。但是，目前大多数的研究成果集中于文本到图像、文本到视频以及歌词到音频等跨模态相关任务上，而关于如何为特定的视频通过跨模态检索得到合适的音乐这一跨模态的相关研究却很有限。此外，大多现有的关于视频和音频跨模态的研究依赖于元数据（例如关键字，标签或描述）。本文介绍了一种基于音频和视频这两种模态数据内容的跨模态检索的方法，该方法以新型的双流处理网络为框架，并通过神经网络学习两模态数据在公共子空间的特征表达，以计算音频和视频数据之间的相似度。本文所提出的方法的创新点主要在以下三个方面:1）在原有的提取各模态特征的模型基础上引入注意力机制，以此得到了视频和音频的特征选择模型，并筛选出相应的特征表达。2）使用了样本挖掘机制，剔除了无效样本，使得数据的训练更加高效。3）从计算模态间相似性和保持模态内结构不变两方面出发，设计了相应的损失函数进行模型的训练。且所提出的模型在VEGAS数据集和自建数据集上都取得了较高的准确度。      

Next-generation content representation, creation, and searching fornew-media applications in education

Content creation, editing, and searching are extremely time-consuming tasks that often require substantial training and experience, especially when high-quality audio and video are involved. New media represents a new paradigm for multimedia information representation and processing, in which the emphasis is placed on the actual content. It thus brings the tasks of content creation and searching much closer to actual users and enables them to be active producers of audio-visual information rather than passive recipients. We discuss the state of the art and present next-generation techniques for content representation, searching, creation and editing. We discuss our experiences in developing a Web-based distributed compressed video editing and searching system (WebClip), a media-representation language (Flavor) and an object-based video authoring system (Zest) based on it, and a large image/video search engine for the World Wide Web (WebSEEk). We also present a case study of new media applications based on specific planned multimedia education experiments with the above systems in several K-12 schools in Manhattan, NY     

VideOlympics: Real-Time Evaluation of Multimedia Retrieval Systems

The first VideOlympics brings content-based analysis to the archive and allows for many-to- many communication between video search engines and their audience It was a great Success. The VideOlympics provided the excitement of a competition without the associated stress on the participants. For the first time, the audience was able to compare different multimedia retrieval systems on the same tasks and see how they performed with unrehearsed topics. Many audience members felt they understood the technology's capabilities after seeing it in live action and in several system variations.     

Content based video indexing and retrieval

Video management tools and techniques are based on pixels rather than perceived content. Thus, state-of-the-art video editing systems can easily manipulate such things as time codes and image frames, but they cannot “know,” for example, what a basketball is. Our research addresses four areas of content-based video management     

一种基于内容的数字视频典型帧提取方法

典型帧提取是多媒体视频检索的一个重要技术分支。本文提出一种基于内容的数字视频典型帧提取方法，并通过实验对算法的有效性作了验证。     

ViewStation applications: implications for network traffic

This paper describes applications built on the ViewStation, a distributed multimedia system based on Unix workstations and a gigabit per second local area network. A key tenet of the ViewStation project is the delivery of media data not just to the desktop but all the way to the application program. As processing power continues to improve, our approach enables applications that perform intensive processing of audio and video data. We hypothesize that as media data are shaped by this software-based processing, the resultant network traffic patterns will be dominated more by software behavior than by so-called real-time issues. We have written applications that directly process live video to provide more responsive human-computer interaction. We have also developed applications to explore the potential of media processing to support content-based retrieval of prerecorded television broadcasts. These applications perform intelligent processing on video, as well as straightforward presentation. They demonstrate the utility of network-based multimedia systems that deliver audio and video data all the way to the application. The network requirements of the applications are modeled as a combination of bursty transfers and periodic packet-trains