多媒体结构的设计与选择

构造一个上下文敏感,具有可交互性的多媒体表示远比单纯构造视听信息或者单纯构造文本信息复杂得多.多媒体对象之间的交互需要有一个用来同步和异步交互的,包括立即响应的时空关系组织起来.根据构造环境的需要来考虑多媒体的结构构造.先通过介绍影响多媒体结构构造的7个要素来认识多媒体,然后在4种典型的构造模式中进行多媒体环境的选择.这4种构造模式分别是:结构化、时间关系、图关系、脚本形式.通过各种各样的观察和分析发现,结构化的多媒体构造为多媒体的表示提供了最为有效的表示形式.     

Eventor: an authoring system for interactive multimedia applications

Programming, testing, and maintaining interactive multimedia applications (IMAs) are still difficult and expensive, while substantial progress has been made to reduce the burden on authors. As IMAs get larger and more complex the difficulties will increase. To overcome the complexity of such IMAs, we argue that authoring systems should provide such facilities as (1) a traditional and intuitivedivide-and-conquer paradigm for solving large and complex problems in various fields, (2)formal specification of the behaviors of IMAs for checking the syntactic correctness of visual expressions or semantic anomalies, and (3)automatic aids like validation of temporal constraints and verification of visual expressions. In this paper, we investigate the properties of IMAs for recognizing the inherent interactivity and concurrency. We propose a specification method based on Milner'sCalculus of Communicating Systems (CCS), which is a well-known formal mechanism for specifying the concurrency in various distributed applications. We also design and implement an authoring system calledEventor (Event Editor), which is based on CCS and composed of three tools: a Temporal Synchronizer, a Spatial Synchronizer, and a User Interaction Builder. They focus on describing the temporal and spatial synchronizations and user interactions while they rely on existing tools in Intel's Digital Video Interactive (DVI) for supporting other functionalities. By editing a simple computer aided instruction (CAI) application, we illustrate that our specification mechanism is well-suited for handling the interactivity of multimedia applications, and Eventor is a simple, efficient, and powerful enough tool to handle practical applications. Especially the incremental refinement and the formal specification based on the CCS allow Eventor to be extended with formal verifications to cope with large and complex applications.     

多媒体智能：当多媒体遇到人工智能

过去10年中涌现出大量新兴的多媒体应用和服务,带来了很多可以用于多媒体前沿研究的多媒体数据。多媒体研究在图像/视频内容分析、多媒体搜索和推荐、流媒体服务和多媒体内容分发等方向均取得了重要进展。与此同时,由于在深度学习领域所取得的重大突破,人工智能(artificial intelligence,AI)在20世纪50年代被正式视为一门学科之后,迎来了一次“新”的发展浪潮。因此,一个问题就自然而然地出现了：当多媒体遇到人工智能时会带来什么？为了回答这个问题,本文通过研究多媒体和人工智能之间的相互影响引入了多媒体智能的概念。从两个方面探讨多媒体与人工智能之间的相互影响：一是多媒体促使人工智能向着更具可解释性的方向发展;二是人工智能反过来为多媒体研究注入了新的思维方式。这两个方面形成了一个良性循环,多媒体和人工智能在其中不断促进彼此发展。本文对相关研究及进展进行了讨论,并围绕值得进一步探索的研究方向分享见解。希望可以对多媒体智能的未来发展带来新的研究思路。     

Context aware guidance for multimedia authoring: harmonizing domain and discourse knowledge

This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend the existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype.     

User interfaces for semantic authoring of textual content: A systematic literature review

Practical approaches for managing and supporting the life-cycle of semantic content on the Web of Data have recently made quite some progress. In particular in the area of the user-friendly manual and semi-automatic creation of rich semantic content we have observed recently a large number of approaches and systems being described in the literature. With this survey we aim to provide an overview on the rapidly emerging field of Semantic Content Authoring (SCA). We conducted a systematic literature review comprising a thorough analysis of 31 primary studies out of 175 initially retrieved papers addressing the semantic authoring of textual content. We obtained a comprehensive set of quality attributes for SCA systems together with corresponding user interface features suggested for their realization. The quality attributes include aspects such as usability, automation, generalizability, collaboration, customizability and evolvability. The primary studies were surveyed in the light of these quality attributes and we performed a thorough analysis of four SCA systems. The proposed quality attributes and UI features facilitate the evaluation of existing approaches and the development of novel more effective and intuitive semantic authoring interfaces.     

Assessment and multimedia authoring – a tool for externalising understanding

Abstract Multimedia authoring software has a number of features that, given teacher support, should provide conditions that help pupils create material that genuinely reflects their understanding. This paper reviews the ways in which use of this technology may also support improvements in assessment. Drawing on research findings it describes how pupil-produced multimedia documents can reveal aspects of performance that traditional tests may not recognise. If such assessment is to be accepted at the policy level, a shared set of criteria for achievement must be developed in terms of technical manipulation and content exposition. Lines of enquiry for further investigation of multimedia authoring, pupil learning and assessment are proposed.     

Interactive authoring of multimedia documents in a constraint-based authoring system

As multimedia applications spread widely, it is crucial for programming and design support systems to handle “time” in multimedia documents effectively and flexibly. This paper presents a set of interactive system support tools for designing and maintaining the temporal behavior of multimedia documents. The tool set provides mechanisms for anomaly detection, temporal query processing, and interactive scheduling. It is based on a fast incremental constraint solver we have developed, which can be adapted by any constraint-based system. The incremental constraint solver provides immediate feedback to the user, supporting a highly interactive design process. Combined with existing optimal layout generation mechanisms proposed in the literature, our tools effectively utilize the flexibility provided by constraint-based systems.     

Content-based management of video in a multimedia authoring environment

This paper describes architecture and implementation of a multimedia authoring environment which features content-based management of video.The goal of our work is to provide multimedia application developers with a useful video database, where they gather video materials from various sources, store them in a organized format, and use them in multimedia applications. The principal design concepts are that (1) it should have a flexible means to store video materials in a organized format so that video clips can be retrieved easily, and that (2) it should have a view to each application so that various applications can share the video materials.To meet these requirements we designed a multimedia authoring system based on a video object model. The video object model features (1) a method to attach annotations not only to scenes but also to things shot in the video, and (2) a method to define video clips independently from physical video data.Our experiences in using the system to develop a multimedia CAI application are also described to show the advantages of our system.     

HyperTree: a structural approach to web authoring

The expansion of the World Wide Web (WWW) has created an increasing need for tools capable of supporting WWW authors in composing documents using the HyperText Markup Language (HTML). Currently, most web authors use tools which are basically ordinary text editors and have additional features to facilitate the easy and correct use of HTML tags. This approach places the burden on the web author to design and then create the entire web site in a top-down fashion, without any explicit support for the structural design of the site. In this paper we discuss an alternative structural approach to Web authoring, which is based on the use of the HyperTree hypermedia system as the central authoring tool. The advantages of using HyperTree are two-dimensional. Firstly, web authors can manage a web site as a single complete hypermedia database. For example, HyperTree provides facilities like the automatic creation of indices and the discovery of link inconsistencies. Additionally, it organizes the web pages in an easy to understand hierarchy without using any HTML directly. Secondly, web end-users can benefit from the use of HyperTree, since seeking information in structured web sites is generally less disorientating and develops fewer cognitive overheads. ©1997 John Wiley & Sons, Ltd.     

基于神经网络的多媒体通信控制机制研究

在Internet网络的多媒体通信系统中需要解决QoS控制问题,如视频、音频等多媒体数据的同步、网络拥塞控制、多媒体数据传输的QoS协商控制、视频平滑,以及连续多媒体系统的CPU调度等.为了解决好这些控制问题,提出一种基于神经网络的多媒体通信控制机制,把人工智能与多媒体通信技术紧密结合起来,并在Internet网络环境下开发了实用的多媒体通信系统.运行结果表明,该系统效果优越.     

A hypermedia authoring tool for augmented and virtual reality

Most existing hypermedia authoring systems are intended for use on desktop computers. These systems are typically designed for the creation of 2D documents and therefore employ 2D authoring mechanisms. In contrast, authoring systems for nontraditional multimedia/hypermedia experiences for 3D virtual or augmented worlds focus mainly on creating separate media objects and embedding them within the user's surroundings. As a result, linking these media objects to create 3D hypermedia is a tedious manual task. To address this issue, we present an authoring tool for creating and editing linked 3D hypermedia narratives that are interwoven with a wearable computer user's surrounding environment. Our system is designed for use by authors who are not programmers, and allows them to preview their results on a desktop workstation, as well as with an augmented or virtual reality system.     

Putting AI in entertainment: an AI authoring tool for simulation and games

We have been working on techniques to write better AI behavior for simulations and games. We are focusing on a visual authoring tool that provides a way to quickly synthesize complex behavior, and are building a corresponding AI engine to run with a simulation or game. For game development uses, we see our tool as making the AI understandable to game designers and end users, as well as improving developers' productivity. This work will also be useful for simulation developers and subsequently for analysts, operators, and instructors. We explain some of our tool's features and how the AI engine processes the resulting content.     

mwKAT: A meta-tool-generated knowledge acquisition tool for multimedia workstations development

mwKAT is an interactive knowledge acquisition tool for acquiring domain knowledge about multimedia components. It constructs knowledge bases for a consulting system that produces the design specification for a multimedia workstation according to the user requirements.mwKAT is generated from and executed inGAS, a primitives-based generic knowledge acquisition meta-tool. It contains three acquisition primitives, namely, parameter proposing, constraint proposing, and fix proposing to construct an intermediate knowledge base represented by a dependency model. These primitives identify necessary domain knowledge and guide users to propose significant components, constraints, and fix methods into the dependency model.mwKAT also invokes knowledge verification and validation primitives to verify the completeness, consistency, compilability, and correctness of the intermediate knowledge base.     

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: the IMMPS project, the PreGen system, and the StrMP system

This paper contributes to a Standard Reference Model (RM) for intelligent multimedia presentation systems. In the past few years, we have developed several intelligent presentation systems at the Multimedia Information NEtwork (MINE) Lab at Tamkang University. We describe these systems in terms of architecture and function of components. The first system, Intelligent MultiMedia Presentation System (IMMPS), allows a presentation designer to construct knowledge rules reflecting the background of individual audiences. Therefore, the generated presentation is able to learn from the audiences, which results in different presentation for each individual. The second system (PreGen) uses inference rules for the automatic generation of multimedia presentations. An ICON programming technique is also used in the PreGen project. Another system for designing structured multimedia presentations using data flow/control flow diagrams is also discussed. In order to run multimedia presentations generated by different systems, we are developing a multimedia abstract machine, which is a time Petri net based software simulator for running multimedia assembly instructions.     

Interactive story authoring: A viable form of creative expression for the classroom

The unprecedented growth in numbers of children playing computer games has stimulated discussion and research regarding what, if any, educational value these games have for teaching and learning. The research on this topic has primarily focused on children as players of computer games rather than builders/constructors of computer games. Recently, several game companies, such as BioWare Corp. and Bethesda Softworks, have released game story creation tools to the public, along with their games. However, a major obstacle to using these commercial tools is the level of programming experience required to create interactive game stories. In this paper, we demonstrate that a commercial game story construction tool, BioWare Corp.’s Aurora Toolset, can be augmented by our new tool, ScriptEase, to enable students in two grade ten English classes to successfully construct interactive game stories. We present evidence that describes the relationship between interactive story authoring and traditional story authoring, along with a series of factors that can potentially affect success at these activities: gender, creativity, intellectual ability, previous experiences with programming, time playing computer games, and time spent online. Results indicate that students can successfully construct sophisticated interactive stories with very little training. The results also show no gender differences in the quality of these interactive stories, regardless of programming experience or the amount of time per week playing computer games or participating in general online activities, although a subset of female students did show a slightly higher level of performance on interactive story authoring. In the educational context of this study, we show that ScriptEase provides an easy-to-use tool for interactive story authoring in a constructionist learning environment.     

视频处理与压缩技术

视频处理与压缩是多媒体计算与通信领域的核心主题之一,是连接视频采集传输和视觉分析理解的关键桥梁,也是诸多视频应用的基础。当前“5G+超高清+AI”正在引发多媒体计算与通信领域的新一轮重大技术革新,视频处理与压缩技术正在发生深刻变革,亟需建立视频大数据高效紧凑表示理论和方法。为此,学术研究机构和工业界对视频大数据的视觉表示机理、视觉信息紧凑表达、视频信号重建与恢复、高层与低层视觉融合处理方法及相应硬件技术等前沿领域进行了广泛深入研究。本文从数字信号处理基础理论出发,分析了当前视频处理与压缩领域的热点问题和研究内容,包括基于统计先验模型的视频数据表示模型及处理方法、融合深度网络模型的视频处理技术、视频压缩技术以及视频压缩标准进展等领域。详细描述了视频超分辨率、视频重建与恢复、视频压缩技术等领域面临的前沿动态、发展趋势、技术瓶颈和标准化进程等内容,对国际国内研究内容和发展现状进行了综合对比与分析,并展望了视频处理与压缩技术的发展与演进方向。更高质量视觉效果和高效率视觉表达之间将不再是单独研究的个体,融合类脑视觉系统及编码机理的视频处理与压缩技术将是未来研究的重要领域之一。     

Optimum adaptation trajectories for streamed multimedia

Most adaptive delivery mechanisms for streaming multimedia content do not explicitly consider user-perceived quality when making adaptations. We propose that an optimal adaptation trajectory through the set of possible encodings exists and that it indicates how to adapt encoding quality in response to changes in network conditions to maximize user-perceived quality. Such an optimum adaptation trajectory can be used with any transmission adaptation policy. We describe the subjective tests we carried out to find such trajectories for a number of different MPEG-4 video clips and indicate how this knowledge could be used in the operation of a practical system.     

MediaTouch: A Native Authoring Tool for MHEG-5 Applications

MediaTouch is a visual based authoring tool for creating MHEG-5 applications. It is based on the native approach, which enables the author to operate at the level of MHEG-5 objects, in order to get the most out of the standard's features. Specialized editing functions facilitate the author's work, especially when setting properties such as the hierarchical structure of an application, dynamic behavior and appearance of objects. The paper introduces the problem of authoring multimedia applications, motivating the adoption of a standard solution, and explaining the functionality of MediaTouch, and in so doing, providing an example of use.     

AI paradigms

The aim of this paper is comparative analysis of most important AI paradigms. An AI paradigm is defined as the pair composed by a concept of intelligence and a methodology in which intelligent computer systems are developed and operated. Three paradigms, the behaviourist paradigm, the agent paradigm, and the artificial life paradigm are discussed.     

Towards a consolidated model for a collaborative courseware authoring system

Abstract In an effort to reduce the expensive resource requirements in courseware production and to allow effective management of, and communication and collaboration between authors, a model is needed to harmonize the diverse theoretical backgrounds such a process requires. To this end we look at five abstract models which where chosen for their apparent appropriateness within the general area of collaborative authoring and reuse of multimedia courseware. These models consist of a functional model of hypertext, an infrastructure model for an open collaborative authoring system, two object-oriented communication models, and a courseware production model. We attempt a consolidation of these models and ultimately make several conclusions, which are exposed during a conceptual harmonisation. The principal conclusion is that a hypermedia logical model is not by itself adequate to support collaboration and reuse. Our modeling conclusions have driven the design of a system which will be implemented and tested.