多媒体结构的设计与选择

构造一个上下文敏感,具有可交互性的多媒体表示远比单纯构造视听信息或者单纯构造文本信息复杂得多.多媒体对象之间的交互需要有一个用来同步和异步交互的,包括立即响应的时空关系组织起来.根据构造环境的需要来考虑多媒体的结构构造.先通过介绍影响多媒体结构构造的7个要素来认识多媒体,然后在4种典型的构造模式中进行多媒体环境的选择.这4种构造模式分别是:结构化、时间关系、图关系、脚本形式.通过各种各样的观察和分析发现,结构化的多媒体构造为多媒体的表示提供了最为有效的表示形式.     

Eventor: an authoring system for interactive multimedia applications

Programming, testing, and maintaining interactive multimedia applications (IMAs) are still difficult and expensive, while substantial progress has been made to reduce the burden on authors. As IMAs get larger and more complex the difficulties will increase. To overcome the complexity of such IMAs, we argue that authoring systems should provide such facilities as (1) a traditional and intuitivedivide-and-conquer paradigm for solving large and complex problems in various fields, (2)formal specification of the behaviors of IMAs for checking the syntactic correctness of visual expressions or semantic anomalies, and (3)automatic aids like validation of temporal constraints and verification of visual expressions. In this paper, we investigate the properties of IMAs for recognizing the inherent interactivity and concurrency. We propose a specification method based on Milner'sCalculus of Communicating Systems (CCS), which is a well-known formal mechanism for specifying the concurrency in various distributed applications. We also design and implement an authoring system calledEventor (Event Editor), which is based on CCS and composed of three tools: a Temporal Synchronizer, a Spatial Synchronizer, and a User Interaction Builder. They focus on describing the temporal and spatial synchronizations and user interactions while they rely on existing tools in Intel's Digital Video Interactive (DVI) for supporting other functionalities. By editing a simple computer aided instruction (CAI) application, we illustrate that our specification mechanism is well-suited for handling the interactivity of multimedia applications, and Eventor is a simple, efficient, and powerful enough tool to handle practical applications. Especially the incremental refinement and the formal specification based on the CCS allow Eventor to be extended with formal verifications to cope with large and complex applications.     

Context aware guidance for multimedia authoring: harmonizing domain and discourse knowledge

This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend the existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype.     

Interactive authoring of multimedia documents in a constraint-based authoring system

As multimedia applications spread widely, it is crucial for programming and design support systems to handle “time” in multimedia documents effectively and flexibly. This paper presents a set of interactive system support tools for designing and maintaining the temporal behavior of multimedia documents. The tool set provides mechanisms for anomaly detection, temporal query processing, and interactive scheduling. It is based on a fast incremental constraint solver we have developed, which can be adapted by any constraint-based system. The incremental constraint solver provides immediate feedback to the user, supporting a highly interactive design process. Combined with existing optimal layout generation mechanisms proposed in the literature, our tools effectively utilize the flexibility provided by constraint-based systems.     

Assessment and multimedia authoring – a tool for externalising understanding

Abstract Multimedia authoring software has a number of features that, given teacher support, should provide conditions that help pupils create material that genuinely reflects their understanding. This paper reviews the ways in which use of this technology may also support improvements in assessment. Drawing on research findings it describes how pupil-produced multimedia documents can reveal aspects of performance that traditional tests may not recognise. If such assessment is to be accepted at the policy level, a shared set of criteria for achievement must be developed in terms of technical manipulation and content exposition. Lines of enquiry for further investigation of multimedia authoring, pupil learning and assessment are proposed.     

Content-based management of video in a multimedia authoring environment

This paper describes architecture and implementation of a multimedia authoring environment which features content-based management of video.The goal of our work is to provide multimedia application developers with a useful video database, where they gather video materials from various sources, store them in a organized format, and use them in multimedia applications. The principal design concepts are that (1) it should have a flexible means to store video materials in a organized format so that video clips can be retrieved easily, and that (2) it should have a view to each application so that various applications can share the video materials.To meet these requirements we designed a multimedia authoring system based on a video object model. The video object model features (1) a method to attach annotations not only to scenes but also to things shot in the video, and (2) a method to define video clips independently from physical video data.Our experiences in using the system to develop a multimedia CAI application are also described to show the advantages of our system.     

基于神经网络的多媒体通信控制机制研究

在Internet网络的多媒体通信系统中需要解决QoS控制问题,如视频、音频等多媒体数据的同步、网络拥塞控制、多媒体数据传输的QoS协商控制、视频平滑,以及连续多媒体系统的CPU调度等.为了解决好这些控制问题,提出一种基于神经网络的多媒体通信控制机制,把人工智能与多媒体通信技术紧密结合起来,并在Internet网络环境下开发了实用的多媒体通信系统.运行结果表明,该系统效果优越.     

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: the IMMPS project, the PreGen system, and the StrMP system

This paper contributes to a Standard Reference Model (RM) for intelligent multimedia presentation systems. In the past few years, we have developed several intelligent presentation systems at the Multimedia Information NEtwork (MINE) Lab at Tamkang University. We describe these systems in terms of architecture and function of components. The first system, Intelligent MultiMedia Presentation System (IMMPS), allows a presentation designer to construct knowledge rules reflecting the background of individual audiences. Therefore, the generated presentation is able to learn from the audiences, which results in different presentation for each individual. The second system (PreGen) uses inference rules for the automatic generation of multimedia presentations. An ICON programming technique is also used in the PreGen project. Another system for designing structured multimedia presentations using data flow/control flow diagrams is also discussed. In order to run multimedia presentations generated by different systems, we are developing a multimedia abstract machine, which is a time Petri net based software simulator for running multimedia assembly instructions.     

Putting AI in entertainment: an AI authoring tool for simulation and games

We have been working on techniques to write better AI behavior for simulations and games. We are focusing on a visual authoring tool that provides a way to quickly synthesize complex behavior, and are building a corresponding AI engine to run with a simulation or game. For game development uses, we see our tool as making the AI understandable to game designers and end users, as well as improving developers' productivity. This work will also be useful for simulation developers and subsequently for analysts, operators, and instructors. We explain some of our tool's features and how the AI engine processes the resulting content.     

MediaTouch: A Native Authoring Tool for MHEG-5 Applications

MediaTouch is a visual based authoring tool for creating MHEG-5 applications. It is based on the native approach, which enables the author to operate at the level of MHEG-5 objects, in order to get the most out of the standard's features. Specialized editing functions facilitate the author's work, especially when setting properties such as the hierarchical structure of an application, dynamic behavior and appearance of objects. The paper introduces the problem of authoring multimedia applications, motivating the adoption of a standard solution, and explaining the functionality of MediaTouch, and in so doing, providing an example of use.     

多媒体智能：当多媒体遇到人工智能

过去10年中涌现出大量新兴的多媒体应用和服务,带来了很多可以用于多媒体前沿研究的多媒体数据。多媒体研究在图像/视频内容分析、多媒体搜索和推荐、流媒体服务和多媒体内容分发等方向均取得了重要进展。与此同时,由于在深度学习领域所取得的重大突破,人工智能(artificial intelligence,AI)在20世纪50年代被正式视为一门学科之后,迎来了一次“新”的发展浪潮。因此,一个问题就自然而然地出现了：当多媒体遇到人工智能时会带来什么？为了回答这个问题,本文通过研究多媒体和人工智能之间的相互影响引入了多媒体智能的概念。从两个方面探讨多媒体与人工智能之间的相互影响：一是多媒体促使人工智能向着更具可解释性的方向发展;二是人工智能反过来为多媒体研究注入了新的思维方式。这两个方面形成了一个良性循环,多媒体和人工智能在其中不断促进彼此发展。本文对相关研究及进展进行了讨论,并围绕值得进一步探索的研究方向分享见解。希望可以对多媒体智能的未来发展带来新的研究思路。     

mwKAT: A meta-tool-generated knowledge acquisition tool for multimedia workstations development

mwKAT is an interactive knowledge acquisition tool for acquiring domain knowledge about multimedia components. It constructs knowledge bases for a consulting system that produces the design specification for a multimedia workstation according to the user requirements.mwKAT is generated from and executed inGAS, a primitives-based generic knowledge acquisition meta-tool. It contains three acquisition primitives, namely, parameter proposing, constraint proposing, and fix proposing to construct an intermediate knowledge base represented by a dependency model. These primitives identify necessary domain knowledge and guide users to propose significant components, constraints, and fix methods into the dependency model.mwKAT also invokes knowledge verification and validation primitives to verify the completeness, consistency, compilability, and correctness of the intermediate knowledge base.     

视频处理与压缩技术

视频处理与压缩是多媒体计算与通信领域的核心主题之一,是连接视频采集传输和视觉分析理解的关键桥梁,也是诸多视频应用的基础。当前“5G+超高清+AI”正在引发多媒体计算与通信领域的新一轮重大技术革新,视频处理与压缩技术正在发生深刻变革,亟需建立视频大数据高效紧凑表示理论和方法。为此,学术研究机构和工业界对视频大数据的视觉表示机理、视觉信息紧凑表达、视频信号重建与恢复、高层与低层视觉融合处理方法及相应硬件技术等前沿领域进行了广泛深入研究。本文从数字信号处理基础理论出发,分析了当前视频处理与压缩领域的热点问题和研究内容,包括基于统计先验模型的视频数据表示模型及处理方法、融合深度网络模型的视频处理技术、视频压缩技术以及视频压缩标准进展等领域。详细描述了视频超分辨率、视频重建与恢复、视频压缩技术等领域面临的前沿动态、发展趋势、技术瓶颈和标准化进程等内容,对国际国内研究内容和发展现状进行了综合对比与分析,并展望了视频处理与压缩技术的发展与演进方向。更高质量视觉效果和高效率视觉表达之间将不再是单独研究的个体,融合类脑视觉系统及编码机理的视频处理与压缩技术将是未来研究的重要领域之一。     

Interactive story authoring: A viable form of creative expression for the classroom

The unprecedented growth in numbers of children playing computer games has stimulated discussion and research regarding what, if any, educational value these games have for teaching and learning. The research on this topic has primarily focused on children as players of computer games rather than builders/constructors of computer games. Recently, several game companies, such as BioWare Corp. and Bethesda Softworks, have released game story creation tools to the public, along with their games. However, a major obstacle to using these commercial tools is the level of programming experience required to create interactive game stories. In this paper, we demonstrate that a commercial game story construction tool, BioWare Corp.’s Aurora Toolset, can be augmented by our new tool, ScriptEase, to enable students in two grade ten English classes to successfully construct interactive game stories. We present evidence that describes the relationship between interactive story authoring and traditional story authoring, along with a series of factors that can potentially affect success at these activities: gender, creativity, intellectual ability, previous experiences with programming, time playing computer games, and time spent online. Results indicate that students can successfully construct sophisticated interactive stories with very little training. The results also show no gender differences in the quality of these interactive stories, regardless of programming experience or the amount of time per week playing computer games or participating in general online activities, although a subset of female students did show a slightly higher level of performance on interactive story authoring. In the educational context of this study, we show that ScriptEase provides an easy-to-use tool for interactive story authoring in a constructionist learning environment.     

Optimum adaptation trajectories for streamed multimedia

Most adaptive delivery mechanisms for streaming multimedia content do not explicitly consider user-perceived quality when making adaptations. We propose that an optimal adaptation trajectory through the set of possible encodings exists and that it indicates how to adapt encoding quality in response to changes in network conditions to maximize user-perceived quality. Such an optimum adaptation trajectory can be used with any transmission adaptation policy. We describe the subjective tests we carried out to find such trajectories for a number of different MPEG-4 video clips and indicate how this knowledge could be used in the operation of a practical system.     

AI paradigms

The aim of this paper is comparative analysis of most important AI paradigms. An AI paradigm is defined as the pair composed by a concept of intelligence and a methodology in which intelligent computer systems are developed and operated. Three paradigms, the behaviourist paradigm, the agent paradigm, and the artificial life paradigm are discussed.     

The foundations of a theory-aware authoring tool for CSCL design

One of the most useful ways to enhance collaboration is to create scenarios where learners are able to interact more effectively. Nevertheless, the design of pedagogically sound and well-thought-out collaborative learning scenarios is a complex issue. This is due to the context of group learning where the synergy among learners’ interactions affects learning processes and, hence, the learning outcome. Although many advances have been made to support the designing of collaborative learning scenarios through technology, a more systematic approach is lacking. With the limitations of the current designing methods and tools, it is difficult to develop intelligent authoring systems that can guide users in order to produce more effective collaboration. One of the main difficulties with creating a more consistent (computer-understandable) approach to designing collaboration is the necessity of proposing better ways to formalize the group learning processes. In this paper, we present an innovative approach that uses ontologies and concepts from learning theories to create a framework that represents collaborative learning and its processes. Ontologies provide the necessary formalization to represent collaboration, while learning theories provide the concepts to justify and support the development of effective learning scenarios. Such an approach contributes to establish the foundations for the development of the next generation of intelligent authoring systems referred to as theory-aware systems. To verify the viability and usefulness of our proposed ontological framework in the context of systematic design, the development and use of an intelligent authoring tool for CSCL design is presented. This system is able to reason on ontologies to give suggestions that help users to create theory-compliant collaborative learning scenarios. We carried out several experiments with teachers in a geometry drawing course and the results indicate that the system helps teachers to create and interchange their scenarios more easily and facilitates the selection of important pedagogical strategies that influence positively the designing and effectiveness of group activities.     

Bringing wireless video into networked multimedia systems

This paper describes the main issues regarding the development of a portable system capable of making wireless connections to a TCP/IP network for real-time video delivery. Compression is used along with packet radio technology, providing a fully mobile and compact device. It is expected that using such a device, one camera-person will be able to make a report with his/her camera attached to a small battery-powered box which will transmit NTSC-quality digitalized and compressed images to a central gateway connected to the Internet. Some uses for the system are discussed, one of them being an application aimed to control combustion chambers in a petrol refinery.     

Parallel architectures for AI semantic network processing

Artificial intelligence (AI) applications are growing in several fields, and in many such applications knowledge bases must be manipulated. This activity is usually performed by an external agent such as a central processor, but often this cannot supply the speed required. Knowledge-oriented architectures provide an efficient execution of knowledge manipulations. This paper provides an introduction to a particular subset of knowledge-oriented architectures, the semantic network approach, which is one of the most commonly used methods of representing and manipulating knowledge in the AI field. A brief overview of the semantic network components is presented in order to provide a background to the topic. The purpose of this paper is to review the proposed, implemented and/or simulated architectures for semantic network processing, and to discuss the capabilities and limitations of such architectures.     

Urban Tapestries: Wireless networking, public authoring and social knowledge

Urban Tapestries is an action research project combining social research interaction design and cultural production. The project is developing a prototype system for accessing and publishing location specific content wirelessly, and creating sound maps of environments and journeys that can be shared remotely.