Eventor: an authoring system for interactive multimedia applications

Programming, testing, and maintaining interactive multimedia applications (IMAs) are still difficult and expensive, while substantial progress has been made to reduce the burden on authors. As IMAs get larger and more complex the difficulties will increase. To overcome the complexity of such IMAs, we argue that authoring systems should provide such facilities as (1) a traditional and intuitivedivide-and-conquer paradigm for solving large and complex problems in various fields, (2)formal specification of the behaviors of IMAs for checking the syntactic correctness of visual expressions or semantic anomalies, and (3)automatic aids like validation of temporal constraints and verification of visual expressions. In this paper, we investigate the properties of IMAs for recognizing the inherent interactivity and concurrency. We propose a specification method based on Milner'sCalculus of Communicating Systems (CCS), which is a well-known formal mechanism for specifying the concurrency in various distributed applications. We also design and implement an authoring system calledEventor (Event Editor), which is based on CCS and composed of three tools: a Temporal Synchronizer, a Spatial Synchronizer, and a User Interaction Builder. They focus on describing the temporal and spatial synchronizations and user interactions while they rely on existing tools in Intel's Digital Video Interactive (DVI) for supporting other functionalities. By editing a simple computer aided instruction (CAI) application, we illustrate that our specification mechanism is well-suited for handling the interactivity of multimedia applications, and Eventor is a simple, efficient, and powerful enough tool to handle practical applications. Especially the incremental refinement and the formal specification based on the CCS allow Eventor to be extended with formal verifications to cope with large and complex applications.     

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: the IMMPS project, the PreGen system, and the StrMP system

This paper contributes to a Standard Reference Model (RM) for intelligent multimedia presentation systems. In the past few years, we have developed several intelligent presentation systems at the Multimedia Information NEtwork (MINE) Lab at Tamkang University. We describe these systems in terms of architecture and function of components. The first system, Intelligent MultiMedia Presentation System (IMMPS), allows a presentation designer to construct knowledge rules reflecting the background of individual audiences. Therefore, the generated presentation is able to learn from the audiences, which results in different presentation for each individual. The second system (PreGen) uses inference rules for the automatic generation of multimedia presentations. An ICON programming technique is also used in the PreGen project. Another system for designing structured multimedia presentations using data flow/control flow diagrams is also discussed. In order to run multimedia presentations generated by different systems, we are developing a multimedia abstract machine, which is a time Petri net based software simulator for running multimedia assembly instructions.     

Assessment and multimedia authoring – a tool for externalising understanding

Abstract Multimedia authoring software has a number of features that, given teacher support, should provide conditions that help pupils create material that genuinely reflects their understanding. This paper reviews the ways in which use of this technology may also support improvements in assessment. Drawing on research findings it describes how pupil-produced multimedia documents can reveal aspects of performance that traditional tests may not recognise. If such assessment is to be accepted at the policy level, a shared set of criteria for achievement must be developed in terms of technical manipulation and content exposition. Lines of enquiry for further investigation of multimedia authoring, pupil learning and assessment are proposed.     

AI meets authoring: User models for untelligent multimedia

Authoring is a complex, knowledge-intensive activity which until recently has been performed exclusively by humans. New computer-based techniques have added horsepower rather than intelligence to traditional approaches, and have not addressed their principal limitations, chief of which is the inability to tailor presentations to individual users at run-time.We believe a model of the user is needed to support this kind of run-time determination of formand content. We describe our approach to the acquisition, representation and exploitation of user models: themost plausible user model is the result of an abductiverecognition process and it incorporates assumptions about the user which are then used to constrain thedesign by abduction of the best presentation. Both recognition and design processes are performed at run-time. We describe a prototypical implementation designed to demonstrate these ideas in the domain of video authoring.Our approach to authoring is intended to apply across multiple media; we have demonstrated these ideas with video because authoring in the video medium with traditional approaches inherits and exacerbates the problems from traditional media, and because the popularity of video as an authoring medium continues to grow.     

Canonical processes for creating personalized semantically rich multimedia presentations

Authoring of multimedia content can be considered as composing media assets such as images, videos, text, and audio in time, space, and interaction into a coherent multimedia presentation. Personalization of such content means that it reflects the users’ or user groups’ profile information and context information. Enriching the multimedia content with semantically rich metadata allows for a better search and retrieval of the content. To actually create personalized semantically-rich multimedia content, a manual authoring of the many different documents for all the different users’ and user groups’ needs is not feasible. Rather a (semi-)automatic authoring of the content seems reasonable. We have analyzed in detail today’s approaches and systems for authoring, personalizing, and semantically enriching multimedia presentations. Based on this analysis, we derived a general creation chain for the (semi-)automatic generation of such content. In this paper, we introduce this creation chain. We present our software engineering support for the chain, the component framework SemanticMM4U. The canonical processes supported by the creation chain and SemanticMM4U framework are described in detail. We also provide an explicit mapping of SemanticMM4U framework components to the processes and argue for the benefits of defining canonical processes for creating personalized semantically rich multimedia presentations.     

A hypermedia authoring tool for augmented and virtual reality

Most existing hypermedia authoring systems are intended for use on desktop computers. These systems are typically designed for the creation of 2D documents and therefore employ 2D authoring mechanisms. In contrast, authoring systems for nontraditional multimedia/hypermedia experiences for 3D virtual or augmented worlds focus mainly on creating separate media objects and embedding them within the user's surroundings. As a result, linking these media objects to create 3D hypermedia is a tedious manual task. To address this issue, we present an authoring tool for creating and editing linked 3D hypermedia narratives that are interwoven with a wearable computer user's surrounding environment. Our system is designed for use by authors who are not programmers, and allows them to preview their results on a desktop workstation, as well as with an augmented or virtual reality system.     

Urban Tapestries: Wireless networking, public authoring and social knowledge

Urban Tapestries is an action research project combining social research interaction design and cultural production. The project is developing a prototype system for accessing and publishing location specific content wirelessly, and creating sound maps of environments and journeys that can be shared remotely.     

Poster3D: a system for three-dimensional authoring and presentation in immersive virtual environments

We developed a simple system for three-dimensional (3D) authoring and presentation in immersive virtual reality environments. Our system enables a user to create and edit content both through direct manipulation using hand gestures in an immersive environment, and by using a 2D GUI on a PC desktop. The system also exploits various features of 3D information visualization to produce effective and interactive presentations. In a system trial, users found that the system enhanced their experience. The system helps users create 3D educational materials more easily and learn through the materials with greater interest and motivation. Noritaka Osawa received his B.S., M.S. and Ph.D. degrees in information science from the University of Tokyo in 1983, 1985, and 1988 respectively. After working for a software company and the University of Electro-communications, he has been at the National Institute of Multimedia Education since 1998. He has also been jointly appointed at the Graduate University for Advanced Studies since 2001. He now works for them as a Professor. His current research interests include human–computer interfaces using virtual reality technologies, information visualization, and system software. Sangtae Kim received B.A. degree in Visual Communication from YougIn University, Korea, in 1996, and M.A. and Ph.D. degrees in Design, from the University of Tsukuba, 2001, and 2004, respectively. He is currently an Assistant Professor of the institute of Art and Design, The University of Tsukuba, Ibaraki, Japan. His research interests include visual communication, dynamic information graphics, multimedia systems, 3DCG, and Web3D technology. He planned for the MDD (Multi-Dynamic Documentation) project in 2003. At present, he is involved in the development of Web3D contents (Interactive Anatomy system) that uses MDD technology.     

A unified multimedia and semantic perspective for data retrieval in the semantic web

In recent years, the emerging diffusion of peer-to-peer networks is going beyond the single-domain paradigm like, for instance, the mono-thematic file sharing one (e.g. Napster for music). Peers are more and more heterogeneous data sources which need to share data with commercial, educational, and/or collaboration purposes, just to mention a few. Moreover, in current information processing applications data cannot be meaningfully searched by precise database queries that would return exact matches (e.g. when dealing with multimedia, proteomic, statistical data).     

AutoBrief: a multimedia presentation system for assisting data analysis

We present an approach to generating multimedia presentations that integrates hierarchical planning to achieve communicative goals, and task-based graphic design. A planning process decomposes domain-specific goals to domain-independent goals, which in turn are realized by media-specific techniques such as task-based graphic design. We apply our approach to developing AutoBrief, a system that summarizes large data sets using natural language and information graphics. Finally, we analyze AutoBrief in terms of the standard reference model (SRM).     

Context aware messaging service: A dynamical messaging delivery using location information and schedule information

This paper describes a context-based dynamic messaging system. It uses schedule information and location information of the message addressee. According to the addressee's communication context (schedule, location, and available media), the system selects the most suitable telephone number or e-mail address, and redirects each incoming message dynamically. It also writes the schedule and location information of users into an HTML file which users can share on the WWW. It will work as mobile groupware for mobile workers. Evaluating the system during a two-month experiment in Tokyo, we found that our system produced smooth communication and a greater feeling of teamwork.     

The IP multimedia domain: service architecture for the delivery of voice, data, and next generation multimedia applications

The IP Multimedia Subsystem (IMS) is defined by the 3 ^rd Generation Partnership Project (3GPP) as a new core network domain. IMS provides a service control platform that allows creation of new multimedia and multi-session applications utilizing wireless and wireline transport capabilities. In this paper we will cover the concepts and standards defining IMS and review the network architecture from a mobile perspective. We will see how IMS interacts with the Packet Switched Domain (e.g. Wireless LAN, GPRS, and UMTS networks), the Internet, and application services. Then we will examine the key IMS capabilities and show how they can be combined to create new mobile IP services. Finally, we present a software architecture, which is enabled by IMS and allows development of unique applications (with multimedia/multi-session functionality, single/multi-user, service to user). The software architecture is illustrated by an example of a prototype application. This work was done when the author was with Siemens Communications Inc., Boca Raton, FL     

The IP multimedia domain: service architecture for the delivery of voice, data, and next generation multimedia applications

The IP Multimedia Subsystem (IMS) is defined by the 3^rd Generation Partnership Project (3GPP) as a new core network domain. IMS provides a service control platform that allows creation of new multimedia and multi-session applications utilizing wireless and wireline transport capabilities. In this paper we will cover the concepts and standards defining IMS and review the network architecture from a mobile perspective. We will see how IMS interacts with the Packet Switched Domain (e.g. Wireless LAN, GPRS, and UMTS networks), the Internet, and application services. Then we will examine the key IMS capabilities and show how they can be combined to create new mobile IP services. Finally, we present a software architecture, which is enabled by IMS and allows development of unique applications (with multimedia/multi-session functionality, single/multi-user, service to user). The software architecture is illustrated by an example of a prototype application. This work was done when the author was with Siemens Communications Inc., Boca Raton, FL     

Using visual and text features for direct marketing on multimedia messaging services domain

Traditionally, direct marketing companies have relied on pre-testing to select the best offers to send to their audience. Companies systematically dispatch the offers under consideration to a limited sample of potential buyers, rank them with respect to their performance and, based on this ranking, decide which offers to send to the wider population. Though this pre-testing process is simple and widely used, recently the industry has been under increased pressure to further optimize learning, in particular when facing severe time and learning space constraints. The main contribution of the present work is to demonstrate that direct marketing firms can exploit the information on visual content to optimize the learning phase. This paper proposes a two-phase learning strategy based on a cascade of regression methods that takes advantage of the visual and text features to improve and accelerate the learning process. Experiments in the domain of a commercial Multimedia Messaging Service (MMS) show the effectiveness of the proposed methods and a significant improvement over traditional learning techniques. The proposed approach can be used in any multimedia direct marketing domain in which offers comprise both a visual and text component.

MIME: a portable and robust multimedia format for Internet mail

Although multimedia systems are rapidly becoming more widely available and affordable, most present-day multimedia applications are not designed with interoperability in mind. Multimedia data from one application can be used in another only in relatively specialized circumstances or with special translator software, effort, and expense. While this situation is undesirable, it is not a critical impediment to the use of multimedia in many application domains. In others, however, the lack of interoperability is crucial, and nowhere is this more the case than for multimedia electronic mail. MIME (Multipurpose Internet Mail Extensions), the new standard format defined by an Internet Engineering Task Force Working Group, offers a simple standardized way to represent and encode a wide variety of media types, including images, audio, video, and non-ASCII textual data, for transmission via Internet mail. MIME extends Internet mail in a manner that is simple, completely backward-compatible, yet flexible and open to extension. In addition to enhanced functionality for Internet mail, the new mechanism offers the promise of interconnecting X.400 islands without the loss of functionality currently found in X.400-to-Internet gateways. This paper describes the general approach and rationale of the new mechanisms for Internet multimedia mail.     

Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures

The present study aimed at extending research on multimedia design principles by investigating their validity as a function of learners’ reading comprehension and scientific literacy. Students (N = 125; age: M = 15.11 years) learned about cell reproduction during their regular Biology lessons in one of six conditions resulting from cross-varying multimedia (text only vs. text plus animations) and text modality (spoken vs. written vs. spoken and written). Recall and transfer were assessed immediately after learning and again 1 week later. Overall, adding animations to text as well as using spoken rather than written text improved only immediate recall; in addition, a multimedia effect for delayed recall was observed for learners with higher levels of scientific literacy. A redundant presentation of text proved harmful especially for delayed performance measures. Reading comprehension did not moderate multimedia design effects. Students with more suitable cognitive prerequisites were better able to maintain performance from the immediate to the delayed tests. Future multimedia research should further investigate the boundary conditions that moderate multimedia effectiveness.     

Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect

The purpose of this study was to investigate the effects of modality (written text vs. spoken text) and visual cueing (low cueing vs. high cueing) on the learning and mental effort of participants studying a computer-based static diagram at their own pace. Participants were randomly assigned to four versions of the computer-based materials formed into a 2 × 2 factorial design by crossing modality with cueing. The results revealed a reverse modality effect, wherein participants studying written text outperformed those studying spoken text on tests of free recall, matching, comprehension, and spatial recall, but not mental effort. Information cueing did not significantly affect either performance or mental effort. These findings are discussed in the context of two popular explanations of the modality effect: the cognitive resources explanation and the perceptual resources explanation. The results were best explained from a perceptual resources viewpoint.     

PITAGORAS-PSP: Including domain knowledge in a multi-objective approach for protein structure prediction

Whenever evolutionary algorithms are used to solve certain classes of problems such as those that present a huge search space, the incorporation of problem-specific knowledge is required to achieve adequate levels of performance. In this paper, we propose a multi-objective optimization-based procedure that includes such a domain-specific knowledge to cope with a difficult problem, the protein structure prediction (PSP). This problem is considered to be an open problem as there is no recognized “best” procedure to find solutions. It presents a vast search space and the analysis of each protein conformation requires significant amount of computing time. In our procedure, we provide a reduction of the search space by using the dependent rotamer library and include new heuristics to improve a multi-objective approach to PSP based on the PAES algorithm. As it is shown in the paper, by using benchmark proteins from the CASP8 set, this hybrid PSP procedure provides competitive results when it is compared with some of the better proposals appeared up to now.