An Exploratory Study: Supporting Collaborative Multimedia Systems Design

Multimedia systems design generally requires a collaborative effort from a group of designers with a variety of backgrounds and tasks, such as content experts, instructional designers, media specialists, users, and so forth. However, currently available design tools on the market are mainly designed for a single user. Tools intended to support a collaborative design process should coordinate independent activities of individual designers.This research investigated support for work groups engaged in designing multimedia systems. Specifically, it discussed a new collaborative design environment, called the KMS (Knowledge Management System)-based design environment, in which multimedia designers could share their design knowledge freely. Through two experimental groups, the research investigated impacts of the KMS-based design environment on their collaborative design activities (knowledge creating, knowledge securing, knowledge distributing, and knowledge retrieving activities). The research findings showed that the KMS-based design environment was a promising environment for collaborative multimedia systems design. More specifically, the research findings indicated that the KMS-based design environment supported creating, securing, and retrieving knowledge, but it did not support distributing knowledge. In addition, the research found that the social interactions between group members played important roles in the success of the collaborative multimedia systems design and that the KMS-based design environment did not support the socialization of group members. Furthermore, the research found that the inability of the KMS-based design environment to support the socialization was linked to its low performance level in supporting the knowledge distributing activity. The research explored the desired features of a collaborative support tool for multimedia systems design.     

Effects of task experience and layout on learning from text and pictures with or without unnecessary picture descriptions

The presentation of extraneous (i.e., irrelevant or unnecessary) information may hamper learning with multimedia. The present study examined whether people can learn to ignore unnecessary information with increasing experience with the task and whether this depends on the layout of that information. In two experiments, participants learned about the process of mitosis from a multimedia slideshow, with each slide presenting a combination of expository text and a picture on one of the stages in the process. Slides either contained no unnecessary text (control condition) or unnecessary text (i.e., merely describing the picture) either integrated in the picture (integrated condition) or presented underneath the picture (separated condition). Knowledge about the studied mitosis phase was tested immediately after each slide using a cloze test. Across Experiments 1 and 2, we did not find a reliable negative effect of the unnecessary text on cloze test performance. As a result, the question of whether task experience would reduce or eliminate that negative effect could not be answered. The eye movement data did confirm, however, that participants attended less to the unnecessary information with increasing task experience, suggesting that students can adapt their study strategy and learn to ignore unnecessary information.     

Metadata-driven multimedia transcoding for distance learning

Advances in multimedia processing capabilities of electronic devices and the rapid growth of the Internet have contributed to the proliferation of collaborative applications such as distance learning (DL) webcasting. A key technical challenge in such DL systems is providing access to rich media content to any user regardless of device capabilities, network heterogeneity, and personal preferences. A novel MPEG-21-based adaptation architecture is presented to overcome these challenges by performing (1) application layer transcoding that adapts the presentation format of DL content to match device media decoding capabilities and user-desired modality, (2) bitstream transcoding to adapt multimedia to match device processing capabilities and encoding bit-rate supported by the network. Experimental results indicate that the proposed system delivers personalized DL content to meet end-user environmental restrictions with small transcoding overhead. This work is partially supported by the Network for Effective Collaboration Technologies through Advanced Research (NECTAR). The authors would like to thank the Knowledge Media Design Institute (KMDI) at the University of Toronto for use of their resources.     

Canonical processes for creating personalized semantically rich multimedia presentations

Authoring of multimedia content can be considered as composing media assets such as images, videos, text, and audio in time, space, and interaction into a coherent multimedia presentation. Personalization of such content means that it reflects the users’ or user groups’ profile information and context information. Enriching the multimedia content with semantically rich metadata allows for a better search and retrieval of the content. To actually create personalized semantically-rich multimedia content, a manual authoring of the many different documents for all the different users’ and user groups’ needs is not feasible. Rather a (semi-)automatic authoring of the content seems reasonable. We have analyzed in detail today’s approaches and systems for authoring, personalizing, and semantically enriching multimedia presentations. Based on this analysis, we derived a general creation chain for the (semi-)automatic generation of such content. In this paper, we introduce this creation chain. We present our software engineering support for the chain, the component framework SemanticMM4U. The canonical processes supported by the creation chain and SemanticMM4U framework are described in detail. We also provide an explicit mapping of SemanticMM4U framework components to the processes and argue for the benefits of defining canonical processes for creating personalized semantically rich multimedia presentations.     

How charismatic trainers inspire others to learn through positive affectivity

We focus on how charismatic trainer behaviors, such as visionary content, intellectual stimulation and individual attention, influence affective, cognitive and skill-based learning outcomes during computer-based training. Using an experimental design, 92 undergraduates were presented with an on-line Excel training program that contained narration that was either charismatic or non-charismatic. Path analysis results indicate that participants who viewed videos narrated by a charismatic trainer (as opposed to a non-charismatic trainer) had positive reactions to the trainer, which was related to positive affectivity. Trainee positive affectivity was subsequently related to recall one week after training, and this recall was related to skill transfer one week after training. The results confirmed that charismatic trainers influence skills acquisition through boosting positive affect.     

A method and advisor tool for multimedia user interface design

This paper describes a multimedia user interface design method and a design assistant tool which supports the method. The method covers specification of user requirements and information architecture, selection of appropriate media to represent the information content, design for directing attention to important information and interaction design to enhance user engagement. Guidelines for media selection and design for attractiveness, i.e. usability and user experience, are given. The method was evaluated in a case study design of a crowd control simulation training system, which demonstrated the method was usable and gave good solutions against an expert gold standard design. The tool provides advice on media selection and attention effects that match specification of the information content expressed as information types and communication goals. A usability evaluation was carried out to measure the usefulness and effectiveness of the tool in comparison to the method, and the results showed that the tool has a positive impact on multimedia design.     

Simulation-based learning: The learning–forgetting–relearning process and impact of learning history

The results of empirical experiments evaluating the effectiveness and efficiency of the learning–forgetting–relearning process in a dynamic project management simulation environment are reported. Sixty-six graduate engineering students performed repetitive simulation-runs with a break period of several weeks between the runs. The students used a teaching tool called the project management trainer (PMT) that simulates a generic dynamic, stochastic project management environment. In this research, we focused on the effect of history recording mechanism on the learning forgetting process. Manual or automatic history recording mechanisms were used by the experimental group, while the control group did not use any history recording mechanism. The findings indicate that for the initial learning phase, the manual mechanism is better than the automatic mechanism. However, for the relearning phase, the break period length influenced the performance after the break. When the break period is short, the manual history keeping mechanism is better, but for a long period break, there is no significant difference. A comparison between the experimental group and the control group revealed that using any history recording mechanism reduced forgetting. Based on the findings, some practical implications of using simulators to improve the learning–forgetting process are discussed.     

Meta-Data and Media Management in a Multimedia Interactive Telelearning System

This paper describes the system design and the components of a web-based multimedia interactive Telelearning system (MITS) with emphasis on metadata and media content management mechanisms. Metadata model is defined and exploited in designing the overall architecture to support courseware creating, management, delivery, and learner performance evaluation. Multimedia document architecture is used to describe the media contents which represent the course material. Metadata and media contents are generated, stored, and utilized to facilitate the search and on-demand presentation of learning objects. Learner preference, performance and progress information is maintained by profile database to make courseware presentation adaptable and more interactive. The system provides a multimedia interactive learning environment with easy-to-search and highly reusable learning objects.     

Computational support to record and re-experience visits

Information technology research has mainly focussed on supporting visitors with guides, whilst a lot of opportunities for the development of technology for collecting, manipulating and re-travelling through material from the visits remain unexplored. On the basis of observations from a specific setting, we have developed prototype technology to combine multimedia content with position information. In particular, we developed mobile support to record and organise multimedia using the walked path in order to preserve more of the experience of the visit. We created tools to edit and store multimedia paths. Additional components make it possible to configure a mixed-media environment to navigate a multimedia path, using such physical interfaces as gesturing, and link the recorded media to other artefacts. When discussing field observations of current practices and prototype trials, we investigate what kind of functionality is needed to support the whole activity of collecting, manipulating and playing multimedia content in combination with position information.     

Exploring augmented reality for worker assistance versus training

This paper aims at advancing the fundamental understanding of the affordances of Augmented Reality (AR) as a workplace-based learning and training technology in supporting manual or semi-automated manufacturing tasks that involve both complex manipulation and reasoning. Between-subject laboratory experiments involving 20 participants are conducted on a real-life electro-mechanical assembly task to investigate the impacts of various modes of information delivery through AR compared to traditional training methods on task efficiency, number of errors, learning, independence, and cognitive load. The AR application is developed in Unity and deployed on HoloLens 2 headsets. Interviews with experts from industry and academia are also conducted to create new insights into the affordances of AR as a training versus assistive tool for manufacturing workers, as well as the need for intelligent mechanisms that enable adaptive and personalized interactions between workers and AR. The findings indicate that despite comparable performance between the AR and control groups in terms of task completion time, learning curve, and independence from instructions, AR dramatically decreases the number of errors compared to traditional instruction, which is sustained after the AR support is removed. Several insights drawn from the experiments and expert interviews are discussed to inform the design of future AR technologies for both training and assisting incumbent and future manufacturing workers on complex manipulation and reasoning tasks.     

Why does signaling enhance multimedia learning? Evidence from eye movements

Previous studies have suggested that signaling enhances multimedia learning. However, there is not enough evidence showing why signaling leads to better performance. The goal of this study was to examine the effects of signaling on learning outcomes and to reveal the underlying reasons for this effect by using eye movement measures. The participants were 40 undergraduate students who were presented with either signaled or nonsignaled multimedia materials. Labels in the illustration were signaled by temporarily changing the color of the items. The results suggest that the signaled group outperformed the nonsignaled group on transfer and matching tests. Eye movement data shows that signaling guided attention to relevant information and improved the efficiency and effectiveness of finding necessary information.     

Using Multimedia to Support Cooperative Editing

By Cooperative Editing we mean the coordinated manipulation of information by a group of authors. During the editing process the (co-)authors need to communicate their ideas, drafts and constraints (remotely or face-to-face) until a final version of the information is achieved. For the different phases of this process - discussion of ideas, editing, cross-checking - different media or media integration are adequate. Furthermore, analysing the transition from individual work to group work, within different human activities, two pitfalls are often detected if computer support is considered: a) technological communication difficulties, especially if the group is remotely located on heterogeneous hardware, associated with a fall in productivity and frequent social inadequacies of the group's computer support; b) the lack of integrated media processing tools available for group editing. In order to solve these problems, we suggest that Multimedia can be applied in two ways: to effectively support the necessary group communication links; and to enhance the expressiveness of the information edited. To test this statement we have been conceptualising and implementing a prototype system. Most of the techniques involved can be used in other tools that need multimedia capabilities or that support other specific types of group activities. The innovative aspects of the work are the use of multimedia techniques to support demanding applications, possibly on cross-platforms, and the integration of several concepts to support cooperation.     

An approach to human motor skill training for uniform group performance

Prior research has focused on designing training approaches for novice operators to support maximum motor skill development. However, in production operations, workers must be trained to uniform performance levels to prevent ‘bottlenecks’ or work-in-process inventory accumulation. This study introduces a new approach to support assignment of training protocols to operators to achieve comparable levels of motor performance. Thirty-six participants performed a computer-based motor test. Based on performance classification results, each participant was assigned to a specific haptic virtual reality training condition. Results revealed participants identified as ‘medium’ or ‘low’ performers achieved levels of motor performance comparable to ‘high’ performers through 1-h training.Relevance to IndustryFindings can be applied to operator training in manual assembly operations, promoting a group of novice workers to achieve uniform performance levels and mitigating production bottlenecks.     

Dynamicmultimedia protocol stacks

Fixed end-system protocols are unable to support the wide range of applications requirements on top of current networks without adding overhead in the form of unnecessary functionality for multiple combinations of application requirements and networks.This paper presents the Real-time Wide Area Network Dissemination Architecture Protocol (RWANDA), which dynamically configures multimedia protocol stacks to support a wide range of application requirements and to increase performance. It overcomes synchronous limitations by providing an asynchronous group communication model. Applications only pay for required quality of service (QoS) such as multicast, virtual synchrony and encrypted communication.In RWANDA, information sources use channels to disseminate information to a potentially large and changing set of channel subscribers. RWANDA recognises the differing media characteristics and transport requirements of multimedia by providing a protocol composition framework that extends to incorporate yet unsupported communication protocols, qualities of service and optimised multimedia stacks. RWANDA provides an asynchronous foundation necessary for developing a large-scale wide area network continuous media protocol.     

The effect of multimedia discrepancies on metacognitive judgments

This study investigated undergraduate students’ metacognitive judgments while learning about complex science topics using multimedia material (text and graph). A within-subjects design was used to examine the effect of discrepancies on study-time allocation, metacognitive judgments and inference generation. There were three types of discrepancies: none, text (between two ideas in the text) and text and graph (between the text and graph). Forty (N = 40) participants completed 12 trials where they were asked to provide six judgments: Ease of Learning judgments (EOLs), immediate and delayed Judgments of Learning (JOLs) for both text and graph and Retrospective Confidence Judgments (RCJs). Overall, participants provided significantly lower JOLs for content that contained discrepancies but RCJs remained high across conditions. Discrepancies did not influence study-time allocation, but did significantly influence inference scores. Overall, results suggest that participants’ judgments are different for different aspects of multimedia content and may be based on the text.     

Social interaction for efficient agent learning from human reward

Learning from rewards generated by a human trainer observing an agent in action has been proven to be a powerful method for teaching autonomous agents to perform challenging tasks, especially for those non-technical users. Since the efficacy of this approach depends critically on the reward the trainer provides, we consider how the interaction between the trainer and the agent should be designed so as to increase the efficiency of the training process. This article investigates the influence of the agent’s socio-competitive feedback on the human trainer’s training behavior and the agent’s learning. The results of our user study with 85 participants suggest that the agent’s passive socio-competitive feedback—showing performance and score of agents trained by trainers in a leaderboard—substantially increases the engagement of the participants in the game task and improves the agents’ performance, even though the participants do not directly play the game but instead train the agent to do so. Moreover, making this feedback active—sending the trainer her agent’s performance relative to others—further induces more participants to train agents longer and improves the agent’s learning. Our further analysis shows that agents trained by trainers affected by both the passive and active social feedback could obtain a higher performance under a score mechanism that could be optimized from the trainer’s perspective and the agent’s additional active social feedback can keep participants to further train agents to learn policies that can obtain a higher performance under such a score mechanism.     

The impact of media on collaborative learning in virtual settings: The perspective of social construction

This study examines an alternative function of information sharing – social construction of meaning. Drawing on social construction, social interaction, and task closure theories, we explored the influence of both the media environment in which students are situated and the medium that group members choose to communicate with one another on the intricate relationships among breadth of information sharing, depth of information sharing, and performance of computer-supported collaborative learning (CSCL). A total of 126 students participated in the experiment – including 63 students (15 groups of four students and one group of three students) in the control and experimental groups respectively. Our findings show that most of the proposed hypotheses are supported. Intersubjective interpretation underlies groups information sharing and plays a key role in student learning performance. Evidence shows that when facing a relatively complex task in multimedia environments, students who choose to utilize a medium lower in social presence (i.e., electronic information sharing) are more likely to achieve task closure than a medium higher in social presence (i.e., verbal information sharing). This in turn leads to higher learning performance. The implications for both theory and pedagogy are also discussed.     

Development and evaluation of a system enhancing Second Life to support synchronous role‐based collaborative learning

Research and commercial interest toward 3D virtual worlds are recently growing because they probably represent the new direction for the next generation of web applications. Although these environments present several features that are useful for informal collaboration, structured collaboration is required to effectively use them in a working or in a didactical setting. This paper presents a system supporting synchronous collaborative learning by naturally enriching Learning Management System services with meeting management and multimedia features. Monitoring and moderation of discussions are also managed at a single group and at the teaching level. The Second Life (SL) environment has been integrated with two ad hoc developed Moodle plug‐ins and SL objects have been designed, modeled, and programmed to support synchronous role‐based collaborative activities. We also enriched SL with tools to support the capturing and displaying of textual information during collaborative sessions for successive retrieval. In addition, the multimedia support has been enhanced with functionalities for navigating multimedia contents. We also report on an empirical study aiming at evaluating the use of the proposed SL collaborative learning as compared with face‐to‐face group collaboration. Results show that the two approaches are statistically undistinguishable in terms of performance, comfort with communication, and overall satisfaction. Copyright © 2009 John Wiley & Sons, Ltd.     

Exploring context and content links in social media: a latent space method

Social media networks contain both content and context-specific information. Most existing methods work with either of the two for the purpose of multimedia mining and retrieval. In reality, both content and context information are rich sources of information for mining, and the full power of mining and processing algorithms can be realized only with the use of a combination of the two. This paper proposes a new algorithm which mines both context and content links in social media networks to discover the underlying latent semantic space. This mapping of the multimedia objects into latent feature vectors enables the use of any off-the-shelf multimedia retrieval algorithms. Compared to the state-of-the-art latent methods in multimedia analysis, this algorithm effectively solves the problem of sparse context links by mining the geometric structure underlying the content links between multimedia objects. Specifically for multimedia annotation, we show that an effective algorithm can be developed to directly construct annotation models by simultaneously leveraging both context and content information based on latent structure between correlated semantic concepts. We conduct experiments on the Flickr data set, which contains user tags linked with images. We illustrate the advantages of our approach over the state-of-the-art multimedia retrieval techniques.