Creating interactive multimedia-based educational courseware: cognition in learning

Multimedia involves a combination of multiple modes such as text, graphics, audio and video to present information for learners. Multimedia-based learning has increasingly proliferated in educational circles, having been shaped by rapid technological advancements. In designing pedagogically sound multimedia systems a high premium needs to be placed on leveraging a judicious mix of various presentation modes to cater to users differing learning styles and needs. This will ensure that learning is optimised which is essentially learner-centric in nature in multimedia rich learning environments. Learner-centric in the sense that learners have control over the pace of their own learning and are empowered to take charge of their learning trajectories. In recent times, cognitive psychology has progressed in its empirical research development and many of the major findings in this field have been of use in the instructional design of multimedia educational resources. In this paper, we review some of these key principles and their implications in the development of multimedia contentware that is both aesthetically appealing and pedagogically engaging.     

Image representations and feature selection for multimedia database search

The success of a multimedia information system depends heavily on the way the data is represented. Although there are "natural" ways to represent numerical data, it is not clear what is a good way to represent multimedia data, such as images, video, or sound. We investigate various image representations where the quality of the representation is judged based on how well a system for searching through an image database can perform-although the same techniques and representations can be used for other types of object detection tasks or multimedia data analysis problems. The system is based on a machine learning method used to develop object detection models from example images that can subsequently be used for examples to detect-search-images of a particular object in an image database. As a base classifier for the detection task, we use support vector machines (SVM), a kernel based learning method. Within the framework of kernel classifiers, we investigate new image representations/kernels derived from probabilistic models of the class of images considered and present a new feature selection method which can be used to reduce the dimensionality of the image representation without significant losses in terms of the performance of the detection-search-system.     

Multimedia design: the effects of relating multimodal information

Abstract Few models describe learner behaviour during the simultaneous processing of several types of information, yet this is the defining characteristic of the use of multimedia tools, which bring together media in different informational formats (fixed or moving images, sound, text). Following studies in cognitive psychology concerning the increase in the ability to form mental images of words, this study was aimed at defining how different multimedia presentation modes affect the learning of foreign language vocabulary (Russian). A statistically significant effect was observed on word memorisation in the different information presentation modes, suggesting better processing when there is co-referencing of the different sources, especially when the encoding and tests modes are the same. In addition to these experimental results, some principles for the design of multimodal learning tools are discussed.     

Exploring the characteristics of an optimal design of digital materials for concept learning in mathematics: Multimedia learning and variation theory

Design principles emerging from the cognitive theory of multimedia learning can be applied to engage cognitive processing, and teaching methods that have evolved from variation theory can encourage thinking through comparisons in mathematics education. Applying these principles and teaching methods in designing digital material should be a sound proposition. However, there is a disconnection between research in digital educational material and classroom practices. Teachers often have doubts about the effectiveness of the materials. Thus, this paper presents a design-based research of developing a digital material for algebra concept learning. We collaborated with two experienced teachers and a subject expert from a university, and designed some digital learning material that was presented to 68 students through an iterative redesign development cycle; the effectiveness of the final product was tested on another group of 66 students the following year. Characteristics of an optimal design generated from the data collected are presented in this paper. The characteristics may have useful practical implications for instructional designers and teachers and contribute to improvements in the design of digital learning materials.     

多媒体技术研究:2014——深度学习与媒体计算

目的 海量数据的快速增长给多媒体计算带来了深刻挑战。与传统以手工构造为核心的媒体计算模式不同,数据驱动下的深度学习(特征学习)方法成为当前媒体计算主流。方法 重点分析了深度学习在检索排序与标注、多模态检索与语义理解、视频分析与理解等媒体计算方面的最新进展和所面临的挑战,并对未来的发展趋势进行展望。结果 在检索排序与标注方面, 基于深度学习的神经编码等方法取得了很好的效果;在多模态检索与语义理解方面,深度学习被用于弥补不同模态间的“异构鸿沟“以及底层特征与高层语义间的”语义鸿沟“,基于深度学习的组合语义学习成为研究热点;在视频分析与理解方面, 深度神经网络被用于学习视频的有效表示方式及动作识别,并取得了很好的效果。然而,深度学习是一种数据驱动的方法,易受数据噪声影响, 对于在线增量学习方面还不成熟,如何将深度学习与众包计算相结合是一个值得期待的问题。结论 该综述在深入分析现有方法的基础上,对深度学习框架下为解决异构鸿沟和语义鸿沟给出新的思路。     

Learning in high-dimensional multimedia data: the state of the art

During the last decade, the deluge of multimedia data has impacted a wide range of research areas, including multimedia retrieval, 3D tracking, database management, data mining, machine learning, social media analysis, medical imaging, and so on. Machine learning is largely involved in multimedia applications of building models for classification and regression tasks, etc., and the learning principle consists in designing the models based on the information contained in the multimedia dataset. While many paradigms exist and are widely used in the context of machine learning, most of them suffer from the ‘curse of dimensionality’, which means that some strange phenomena appears when data are represented in a high-dimensional space. Given the high dimensionality and the high complexity of multimedia data, it is important to investigate new machine learning algorithms to facilitate multimedia data analysis. To deal with the impact of high dimensionality, an intuitive way is to reduce the dimensionality. On the other hand, some researchers devoted themselves to designing some effective learning schemes for high-dimensional data. In this survey, we cover feature transformation, feature selection and feature encoding, three approaches fighting the consequences of the curse of dimensionality. Next, we briefly introduce some recent progress of effective learning algorithms. Finally, promising future trends on multimedia learning are envisaged.     

一种多媒体数据库管理系统的体系结构

文中简要介绍了一种基于层次－关系模式的多用户媒体数据库管理系统的客户／服务器体系结构以及多媒体声音图象的存储方式。对用于描述多媒体图象，声音及ＭＩＤＩ音乐的数据类型－Ｍ类型进行了扩充。     

The influence of affordances on user preferences for multimedia language learning applications

This study investigates the influence of sensory and cognitive affordances on the user experience of mobile devices for multimedia language learning applications. A primarily audio-based language learning application – ‘Vowel Trainer’, was chosen against a comparison, text and picture-based language learning application – ‘Learn English for Taxi Drivers’. Impressions of the two applications were assessed on two different devices that have virtually the same interface and identical sound output (when headphones are used), but differ in physical size: the iPhone and the iPad. A mixed design was chosen, with native language as a group factor and device type (iPad vs. iPhone) and language application type (audio vs. video) as within groups factors. Assessments of sensory and cognitive affordances were made, along with measurement of learner preferences of each application. Data from 41 participants (21 native English speakers, 20 non-native English speakers) were analysed, revealing device differences in both audio and visual subjective quality ratings, despite only visual quality being affected by the device's physical limitations. We suggest that sensory affordances (indexed by subjective quality) are not simply a function of physical limitations, but are heavily influenced by context. The implications for developing design guidelines for language learning and other multimedia applications are discussed.     

Multimedia integrated annotation based on common space learning

Multimedia automatic annotation, which assigns text labels to multimedia objects, has been widely studied. However, existing methods usually focus on modeling two types of media data or pairwise correlation. In fact, heterogeneous media are complementary to each other and optimizing them simultaneously can further improve accuracy. In this paper, a novel common space learning (CSL) algorithm for multimedia integrated annotation is presented, by which heterogeneous media data can be projected into a unified space and multimedia annotation is transformed to the nearest neighbor search in the space. Optimizing these heterogeneous media simultaneously makes the heterogeneous media complementary to each other and aligned in the common space. We solve the proposed CSL as an optimization problem mainly considering the following issues. First, different types of media objects with the similar labels should be closer in the common space. Second, the media similarity of the original space and the common space should be consistent. We attempt to solve the optimization problem in a sparse and semi-supervised learning framework, thus more unlabeled data can be integrated into the learning process, which can boost the performance of space learning. In addition, we proposed an iterative optimization algorithm to solve the problem. Since the projected samples in the common space share the same representation, the labels for new media object are assigned by a simple nearest neighbor voting mechanism. To the best of our knowledge, our method has made the first attempt to multimedia integrated annotation. Experiments on data sets with up to four media types (image, sound, video and 3D model) show the effectiveness of our proposed approach, as compared with the state-of-the-art methods.

     

Assessing the effects of different multimedia materials on emotions and learning performance for visual and verbal style learners

Multimedia materials are now increasingly used in curricula. However, individual preferences for multimedia materials based on visual and verbal cognitive styles may affect learners' emotions and performance. Therefore, in-depth studies that investigate how different multimedia materials affect learning performance and the emotions of learners with visual and verbal cognitive styles are needed. Additionally, many education scholars have argued that emotions directly affect learning performance. Therefore, a further study that confirms the relationships between learners' emotions and performance for learners with visual and verbal cognitive styles will provide useful knowledge in terms of designing an emotion-based adaptive multimedia learning system for supporting personalized learning. To investigate these issues, the study applies the Style of Processing (SOP) scale to identify verbalizers and visualizers. Moreover, the emotion assessment instrument emWave, which was developed by HeartMath, is applied to assess variations in emotional states for verbalizers and visualizers during learning processes. Three different multimedia materials, static text and image-based multimedia material, video-based multimedia material, and animated interactive multimedia material, were presented to verbalizers and visualizers to investigate how different multimedia materials affect individual learning performance and emotion, and to identify relationships between learning performance and emotion. Experimental results show that video-based multimedia material generates the best learning performance and most positive emotion for verbalizers. Moreover, dynamic multimedia materials containing video and animation are more appropriate for visualizers than static multimedia materials containing text and image. Finally, a partial correlation exists between negative emotion and learning performance; that is, negative emotion and pretest scores considered together and negative emotion alone can predict learning performance of visualizers who use video-based multimedia material for learning.     

Why advice on task selection may hamper learning in on-demand education

In on-demand education, learners are required to plan their own learning trajectory by selecting suitable learning tasks. A positive effect on learning is expected when learners select tasks that help them fulfil their individual learning needs. However, the selection of suitable tasks is a difficult process for learners with little domain knowledge and suboptimal task-selection skills. A common solution for helping learners deal with on-demand education and develop domain-specific skills is to give them advice on task selection. In a randomized experiment, learners (N = 30) worked on learning tasks in the domain of system dynamics and received either advice or no advice on the selection of new learning tasks. Surprisingly, the no-advice group outperformed the advice group on a post-test measuring domain-specific skills. It is concluded that giving advice on task selection prevents learners from thinking about how the process of task selection works. The advice seems to supplant rather than support their considerations why they should perform the advised task, which results in negative effects on learning. Implications for future research on giving advice in on-demand education are discussed.     

Assessing the effects of different multimedia materials on emotions and learning performance for visual and verbal style learners

Multimedia materials are now increasingly used in curricula. However, individual preferences for multimedia materials based on visual and verbal cognitive styles may affect learners' emotions and performance. Therefore, in-depth studies that investigate how different multimedia materials affect learning performance and the emotions of learners with visual and verbal cognitive styles are needed. Additionally, many education scholars have argued that emotions directly affect learning performance. Therefore, a further study that confirms the relationships between learners' emotions and performance for learners with visual and verbal cognitive styles will provide useful knowledge in terms of designing an emotion-based adaptive multimedia learning system for supporting personalized learning. To investigate these issues, the study applies the Style of Processing (SOP) scale to identify verbalizers and visualizers. Moreover, the emotion assessment instrument emWave, which was developed by HeartMath, is applied to assess variations in emotional states for verbalizers and visualizers during learning processes. Three different multimedia materials, static text and image-based multimedia material, video-based multimedia material, and animated interactive multimedia material, were presented to verbalizers and visualizers to investigate how different multimedia materials affect individual learning performance and emotion, and to identify relationships between learning performance and emotion. Experimental results show that video-based multimedia material generates the best learning performance and most positive emotion for verbalizers. Moreover, dynamic multimedia materials containing video and animation are more appropriate for visualizers than static multimedia materials containing text and image. Finally, a partial correlation exists between negative emotion and learning performance; that is, negative emotion and pretest scores considered together and negative emotion alone can predict learning performance of visualizers who use video-based multimedia material for learning.     

Effects of multimedia and schema induced analogical reasoning on science learning

The present study investigates the effects of multimedia and schema induced analogical reasoning on science learning. It involves 89 fourth grade elementary students in the north‐east of the United States. Participants are randomly assigned into four conditions: (a) multimedia with analogy; (b) multimedia without analogy; (c) analogy without multimedia; and (d) non‐multimedia and non‐analogy. The multivariate analyses of covariance reveal significant main effects for multimedia and analogy learning as well as a significant interaction between multimedia and analogy. The findings show that schema induced analogical reasoning can significantly improve science learning and that multimedia becomes more effective when it is integrated with an instructional method such as analogy and less so when it is used only as a visual tool. The study also shows the field dependence/independence as a significant covariate that influences learners' schema induced analogical reasoning in learning. Discussions pertaining to the significance of the findings and their implications for teaching and learning are made. Suggestions for future research are included with an emphasis on developing multimedia supported analogical reasoning for science learning.     

Exploring the relation between learning style models and preferred multimedia types

There are many adaptive learning systems that adapt learning materials to student properties, preferences, and activities. This study is focused on designing such a learning system by relating combinations of different learning styles to preferred types of multimedia materials. We explore a decision model aimed at proposing learning material of an appropriate multimedia type. This study includes 272 student participants. The resulting decision model shows that students prefer well-structured learning texts with color discrimination, and that the hemispheric learning style model is the most important criterion in deciding student preferences for different multimedia learning materials. To provide a more accurate and reliable model for recommending different multimedia types more learning style models must be combined. Kolb's classification and the VAK classification allow us to learn if students prefer an active role in the learning process, and what multimedia type they prefer.     

A Simplified Model for Generating 3D Realistic Sound in the Multimedia and Virtual Reality Systems

It is a key feature to embed 3D realistic sound effect in the future multimedia and virtual reality systems.Recent research on acoustics and psychoacoustics reveals the important cues for sound localization and sound perception.One promising approach to generate 3D realistic sound effect uses two earphones by simulating the sound waveforms from sound source to eardrum.This paper summarizes two methods for generating 3D realistic sound and points out their inherent drawbacks.To overcome these drawbacks we propose a simplified model to generate 3D realistic sound at any positions in the horizontal plane based on the results of sound perception and localization.Experimental results show that the model is correct and efficient.     

Multimedia language learning courseware: a design solution to the production of a series of CD-ROMs

Multimedia software combining video, sound, pictures, text and feedback in an interactive environment is ideally suited for developing a foreign language skill, but is costly and time-consuming to produce. This paper describes the production and the learning rationale of a series of six multimedia CD-ROMs that develop the listening skills of learners of Business English. The problems of cost, time, and quality which were encountered in the production of multiple courseware are described. The programming solution to these problems is put forward. This involved the creation of tools which enabled the subject expert to have complete and direct control over the language learning material.     

Students at the centre: non-lineal narratives and self-conscious learning

This paper reflects on the design and evaluation of a multimedia learning tool developed for students of qualitative research. Two issues are explored which are pertinent to the introduction of multimedia networked learning tools into existing teaching and learning cultures. First, the concept of the non-lineal — in multimedia narratives, in the ethnographic research process, in the learning process and in relation to teaching strategies. Second, drawing from evaluations of the multimedia tool the idea of self-conscious multilineal learning, and continuities between hypermedia learning and conventional teaching and learning practices are discussed. Finally the theoretical and practical implications of these discussions are related to the question of developing 'student-centred learning'.     

多媒体系统中双滑杆音量控制技术

阐述了多媒体系统中的背景音乐与解说声音的作用与意义,具体介绍了一种在多媒体创作中实现背景音乐与解说声音的双滑杆音量控制技术,该技术与目前一些常用的声音控制方法相比,在控制上更为容易、在程序界面也较为简洁。     

WINDOWS ACCESS数据库多媒体编程技术

ＡＣＣＥＳＳ是ＭＩＣＲＯＳＯＦＴ公司推出的ＷＩＮＤＯＷＳ下的一种多媒体数据库，虽然它具备一定的多媒体能力，但其ＡＣＣＥＳＳＢＡＳＩ编程语言未提供多媒体编程接口，本文结合实际应用，介绍了在ＡＣＣＥＳＳ数据库程序中多媒体音频、视频编程技术。并给出了在ＳｏｕｎｄＢｌａｓｔｅｒ音频卡和ＶｉｄｅｏＢｌａｓｔｅｒ视频卡上的编程实例。     

基于Win School 2100辅助教学设计与实践

通过对Win School 2100多媒体网络语音室的介绍,来探讨Win School 2100多媒体网络语音室相对其他语音教室的优势和功能,进而论述Win School 2100多媒体网络语音室在教学应用中的作用。