Challenges and opportunities of multimodal data in human learning: The computer science students' perspective

Multimodal data have the potential to explore emerging learning practices that extend human cognitive capacities. A critical issue stretching in many multimodal learning analytics (MLA) systems and studies is the current focus aimed at supporting researchers to model learner behaviours, rather than directly supporting learners. Moreover, many MLA systems are designed and deployed without learners' involvement. We argue that in order to create MLA interfaces that directly support learning, we need to gain an expanded understanding of how multimodal data can support learners' authentic needs. We present a qualitative study in which 40 computer science students were tracked in an authentic learning activity using wearable and static sensors. Our findings outline learners' curated representations about multimodal data and the non-technical challenges in using these data in their learning practice. The paper discusses 10 dimensions that can serve as guidelines for researchers and designers to create effective and ethically aware student-facing MLA innovations.     

Designing learner-controlled educational interactions based on learning/cognitive style and learner behaviour

Recently, research in individual differences and in particular, learning and cognitive style, has been used as a basis to consider learner preferences in a web-based educational context. Modelling style in a web-based learning environment demands that developers build a specific framework describing how to design a variety of options for learners with different approaches to learning. In this paper two representative examples of educational systems, Flexi-OLM and INSPIRE, that provide learners a variety of options designed according to specific style categorisations, are presented. Experimental results from two empirical studies performed on the systems to investigate learners' learning and cognitive style, and preferences during interaction, are described. It was found that learners do have a preference regarding their interaction, but no obvious link between style and approaches offered, was detected. Derived from an examination of this experimental data, we suggest that while style information can be used to inform the design of learning environments that accommodate learners' individual differences, it would be wise to recommend interactions based on learners' behaviour. Learning environments should allow learners or learners' interaction behaviour to select or trigger the appropriate approach for the particular learner in the specific context. Alternative approaches towards these directions are also discussed.     

The effects of directive self‐explanation prompts to support active processing of multiple representations in a simulation‐based learning environment

Processing of multiple representations in multimedia learning environments is considered to help learners obtain a more complete overview of the domain and gain deeper knowledge. This is based on the idea that relating and translating different representations leads to reflection beyond the boundaries and details of the separate representations. To achieve this, the design of a learning environment should support learners in adequately processing multiple representations. In this study, we compared a scientific inquiry learning environment providing instructional support with directive self‐explanation prompts to relate and translate between representations with a scientific inquiry learning environment providing instructional support with general self‐explanation prompts. Learners who received the directive prompts outperformed the learners who received general prompts on test items assessing domain knowledge. These positive results did not stretch to transfer items and items measuring learners' capabilities to relate and translate representations in general. The results suggest that learner support should promote the active relation of representations and translation between them to foster domain knowledge, and that other forms of support (e.g. extended training) might be necessary to make learners more expert processors of multiple representations.     

Informal learning with PDAs and smartphones

Abstract There has been increasing interest in informal learning in recent years alongside interest in how such learning can be supported by technology. However, relatively little is known about the extent to which adults make use of their own mobile devices to support informal learning. In this study, a survey was used to investigate whether, and to what extent, experienced users of mobile devices use their mobile devices to support intentional informal learning. If so, do they make use of mobile device connectivity to support opportunistic informal learning and does such connectivity support or encourage collaborative informal learning? Experienced mobile device users were recruited from web forums and business, and asked whether they used their devices to support informal learning. A pattern of learning uses emerged, some of which deployed the mobile device capabilities relatively unchanged, others triggered adaptations to typical learning activities to provide a better fit to the needs of the learner. These informal learning activities provided the basis for the design of a flexible mobile learning framework that can be extended to support developments in mobile technology, and increasing use of Web 2.0 technologies by informal learners.     

Natural,practical and social contexts of e‐learning: a critical realist account for learning and technology

Much of the research in educational technology with a primary concern over how technology enhances learning has been criticized as privileging the immediate learning settings over the other dimensions of learners’ social life and the wider social and economic contexts in which learning and technology are located. The ability to develop a rich understanding of learning and technology in various contexts requires careful use of theory that might enable ‘looking beyond learning’. To meet this need, this paper proposes the use of a critical realistic account of learning and technology, in particular Margaret Archer's ‘three orders of reality’ and ‘personal Identity’. Drawing on an empirical case study for illustration, the paper argues that such an approach offers a ‘way out’ to unpack the relationship between learning and technology through a deep exploration of the interrelationship between individual practices of learning, with or without technology, and the bigger picture of how learning intervenes in other dimensions of social life and how other contexts come into play in learning settings.     

DCLM framework: understanding collaboration in open-ended tabletop learning environments

There is a growing understanding of the unique ways that tabletops support effective collaboration; however, this research mostly focuses on environments in which learners work towards a single shared goal. Underpinning this perspective, either implicitly or explicitly, is the theory that collaborative learning is a process of attaining convergent conceptual change. However, this model of collaboration may not apply to all scenarios where learners are working together. In particular, informal, open-ended exploratory environments support (and often promote) shared activities where the goal may not be for all participants to emerge with a single, shared understanding. There is increased interest in understanding the efficacy of designs that support (and encourage) learners to collaborate while seeking divergent goals, ideas, and conceptions. This paper advances a framework (Divergent Collaboration Learning Mechanisms - DCLM) for recognizing and coding collaboration and divergent learning in such environments. We apply the DCLM framework to an informal tabletop environment (Oztoc) as a means of highlighting how DCLM may reveal new productive interactions environments that support divergent forms of collaboration, mentorship, and learning. Analysis of participants’ interactions within Oztoc revealed that participants who have non-aligned goals can still productively collaborate, and in many cases can provide insight and feedback that would not be possible in shared-goal activities. We conclude with an examination of how open-ended exploratory environments can support communities of practice and legitimate peripheral participation, and the importance of divergent inquiry and divergent conceptual change across a range of learning environments.     

Modeling teachers' influence on learners' self-directed use of technology for language learning outside the classroom

Teachers are important social agents who shape the quantity and quality of students' self-directed use of technology for learning outside the classroom. This study aimed to model the influence of teacher behaviors on learners' self-directed technology use. A conceptual model of three types of teacher support (affection support, capacity support and behavior support) that were reported to influence students' self-directed technology use for learning outside the classroom was generated based on interviews with 15 undergraduate foreign language learners. One hundred and sixty undergraduate foreign language learners were then surveyed to test the conceptual model. The path analysis of the survey data suggested that affection support influenced learner self-directed technology use through strengthened perceived usefulness, and that capacity support and behavior support influenced learner self-directed technology use through enhanced facilitating conditions and computer self-efficacy. The research findings highlight the importance of raising teachers' awareness of the different roles they can play and of enhancing their abilities to perform a combination of the roles to promote learner self-directed use of technology for learning outside the classroom.     

Learning technologies: Affective and social issues in computer-supported collaborative learning

This paper is concerned with affective issues in learning technologies in a collaborative context. Traditionally in learning there has been a division between cognition and affect: where cognition is concerned with skills and processes such as thinking and problem-solving and affect with emotional areas such as motivation, attitudes, feelings. Affective issues have been viewed as somewhat problematic in studying learning, so although it is well known that learner attitude, motivation, and emotional state are very important, they have often been excluded from the frame of research, or studied separately from cognitive learning. This position is gradually changing and this paper considers what previous research has been conducted in these areas. It discusses the role of affective factors in three main areas of collaboration: in settings where learners are co-located, in on-line communities and to support and develop socio-emotional skills. It considers relevant developments in these areas, what the outcomes have been and suggests important directions for future research.     

Bootstrapping in a language learning environment

Abstract   This paper addresses a fundamental dilemma in the design of intelligent language learning environments: the more freedom a system offers to learners in the use of the target language, the more unwieldy the data is which the learners produce and the less able the system is to support inferences about learners from that data. It is shown how in a platform where learners and teachers interact, the teachers' feedback which is archived in the system and indexed to the learners' target language production can constitute affordances that support a process of bootstrapping from raw language output to potential insights into the learners' interlanguage and gaps in their grasp of the target language. The approach is illustrated with three types of learner errors uncovered in the corpus of learner English through this bootstrapping heuristic.     

The influence of individual espoused cultural values on self-directed use of technology for language learning beyond the classroom

Students' use of technology for learning beyond the classroom is critical to maximizing the educational potential of technology, and understanding factors that affect learners' adoption of technology is critical. This study examined the influence of individual espoused cultural values on learners' voluntary technology adoption in informal learning contexts. Structural equation modeling was used to test a model of the direct and moderating effects of espoused cultural values on learners' self-directed use of technology for learning beyond the classroom, based on the survey responses of 661 foreign language learners from three countries/regions. The study found strong direct effects but weak moderating effects of cultural values. It further found that long-term orientation and uncertainty avoidance were two individual-level cultural value orientations that played important roles in the voluntary adoption of technology for self-directed learning beyond the classroom.     

Optimal dynamic decision network model for scientific inquiry learning environment

While recent studies employ heuristic to support learners in scientific inquiry learning environments, this study examined the theoretical and practical aspects of decision-theoretic approach to simultaneous reason about learners’ scientific inquiry skills and provision of adaptive pedagogical interventions across time. In this study, the dynamic learner model, represented by three different Dynamic Decision Network (DDN) models, were employed and evaluated through a three-phase empirical study. This paper discusses how insights gained and lessons learned from the evaluations of a preceding model had led to the improvements of subsequent model; before finalizing the optimal design of DDN model. The empirical studies involved six domain experts, 101 first-year university learners, and dataset from our previous research. Each learner participated in a series of activities including a pretest, a session with INQPRO learning environment, a posttest, and an interview session. For each DDN model, the predictive accuracies were computed by comparing the classifications given by the model with (a) the results obtained from the pretest, posttest, and learner self-rating scores, and (b) classifications elicited by domain experts based on the learner interaction logs and the graphs exhibited by each model.     

Group online adaptive learning

Sharing information among multiple learning agents can accelerate learning. It could be particularly useful if learners operate in continuously changing environments, because a learner could benefit from previous experience of another learner to adapt to their new environment. Such group-adaptive learning has numerous applications, from predicting financial time-series, through content recommendation systems, to visual understanding for adaptive autonomous agents. Here we address the problem in the context of online adaptive learning. We formally define the learning settings of Group Online Adaptive Learning and derive an algorithm named Shared Online Adaptive Learning (SOAL) to address it. SOAL avoids explicitly modeling changes or their dynamics, and instead shares information continuously. The key idea is that learners share a common small pool of experts, which they can use in a weighted adaptive way. We define group adaptive regret and prove that SOAL maintains known bounds on the adaptive regret obtained for single adaptive learners. Furthermore, it quickly adapts when learning tasks are related to each other. We demonstrate the benefits of the approach for two domains: vision and text. First, in the visual domain, we study a visual navigation task where a robot learns to navigate based on outdoor video scenes. We show how navigation can improve when knowledge from other robots in related scenes is available. Second, in the text domain, we create a new dataset for the task of assigning submitted papers to relevant editors. This is, inherently, an adaptive learning task due to the dynamic nature of research fields evolving in time. We show how learning to assign editors improves when knowledge from other editors is available. Together, these results demonstrate the benefits for sharing information across learners in concurrently changing environments.

     

Designing learning objects that afford learners the experience of important variations in Chinese characters

Effective teaching should focus the attention of learners to its essential aspects. It follows that instructional software can be designed in such a way that allows learners to experience the important variations in the critical aspects of the content to be learned. This paper reports on the experience of designing such special kinds of instructional learning objects for the learning of Chinese characters. The design of these learning objects takes into consideration not only what Chinese characters are all about but also how learners commonly make errors while they learn to write the characters. Out of the analysis of these learners' errors, variations in the structural features of Chinese characters were pulled out and embodied in the design of the learning objects. Learners tinkering with the learning objects can thus implicitly develop a sense of the structural features or regularity of Chinese characters, which most importantly should prepare the learners to learn more new characters in the future. The main proposal of this paper is the notion of this variation‐affording instructional software that allows learners to attend to the essential aspects of what is to be learned. Furthermore, the idea of the learning object also differs from other instructional software in its small, self‐contained and reusable nature, such that teachers can flexibly embed the learning objects into their own teaching materials.     

What seams do we remove in mobile-assisted seamless learning? A critical review of the literature

Seamless learning refers to the seamless integration of the learning experiences across various dimensions including formal and informal learning contexts, individual and social learning, and physical world and cyberspace. Inspired by the exposition by Chan et al. (2006) on the seamless learning model supported by the setting of one or more mobile device per learner, this paper aims to further investigate the meaning of seamless learning and the potential ways to put it in practice. Through a thorough review of recent academic papers on mobile-assisted seamless learning (MSL), we identify ten dimensions that characterize MSL. We believe that such a framework allows us to identify research gaps in the stated area. A practitioner interested in adopting an MSL design or doing a new design can use our analysis to situate the dimensional space where the constraints or parameters of his or her design problem lie, and look at relevant design and research-based evidence of other related MSL systems to refine her own design.     

Self-directed learning in simulation-based discovery environments

SIMQUEST is an authoring system for designing and creating simulation-based learning environments. The special character of SIMQUEST learning environments is that they include cognitive support for learners which means that they provide learners with support in the discovery process. In SIMQUEST learning environments, a balance is sought between direct guidance of the learning process and sufficient freedom for learners to regulate the learning process themselves. This paper describes the basic mechanisms of the SIMQUEST learning and authoring environments. The functionality authors have in providing the learner with guidance and some of the experiences on how authors use these opportunities and learners employ the cognitive support are reported.     

Designing a mobile system for lifelong learning on the move

The Life‐long Learning Initiative seeks to fulfil a variety of learning needs for Shanghai citizens. Given the popularity of mobile devices in Shanghai, the ability to provide learning in informal settings through mobile devices is a key objective and challenge of the Initiative. In order to learn how to develop usable learning content for lifelong learners on the move, a set of design principles from both pedagogical and usability concerns was identified. Next, a pilot system, based on the design principles, was developed to implement two prototype lessons. Five subjects were recruited to test each prototype lesson using a heuristic walkthrough method and a focus group meeting. Users' feedback showed that both the practical and the micro principles were valued as a method of integrating learning activity into informal settings in their daily lives. Audio was preferred as well. Technical usability concerns which are consistent with previous web experience were also identified. This research builds new knowledge about design principles for lifelong learning on the move and is a milestone in the development of the future learning resource bank for Shanghai citizens.     

The role of digital artefacts on the interactive whiteboard in supporting classroom dialogue

This paper explores how the interactive whiteboard (IWB) might be harnessed to support student learning through classroom dialogue. This powerful and increasingly prevalent technology opens up opportunities for learners to generate, modify, and evaluate new ideas, through multimodal interaction along with talk. Its use can thereby support rich new forms of dialogue that highlight differences between perspectives, and make ideas and reasoning processes more explicit. The emerging account builds upon Bahktin's conception of dialogue and Wegerif's notion of technology ‘opening up a dialogic space for reflection’, but foregrounds the role of mediating artefacts. Classroom dialogue in the context of IWB use is construed as being facilitated by teachers and learners constructing digitally represented knowledge artefacts together. These visible, dynamic, and constantly evolving resources constitute interim records of activity and act as supportive devices for learners' emerging thinking, rather than finished products of dialogue. This primarily theoretical account is illustrated with examples from case studies of UK classroom practice. Analysing lessons in sequence has illuminated how teachers can exploit the IWB through cumulative interaction with a succession of linked digital resources, and through archiving and revisiting earlier artefacts. The tool thereby helps to support the progression of dialogue over time, across settings and even across learner groups. In sum, the article reframes the notion of dialogue for this new context in which students are actively creating and directly manipulating digital artefacts, and offers some practical examples.     

Cognitive load while learning with a graphical computer interface

Abstract  Computer software packages are hard to learn but little is known about how to help new users. A study was made in which 30 learners had to learn the basic commands of a CAD software package using manuals with different formats of presentation of instructions. Ten learners learned using a conventional manual plus a computer, 10 learners learned using a manual with juxtaposed screen images but no computer and 10 learners learned using a manual with integrated screen images but no computer. Each learner was tested with a practical and a written test and the learning time was measured. There was no statistical difference between the groups of learners for the score of the practical and written tests. However, it was found that learners using a manual with juxtaposed or integrated screen images had learned twice as fast as learners using a conventional manual. The discussion questions the use of conventional manuals to learn application software and stress as the key role of the graphical environment in the learning process.