Enhancing multimodal learning through personalized gesture recognition

Gestural recognition systems are important tools for leveraging movement‐based interactions in multimodal learning environments but personalizing these interactions has proven difficult. We offer an adaptable model that uses multimodal analytics, enabling students to define their physical interactions with computer‐assisted learning environments. We argue that these interactions are foundational to developing stronger connections between students' physical actions and digital representations within a multimodal space. Our model uses real time learning analytics for gesture recognition, training a hierarchical hidden‐Markov model with a “one‐shot” construct, learning from user‐defined gestures, and accessing 3 different modes of data: skeleton positions, kinematics features, and internal model parameters. Through an empirical comparison with a “pretrained” model, we show that our model can achieve a higher recognition accuracy in repeatability and recall tasks. This suggests that our approach is a promising way to create productive experiences with gesture‐based educational simulations, promoting personalized interfaces, and analytics of multimodal learning scenarios.     

Developing a survey for assessing preferences in constructivist context‐aware ubiquitous learning environments

This study developed a survey to explore students' preferences in constructivist context‐aware ubiquitous learning environments. A constructivist context‐aware ubiquitous learning (u‐learning) environment survey (CULES) was developed, consisting of eight scales, including ease of use, continuity, relevance, adaptive content, multiple sources, timely guidance, student negotiation, and inquiry learning. The survey responses were gathered from 215 university students from five universities in Taiwan. The students all had actual experience of using u‐learning systems in u‐learning environments. Both exploratory and confirmatory factor analyses showed that the CULES had high reliability and validity. The structural model revealed that the provision of realistic and close‐to‐real‐life information could enhance students' preferences for timely guidance, student negotiation, and inquiry‐learning activities. In addition, the attainment of inquiry learning is quite challenging when designing u‐learning activities, as it involves the enhancement of the other CULES scales.     

Chinese undergraduate students' perceptions of mobile learning: Conceptions,learning profiles,and approaches

Close links between students' conceptions of and approaches to learning were established in the past research. However, only a few quantitative studies investigated this relationship particularly with regard to mobile learning (m‐learning). The correlation between learners' conceptions and approaches to m‐learning was analysed using a partial least squares analysis applied to data obtained from a sample of 971 undergraduate students in China. The results indicated that students' conceptions of m‐learning could be classified into reproductive, transitional, and constructive levels. Students may hold multiple m‐learning applications than a predominant one; hence, examining m‐learning as one monolithic entity may provide limited information. Latent profile analysis identified four learning profiles based on students' preferred m‐learning applications: passive, mixed, surface‐supportive, and high‐engagement.. Moreover, a general trend was observed, whereby students with reproductive and surface‐supportive learning profiles showed a tendency to adopt surface approaches, whereas those expressing constructive and mixed learning profiles were more inclined to adopt deep approaches. Interestingly, students with transitional conceptions and high‐engagement learning profiles tended to take both surface and deep approaches.     

Analysis of learners' navigational behaviour and their learning styles in an online course

Providing adaptive features and personalized support by considering students' learning styles in computer‐assisted learning systems has high potential in making learning easier for students in terms of reducing their efforts or increasing their performance. In this study, the navigational behaviour of students in an online course within a learning management system was investigated, looking at how students with different learning styles prefer to use and learn in such a course. As a result, several differences in the students' navigation patterns were identified. These findings have several implications for improving adaptivity. First, they showed that students with different learning styles use different strategies to learn and navigate through the course, which can be seen as another argument for providing adaptivity. Second, the findings provided information for extending the adaptive functionality in typical learning management systems. Third, the information about differences in navigational behaviour can contribute towards automatic detection of learning styles, helping in making student modeling approaches more accurate.     

A sequenced multimodal learning approach to support students' development of conceptual learning

Virtual learning environments can now be enriched not only with visual and auditory information, but also with tactile and kinesthetic feedback. However, the way to successfully integrate haptic feedback on a multimodal learning environment is still unclear. This study aims to provide guidelines on how visuohaptic simulations can be implemented effectively, thus the research question is: Under what conditions do visual and tactile information support students' development of conceptual learning of force‐related concepts? Participants comprised 170 undergraduate students of a Midwestern University enrolled in a physics for elementary education class. Four experiments were conducted using four different configurations of multimodal learning environments: Visual feedback only, haptic force feedback only, visual and haptic force feedback at the same time, and sequenced modality of haptic feedback first and visual feedback second. Our results suggest that haptic force feedback has the potential to enrich learning when compared with visual only environments. Also, haptic and visual modalities interact better when sequenced one after another rather than presented simultaneously. Finally, exposure to virtual learning environments enhanced by haptic forced feedback was a positive experience, but the ease of use and ease of interpretation was not so evident.     

Virtual learning environment to predict withdrawal by leveraging deep learning

The current evolution in multidisciplinary learning analytics research poses significant challenges for the exploitation of behavior analysis by fusing data streams toward advanced decision-making. The identification of students that are at risk of withdrawals in higher education is connected to numerous educational policies, to enhance their competencies and skills through timely interventions by academia. Predicting student performance is a vital decision-making problem including data from various environment modules that can be fused into a homogenous vector to ascertain decision-making. This research study exploits a temporal sequential classification problem to predict early withdrawal of students, by tapping the power of actionable smart data in the form of students' interactional activities with the online educational system, using the freely available Open University Learning Analytics data set by employing deep long short-term memory (LSTM) model. The deployed LSTM model outperforms baseline logistic regression and artificial neural networks by 10.31% and 6.48% respectively with 97.25% learning accuracy, 92.79% precision, and 85.92% recall.     

Learning linkages: Integrating data streams of multiple modalities and timescales

Increasingly, student work is being conducted on computers and online, producing vast amounts of learning‐related data. The educational analytics fields have produced many insights about learning based solely on tutoring systems' automatically logged data, or “log data.” But log data leave out important contextual information about the learning experience. For example, a student working at a computer might be working independently with few outside influences. Alternatively, he or she might be in a lively classroom, with other students around, talking and offering suggestions. Tools that capture these other experiences have potential to augment and complement log data. However, the collection of rich, multimodal data streams and the increased complexity and heterogeneity in the resulting data pose many challenges to researchers. Here, we present two empirical studies that take advantage of multimodal data sources to enrich our understanding of student learning. We leverage and extend quantitative models of student learning to incorporate insights derived jointly from data collected in multiple modalities (log data, video, and high‐fidelity audio) and contexts (individual vs. collaborative classroom learning). We discuss the unique benefits of multimodal data and present methods that take advantage of such benefits while easing the burden on researchers' time and effort.     

Enhancing middle school students' scientific learning and motivation through agent‐based learning

Facing students' decreasing motivation to pursue scientific study, schools and educators need to coordinate new technologies with pedagogical agents to effectively sustain or promote students' scientific learning and motivation to learn. Although the provision of pedagogical agents in student learning has been studied previously, it is not clear what benefits the strategy might offer with regard to student motivation. This study proposes an agent‐based mechanism that integrates problem‐solving and inquiry‐based instructions to help students better understand complex scientific concepts and to sustain their motivation to learn science. In this study, a quasi‐experiment was conducted to evaluate the performance and feasibility of our proposed mechanism. The results revealed that the agent‐based mechanism was effective and feasible for enhancing students' learning and motivation to learn. The mechanism was associated with increases in the acquisition of knowledge when compared with the control group. Its effect in promoting and sustaining students' motivation was also statistically significant. Detailed discussions of the findings are provided in this study.     

Embodied experiences of place: a study of history learning with mobile technologies

This paper reports an empirical study that takes a multimodal analytical approach to examine how mobile technologies shape students' exploration and experience of place during a history learning activity in situ. In history education, mobile technologies provide opportunities for authentic experiential learning activities that have the potential to re‐mediate students' understanding of space and place through enacted interaction, and to make the learning more memorable. A key question is how learners work with the physical and digital information in the context of that learning experience, and how this supports new experiences and understanding of space and place. Findings suggest that embodied mobile experiences foster the creation of both physical and digital markers, which were instrumental in concretizing the history experience and developing new narratives. The findings also show how different representational forms of digital information mediated interaction in specific ways and how digital augmentation can lead to conflation in student understanding of space and time. These findings inform our understanding of the value of mobile applications in supporting embodied learning experiences and provide key implications for pedagogical design, both in situ and back in the classroom.     

Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics

By collaboratively solving a task, students are challenged to share ideas, express their thoughts, and engage in discussion. Collaborating groups of students may encounter problems concerning cognitive activities (such as a misunderstanding of the task material). If these problems are not addressed and resolved in time, the collaborative process is hindered. The teacher plays an important role in monitoring and solving the occurrence of problems. To provide adaptive support, teachers continuously have to be aware of students' activities in order to identify relevant events, including those that require intervention. Because the amount of available information is high, teachers may be supported by learning analytics. The present experimental study (n = 40) explored the effect of two learning analytics tools (the Concept Trail and Progress Statistics) that give information about students' cognitive activities. The results showed that when teachers had access to learning analytics, they were not better at detecting problematic groups, but they did offer more support in general, and more specifically targeted groups that experienced problems. This could indicate that learning analytics increase teachers' confidence to act, which in turn means students could benefit more from the teacher's presence.     

From signals to knowledge: A conceptual model for multimodal learning analytics

Multimodality in learning analytics and learning science is under the spotlight. The landscape of sensors and wearable trackers that can be used for learning support is evolving rapidly, as well as data collection and analysis methods. Multimodal data can now be collected and processed in real time at an unprecedented scale. With sensors, it is possible to capture observable events of the learning process such as learner's behaviour and the learning context. The learning process, however, consists also of latent attributes, such as the learner's cognitions or emotions. These attributes are unobservable to sensors and need to be elicited by human‐driven interpretations. We conducted a literature survey of experiments using multimodal data to frame the young research field of multimodal learning analytics. The survey explored the multimodal data used in related studies (the input space) and the learning theories selected (the hypothesis space). The survey led to the formulation of the Multimodal Learning Analytics Model whose main objectives are of (O1) mapping the use of multimodal data to enhance the feedback in a learning context; (O2) showing how to combine machine learning with multimodal data; and (O3) aligning the terminology used in the field of machine learning and learning science.     

Applying learning analytics dashboards based on process-oriented feedback to improve students' learning effectiveness

With the development of a technology-supported environment, it is plausible to provide rich process-oriented feedback in a timely manner. In this paper, we developed a learning analytics dashboard (LAD) based on process-oriented feedback in iTutor to offer learners their final scores, sub-scale reports, and corresponding suggestions on further learning content. We adopted a quasi-experimental design to investigate the effectiveness of the report on students' learning. Ninety-four freshman from two classes participated in this research. The two classes were divided into the LAD group and the original analytics report (OAR) based on a product-oriented feedback group. Before the experiment, all the students took the prior knowledge assessment. After a semester's instruction, all the students took the post-test of the summative assessment. Results indicated that students in the LAD group experienced better learning effectiveness than students in the OAR group. LAD based on process-oriented feedback was also effective in improving the skill learning effectiveness of the students with low-level prior knowledge.     

Interplay of prior knowledge,self‐regulation and motivation in complex multimedia learning environments

This study examined the direct and indirect effects of medical clerkship students' prior knowledge, self‐regulation and motivation on learning performance in complex multimedia learning environments. The data from 386 medical clerkship students from six medical schools were analysed using structural equation modeling. The structural model revealed that medical students' prior knowledge directly positively affected their learning outcome, self‐efficacy and performance approach goal orientation. The learners' self‐regulation had a significant positive direct effect on learning outcome. The learners' mastery goal orientation directly affected their learning outcome. Interestingly, inconsistent with our hypothesis, the learners' performance approach goal orientation showed a significant negative direct effect on learning outcome, and performance avoidance goal orientation had a significant positive effect on learning outcome. These findings help develop a more comprehensive understanding of the role of individual characteristics on learning performance of complex tasks in multimedia learning environments.     

Towards an analysis framework for investigating students' engagement and learning in educational blogs

Educational blogs are currently gaining in popularity in schools and higher education institutions, and they are widely promoted as collaborative tools supporting students' active learning. However, literature review on educational blogging revealed a lack of a complete and consistent framework for studying and assessing students' engagement and the impact of blogging on students' learning. This study reports on the application of an analysis framework for evaluating blogging learning activities, based on two well‐documented models, those of Community of Inquiry (CoI) and Social Network Analysis. The framework proposed is examined through an empirical study involving 21 K‐9 students, coming from two classes, in Greece. This investigation shed light into the different ways of students' engagement in a blog‐based project, namely their social and cognitive presence that supported the development of a CoI and learning. The results suggest that the students, through their different roles in the blog, achieved higher thinking and cognitive levels.     

Practices with technology: learning at the boundary between study and work

A study of part‐time student experience of university courses delivered using a range of technologies found that information and communication technology enabled students to move between study and work experience to the benefit of their learning in both contexts. Technology‐based study activities enabled students to participate in learning both as a student and as a member of a practice or work context. Given the increasingly pressured lives of all students in higher education and their aspirations for employment after graduation, this suggests that we would benefit from taking their relationship to work and professional practice into account more directly, in deciding how to integrate technology into their study experience. Teacher conceptions of technology as a tool primarily for information delivery and discussion need to expand to recognize that it can be used to construct learning experiences situated in roles, skills and interactive environments that enhance students' ability to make transitions across the boundaries between contexts of study and work.     

Facilitating application of language skills in authentic environments with a mobile learning system

We uncovered two critical issues in earlier studies: (a) some studies have shown that mobile learning technology is not beneficial for all students due to complexity of learning environments and student prior knowledge, skills, and experience and (b) familiarity of students with the authentic environments in which they learn using mobile technology did not receive much attention in earlier studies. To address these issues, we designed three learning tasks for a class of 26 junior high school students. The students applied language skills by completing the tasks in authentic environments individually in a first task, loosely collaborating with peers in a second task, and tightly collaborating with peers in a third task. A mobile learning system was also designed in this study to support students to accomplish the tasks. The aim of this study was to explore students' learning experiences using the learning system, their perceptions towards the system, and to assess how differently the students perform on the three tasks. According to our design, in the first task, the students took pictures of objects and described them orally or in writing using the mobile learning system. In the second task, after the students completed assignments, each student received comments from a partner through the system. In comments, the partner indicated flaws in student assignments and suggested how to fix them. In the third task, the students completed assignments, shared them using the system, and then exchanged comments with their partners face to face regarding issues related to their completed assignments and suggested how to improve them. Such learning behaviours in the three tasks enabled the students to practise writing and speaking skills. Our results demonstrate that most of the students highly valued our learning system and intend to use it in the future. Furthermore, the results show that the students performed best when they collaborated; namely, student performance was enhanced the most after the third task that required tight collaboration. Based on our results, we learned that students' familiarity with authentic environments is very important and beneficial for their learning. In addition, we learned that even in complex environment, less skilled and experienced students with low prior knowledge can perform well when they tightly collaborate with more skilled and experienced students with high prior knowledge, and our learning system can facilitate such collaboration.     

Analytics of learning tactics and strategies in an online learnersourcing environment

Background

The use of crowdsourcing in a pedagogically supported form to partner with learners in developing novel content is emerging as a viable approach for engaging students in higher-order learning at scale. However, how students behave in this form of crowdsourcing, referred to as learnersourcing, is still insufficiently explored.

Objectives

To contribute to filling this gap, this study explores how students engage with learnersourcing tasks across a range of course and assessment designs.

Methods

We conducted an exploratory study on trace data of 1279 students across three courses, originating from the use of a learnersourcing environment under different assessment designs. We employed a new methodology from the learning analytics (LA) field that aims to represent students' behaviour through two theoretically-derived latent constructs: learning tactics and the learning strategies built upon them.

Results

The study's results demonstrate students use different tactics and strategies, highlight the association of learnersourcing contexts with the identified learning tactics and strategies, indicate a significant association between the strategies and performance and contribute to the employed method's generalisability by applying it to a new context.

Implications

This study provides an example of how learning analytics methods can be employed towards the development of effective learnersourcing systems and, more broadly, technological educational solutions that support learner-centred and data-driven learning at scale. Findings should inform best practices for integrating learnersourcing activities into course design and shed light on the relevance of tactics and strategies to support teachers in making informed pedagogical decisions.     

Enhancing community of inquiry and reflective thinking skills of undergraduates through using learning analytics-based process feedback

This research aims to investigate into the effect of using learning analytics (LA)-based process feedback on students' perceptions of community of inquiry (teaching, social and cognitive presence) and their reflective thinking skills. By using a mixed-method research approach (QUAN + qual), this study was conducted as an experimental design with the pretest–posttest control group. A total of 104 university students who were randomly assigned to the experiment group (EG) and control group (CG) were recruited in this study. The procedure was conducted within the scope of the computing course based on the flipped classroom (FC) model. While the participants in the EG received LA-based process feedback which shows their LA results in a weekly manner, those in the CG did not get any LA-based process feedback. The data were collected through the Community of Inquiry Scale, the Reflective Thinking Scale and a semi-structured student opinion form. The findings indicated that sending feedback including the students' LA results had a statistically significant effect on the students' perceptions of community of inquiry and reflective thinking skills. Based on the findings of the study, several recommendations for teachers, instructional designers and researchers have been made.     

Predicting students' knowledge after playing a serious game based on learning analytics data: A case study

Serious games have proven to be a powerful tool in education to engage, motivate, and help students learn. However, the change in student knowledge after playing games is usually measured with traditional (paper) prequestionnaires–postquestionnaires. We propose a combination of game learning analytics and data mining techniques to predict knowledge change based on in-game student interactions. We have tested this approach in a case study for which we have conducted preexperiments–postexperiments with 227 students playing a previously validated serious game on first aid techniques. We collected student interaction data while students played, using a game learning analytics infrastructure and the standard data format Experience API for Serious Games. After data collection, we developed and tested prediction models to determine whether knowledge, given as posttest results, can be accurately predicted. Additionally, we compared models both with and without pretest information to determine the importance of previous knowledge when predicting postgame knowledge. The high accuracy of the obtained prediction models suggests that serious games can be used not only to teach but also to measure knowledge acquisition after playing. This will simplify serious games application for educational settings and especially in the classroom easing teachers' evaluation tasks.