Dissecting learning tactics in MOOC using ordered network analysis

Background

Select and enact appropriate learning tactics that advance learning has been considered a critical set of skills to successfully complete highly flexible online courses, such as Massive open online courses (MOOCs). However, limited by analytic methods that have been used in the past, such as frequency distribution, sequence mining and process mining, we lack a deep, complete and detailed understanding of the learning tactics used by MOOC learners.

Objectives

In the present study, we proposed four major dimensions to better interpret and understand learning tactics, which are frequency, continuity, sequentiality and role of learning actions within tactics. The aim of this study was to examine to what extent can a new analytic technique, the ordered network analysis (ONA), deepen the understanding of MOOC learning tactics compared to using other methods.

Methods

In particular, we performed a fine-grained analysis of learning tactics detected from more than 4 million learning events in the behavioural trace data of 8788 learners who participated in a large-scale MOOC ‘Flipped Classroom’.

Results and Conclusions

We detected eight learning tactics, and then chose one typical tactic as an example to demonstrate how the ONA technique revealed all four dimensions and provided deeper insights into this MOOC learning tactic. Most importantly, based on the comparison with different methods such as process mining, we found that the ONA method provided a unique opportunity and novel insight into the roles of different learning actions in tactics which was neglected in the past.

Takeaway

In summary, we conclude that ONA is a promising technique that can benefit the research on learning tactics, and ultimately benefit MOOC learners by strengthening the strategic support.     

Where is the teacher? Digital analytics for classroom proxemics

The term Classroom Proxemics refers to how teachers and students use classroom space, and the impact of this and the spatial design on learning and teaching. This study addresses the divide between, on the one hand, substantial work on proxemics based on classroom observations and, on the other hand, emerging work to design automated feedback that helps teachers identify salient patterns in their use of the classroom space. This study documents how digital analytics were designed in service of a senior teacher's practice-based inquiry into classroom proxemics. Indoor positioning data from four teachers were analysed, visualized and used as evidence to compare three distinct learning designs enacted in a physics classroom. This study demonstrates how teachers can make effective use of such visualizations, to gain insight into their classroom practice. This is evidenced by (a) documenting teachers' reflections on visualizations of positioning data, both their own and that of peers and (b) identifying the types of indicator (operationalized as analytical metrics) that foreground the most useful information for teachers to gain insight into their practice.     

Tracing elementary school students’ study tactic use in gStudy by examining a strategic and self-regulated learning

This study investigated, with the help of log file traces (f = 172), how 20 elementary school students used study tactics when studying science within the gStudy learning environment and examined how tactic use contributed to the students’ achievement. The analysis of this study is divided into two parts. First, at the situational level, the focus is on capturing the tactics that were used in different gStudy sessions, classifying the gStudy sessions based on the tactic use, and illustrating the patterned use of tactics during these sessions. Second, at the individual level, the focus is on examining individual students’ typical methods of using tactics, which helps to illustrate how tactic use contributes to the students’ achievement. The gStudy sessions were classified into three categories on the basis of tactic use: rare, moderate, and frequent. Findings indicate that frequent tactic use did not contribute to deep learning. Moderate tactic use was fairly effective for learning, but rare tactic use contributed to deep learning. The results did not show that the use of many study tactics improves learning; rather, they suggest that the distinguishing feature in strategic learning is not the tactic use itself but the way the tactic is performed.     

Patterns of students' collaborations by variations in their learning orientations in blended course designs: How is it associated with academic achievement?

Background

While a number of learner factors have been identified to impact students' collaborative learning, there has been little systematic research into how patterns of students' collaborative learning may differ by their learning orientations.

Objectives

This study aimed to investigate: (1) variations in students' learning orientations by their conceptions, approaches, and perceptions; (2) the patterns of students' collaborations by variations in their learning orientations and (3) the contribution of patterns of collaborations to academic achievement.

Methods

A cohort of 174 Chinese undergraduates in a blended engineering course were surveyed for their conceptions of learning, approaches to learning and to using online learning technologies, and perceptions of e-learning, to identify variations in their learning orientations. Students' collaborations and mode of collaborations were collected through an open-ended social network analysis (SNA) questionnaire.

Results and Conclusions

A hierarchical cluster analysis identified an ‘understanding’ and ‘reproducing’ learning orientations. Based on students' learning orientations and their choices to collaborate, students were categorized into three mutually exclusive collaborative group, namely Understanding Collaborative group, Reproducing Collaborative group and Mixed Collaborative group. SNA centrality measures demonstrated that students in the Understanding Collaborative group had more collaborations and stayed in a better position in terms of capacity to gather information. Both students' approaches to learning and students' average collaborations significantly contributed to their academic achievement, explaining 3% and 4% of variance in their academic achievement respectively. The results suggest that fostering a desirable learning orientation may help improve students' collaborative learning.     

Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics

By collaboratively solving a task, students are challenged to share ideas, express their thoughts, and engage in discussion. Collaborating groups of students may encounter problems concerning cognitive activities (such as a misunderstanding of the task material). If these problems are not addressed and resolved in time, the collaborative process is hindered. The teacher plays an important role in monitoring and solving the occurrence of problems. To provide adaptive support, teachers continuously have to be aware of students' activities in order to identify relevant events, including those that require intervention. Because the amount of available information is high, teachers may be supported by learning analytics. The present experimental study (n = 40) explored the effect of two learning analytics tools (the Concept Trail and Progress Statistics) that give information about students' cognitive activities. The results showed that when teachers had access to learning analytics, they were not better at detecting problematic groups, but they did offer more support in general, and more specifically targeted groups that experienced problems. This could indicate that learning analytics increase teachers' confidence to act, which in turn means students could benefit more from the teacher's presence.     

Analytics of time management strategies in a flipped classroom

This paper aims to explore time management strategies followed by students in a flipped classroom through the analysis of trace data. Specifically, an exploratory study was conducted on the dataset collected in three consecutive offerings of an undergraduate computer engineering course (N = 1,134). Trace data about activities were initially coded for the timeliness of activity completion. Such data were then analysed using agglomerative hierarchical clustering based on Ward's algorithm, first order Markov chains, and inferential statistics to (a) detect time management tactics and strategies from students' learning activities and (b) analyse the effects of personalized analytics-based feedback on time management. The results indicate that meaningful and theoretically relevant time management patterns can be detected from trace data as manifestations of students' tactics and strategies. The study also showed that time management tactics had significant associations with academic performance and were associated with different interventions in personalized analytics-based feedback.     

Analysis of learners' navigational behaviour and their learning styles in an online course

Providing adaptive features and personalized support by considering students' learning styles in computer‐assisted learning systems has high potential in making learning easier for students in terms of reducing their efforts or increasing their performance. In this study, the navigational behaviour of students in an online course within a learning management system was investigated, looking at how students with different learning styles prefer to use and learn in such a course. As a result, several differences in the students' navigation patterns were identified. These findings have several implications for improving adaptivity. First, they showed that students with different learning styles use different strategies to learn and navigate through the course, which can be seen as another argument for providing adaptivity. Second, the findings provided information for extending the adaptive functionality in typical learning management systems. Third, the information about differences in navigational behaviour can contribute towards automatic detection of learning styles, helping in making student modeling approaches more accurate.     

A web-based learning tool improves student performance in statistics: A randomized masked trial

Background

e-status is a web-based tool able to generate different statistical exercises and to provide immediate feedback to students’ answers. Although the use of Information and Communication Technologies (ICTs) is becoming widespread in undergraduate education, there are few experimental studies evaluating its effects on learning.

Method

All of the students (121) from an introductory course for statistics in dentistry were randomly assigned to use the tool with one of two 6-problem sets, known as types A and B. The primary endpoint was the grade difference obtained in the final exam, composed of two blocks of questions related to types A and B. The exam evaluator was masked to the intervention group.

Results

We found that the effect of e-status on the student grade was an improvement of 0.48 points (95% CI: 0.10–0.86) on a ten-point scale. Among the 94 students who actually employed e-status, the effect size was 0.63 (95% CI: 0.17–1.10).

Conclusions

It is feasible to formally assess the learning effect of an innovative tool. Providing e-status exercises to students has a direct effect on learning numerical operations related to statistics. Further effects on higher cognitive levels still have to be explored.     

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.     

Applying learning analytics dashboards based on process-oriented feedback to improve students' learning effectiveness

With the development of a technology-supported environment, it is plausible to provide rich process-oriented feedback in a timely manner. In this paper, we developed a learning analytics dashboard (LAD) based on process-oriented feedback in iTutor to offer learners their final scores, sub-scale reports, and corresponding suggestions on further learning content. We adopted a quasi-experimental design to investigate the effectiveness of the report on students' learning. Ninety-four freshman from two classes participated in this research. The two classes were divided into the LAD group and the original analytics report (OAR) based on a product-oriented feedback group. Before the experiment, all the students took the prior knowledge assessment. After a semester's instruction, all the students took the post-test of the summative assessment. Results indicated that students in the LAD group experienced better learning effectiveness than students in the OAR group. LAD based on process-oriented feedback was also effective in improving the skill learning effectiveness of the students with low-level prior knowledge.     

An exploration of students' strategy use in inquiry-based computer-supported collaborative learning

Abstract The aim of this study is to investigate students' use of cognitive learning strategies in inquiry-based computer-supported collaborative learning (CSCL). A process-oriented interview framework on cognitive activity, self-regulation and motivation, and a coding category for analysing cognitive learning strategies and cognitive self-regulation was developed. The students of an intervention group (n=18) participating in inquiry-based CSCL and a comparison group (n=8) were interviewed six to eight times during the 3 years of the study. The results derived from the mixed-method analysis of altogether 161 interviews were compared between the two groups. The results indicate that the students who participated in the inquiry-based CSCL activities reported deeper-level cognitive strategies such as monitoring, creating representations and sharing information collaboratively. The students of the comparison group reported more surface-level strategies such as memorization. However, the findings concerning the utility of CSCL inquiry on cognitive learning strategies were not uniformly positive. It was found that the students of the comparison group reported significantly more strategies under the category of content evaluation. Nevertheless, the results suggest that computer-supported inquiry-based learning can enhance the use of cognitive strategies that support learning.     

Do zoom meetings really help? A comparative analysis of synchronous and asynchronous online learning during Covid-19 pandemic

Background

Online learning and teaching were globally popularized due to the impact of Covid-19. The pandemic has made both synchronous and asynchronous online learning inevitable in regions privileged with the technological affordance.

Aims

This study was designed to examine and compare the effectiveness of both learning modes through the Community of Inquiry framework.

Materials & Methods

Comparative analyses on a sample of N = 170 undergraduate students who took both synchronous and asynchronous online courses in Spring 2021.

Results

The paired-sample T-tests results indicated a significant difference in social presence, cognitive presence and self-evaluated performance.

Discussion & Conclusion

Teaching presence significantly influenced social presence and cognitive presence in both learning modes. However, under synchronous learning mode, social presence significantly impacted self-evaluation, grades and school identification. While social presence only influenced school identification under asynchronous learning mode. Theoretical and practical implications were also included.     

Does the emotional design of scaffolds enhance learning and motivational outcomes in game-based learning?

Background

In recent years, the importance of emotions in learning has been increasingly recognized. Applying emotional design to induce positive emotions has been considered a means to enhance the instructional effectiveness of digital learning environments. However, only a few studies have examined the specific effects of emotional design in game-based learning.

Objectives

This quasi-experimental study utilized a value-added research approach to investigate whether emotional design applied to scaffolding in a game-based learning environment improves learning and motivational outcomes more than emotionally neutral scaffolding.

Methods

A total of 138 participants, mean age of 11.5 (SD = 0.73) participated in the study. A total of 68 participants played the base version of a fraction learning game (Number Trace), where scaffolding was provided with emotionally neutral mathematical notations, and 70 participants played the value-added version of the game using emotionally designed animated scaffolding agents. Pre-and post-tests were used to measure conceptual fraction knowledge and self-reported measures of situational interest and situational self-efficacy to evaluate motivational outcomes.

Results and Conclusions

Our results indicate that the emotional design applied to scaffolds can improve the educational value of a game-based learning environment by enhancing players' situational interest and situational self-efficacy. However, although the intervention improved the participants' conceptual fraction knowledge, there was no significant difference between the scaffolding conditions in participants' learning outcomes.

Takeaways

The results suggest that emotional design can increase the educational impact of game-based learning by promoting the development of interest, as well as improving self-efficacy.     

From signals to knowledge: A conceptual model for multimodal learning analytics

Multimodality in learning analytics and learning science is under the spotlight. The landscape of sensors and wearable trackers that can be used for learning support is evolving rapidly, as well as data collection and analysis methods. Multimodal data can now be collected and processed in real time at an unprecedented scale. With sensors, it is possible to capture observable events of the learning process such as learner's behaviour and the learning context. The learning process, however, consists also of latent attributes, such as the learner's cognitions or emotions. These attributes are unobservable to sensors and need to be elicited by human‐driven interpretations. We conducted a literature survey of experiments using multimodal data to frame the young research field of multimodal learning analytics. The survey explored the multimodal data used in related studies (the input space) and the learning theories selected (the hypothesis space). The survey led to the formulation of the Multimodal Learning Analytics Model whose main objectives are of (O1) mapping the use of multimodal data to enhance the feedback in a learning context; (O2) showing how to combine machine learning with multimodal data; and (O3) aligning the terminology used in the field of machine learning and learning science.     

Can the use of cognitive and metacognitive self-regulated learning strategies be predicted by learners’ levels of prior knowledge in hypermedia-learning environments?

Research on self-regulated learning (SRL) in hypermedia-learning environments is a growing area of interest, and prior knowledge can influence how students interact with these systems. One hundred twelve (N = 112) undergraduate students’ interactions with MetaTutor, a multi-agent, hypermedia-based learning environment, were investigated, including how prior knowledge affected their use of SRL strategies. We expected that students with high prior knowledge would engage in significantly more cognitive and metacognitive SRL strategies, engage in different sequences of SRL strategies, spend more time engaging in SRL processes, and visit more pages that were relevant to their sub-goals than students with low prior knowledge. Results showed significant differences in the total use of SRL strategies between prior knowledge groups, and more specifically, revealed significant differences in the use of each metacognitive strategy (e.g., judgment of learning), but not each cognitive strategy (e.g., taking notes) between prior knowledge groups. Results also revealed different sequences of use of SRL strategies between prior knowledge groups, and that students spent different amounts of time engaging in SRL processes; however, all students visited similar numbers of relevant pages. These results have important implications on designing multi-agent, hypermedia environments; we can design pedagogical agents that adapt to students’ learning needs, based on their prior knowledge levels.     

Automatic identification of knowledge-transforming content in argument essays developed from multiple sources

Developing knowledge-transforming skills in writing may help students increase learning by actively building knowledge, regardless of the domain. However, many undergraduate students struggle to transform knowledge when drafting essays based on multiple sources. Writing analytics can be used to scaffold knowledge transforming as writers bring evidence to bear in supporting claims. We investigated how to automatically identify sentences representing knowledge transformation in argumentative essays. A synthesis of cognitive theories of writing and Bloom's typology identified 22 linguistic features to model processes of knowledge transforming in a corpus of 38 undergraduates' essays. Findings indicate undergraduates mostly paraphrase or copy information from multiple sources rather than engage deeply with sources' content. Eight linguistic features were important for discriminating evidential sentences as telling versus transforming source knowledge. We trained a machine learning algorithm that accurately classified nearly three of four evidential sentences as knowledge-telling or knowledge-transforming, offering potential for use in future research.     

Factors influencing beliefs for adoption of a learning analytics tool: An empirical study

Present research and development offer various learning analytics tools providing insights into different aspects of learning processes. Adoption of a specific tool for practice is based on how its learning analytics are perceived by educators to support their pedagogical and organizational goals. In this paper, we propose and empirically validate a Learning Analytics Acceptance Model (LAAM) of factors influencing the beliefs of educators concerning the adoption a learning analytics tool. In particular, our model explains how the usage beliefs (i.e., ease-of-use and usefulness perceptions) about the learning analytics of a tool are associated with the intention to adopt the tool. In our study, we considered several factors that could potentially affect the adoption beliefs: i) pedagogical knowledge and information design skills of educators; ii) educators' perceived utility of a learning analytics tool; and iii) educators' perceived ease-of-use of a learning analytics tool. By following the principles of Technology Acceptance Model, the study is done with a sample of educators who experimented with a LOCO-Analyst tool. Our study also determined specific analytics types that are primary antecedence of perceived usefulness (concept comprehension and social interaction) and ease-of-use (interactive visualization).     

Promoting socially shared regulation of learning in CSCL: Progress of socially shared regulation among high- and low-performing groups

Collaborative groups encounter many challenges in their learning. They need to recognize challenges that may hinder collaboration, and to develop appropriate strategies to strengthen collaboration. This study aims to explore how groups progress in their socially shared regulation of learning (SSRL) in the context of computer-supported collaborative learning (CSCL). Teacher education students (N = 103) collaborated in groups of three to four students during a two-month multimedia course. The groups used the Virtual Collaborative Research Institute (VCRI) learning environment along with regulation tools that prompted them to recognize challenges that might hinder their collaboration and to develop SSRL strategies to overcome these challenges.In the data analysis, the groups reported challenges, and the SSRL strategies they employed were analyzed to specify the focus and function of the SSRL. Process discovery was used to explore how groups progressed in their SSRL. The results indicated that depending on the phase of the course, the SSRL focus and function shifted from regulating external challenges towards regulating the cognitive and motivational aspects of their collaboration. However, the high-performing groups progressed in their SSRL in terms of evidencing temporal variety in challenges and SSRL strategies across time, which was not the case with low performing groups.     

Successful implementation of user-centered game based learning in higher education: An example from civil engineering

Goal

The use of an online game for learning in higher education aims to make complex theoretical knowledge more approachable. Permanent repetition will lead to a more in-depth learning.

Objective

To gain insight into whether and to what extent, online games have the potential to contribute to student learning in higher education.

Experimental setting

The online game was used for the first time during a lecture on Structural Concrete at Master’s level, involving 121 seventh semester students.

Methods

Pre-test/post-test experimental control group design with questionnaires and an independent online evaluation.

Results

The minimum learning result of playing the game was equal to that achieved with traditional methods. A factor called “joy” was introduced, according to [Nielsen, J. (2002): User empowerment and the fun factor. In Jakob Nielsen’s Alertbox, July 7, 2002. Available from http://www.useit.com/alertbox/20020707.html.], which was amazingly high.

Conclusion

The experimental findings support the efficacy of game playing. Students enjoyed this kind of e-learning.