Predicting student performance in a blended MOOC

Predicting student performance is a major tool in learning analytics. This study aims to identify how different measures of massive open online course (MOOC) data can be used to identify points of improvement in MOOCs. In the context of MOOCs, student performance is often defined as course completion. However, students could have other learning objectives than MOOC completion. Therefore, we define student performance as obtaining personal learning objective(s). This study examines a subsample of students in a graduate‐level blended MOOC who shared on‐campus course completion as a learning objective. Aggregated activity frequencies, specific course item frequencies, and order of activities were analysed to predict student performance using correlations, multiple regressions, and process mining. All aggregated MOOC activity frequencies related positively to on‐campus exam grade. However, this relation is less clear when controlling for past performance. In total, 65% of the specific course items showed significant correlations with final exam grade. Students who passed the course spread their learning over more days compared with students who failed. Little difference was found in the order of activities within the MOOC between students who passed and who failed. The results are combined with course evaluations to identify points of improvement within the MOOC.     

Creating collaborative groups in a MOOC: a homogeneous engagement grouping approach

ABSTRACT

Collaborative learning can improve the pedagogical effectiveness of MOOCs. Group formation, an essential step in the design of collaborative learning activities, can be challenging in MOOCs given the scale and the wide variety in such contexts. We discuss the need for considering the behaviours of the students in the course to form groups in MOOC contexts, and propose a grouping approach that employs homogeneity in terms of students’ engagement in the course. Two grouping strategies with different degrees of homogeneity are derived from this approach, and their impact to form successful groups is examined in a real MOOC context. The grouping criteria were established using student activity logs (e.g. page-views). The role of the timing of grouping was also examined by carrying out the intervention once in the first and once in the second half of the course. The results indicate that in both interventions, the groups formed with a greater degree of homogeneity had higher rates of task-completion and peer interactions, Additionally, students from these groups reported higher levels of satisfaction with their group experiences. On the other hand, a consistent improvement of all indicators was observed in the second intervention, since student engagement becomes more stable later in the course.     

Dissecting learning tactics in MOOC using ordered network analysis

Background

Select and enact appropriate learning tactics that advance learning has been considered a critical set of skills to successfully complete highly flexible online courses, such as Massive open online courses (MOOCs). However, limited by analytic methods that have been used in the past, such as frequency distribution, sequence mining and process mining, we lack a deep, complete and detailed understanding of the learning tactics used by MOOC learners.

Objectives

In the present study, we proposed four major dimensions to better interpret and understand learning tactics, which are frequency, continuity, sequentiality and role of learning actions within tactics. The aim of this study was to examine to what extent can a new analytic technique, the ordered network analysis (ONA), deepen the understanding of MOOC learning tactics compared to using other methods.

Methods

In particular, we performed a fine-grained analysis of learning tactics detected from more than 4 million learning events in the behavioural trace data of 8788 learners who participated in a large-scale MOOC ‘Flipped Classroom’.

Results and Conclusions

We detected eight learning tactics, and then chose one typical tactic as an example to demonstrate how the ONA technique revealed all four dimensions and provided deeper insights into this MOOC learning tactic. Most importantly, based on the comparison with different methods such as process mining, we found that the ONA method provided a unique opportunity and novel insight into the roles of different learning actions in tactics which was neglected in the past.

Takeaway

In summary, we conclude that ONA is a promising technique that can benefit the research on learning tactics, and ultimately benefit MOOC learners by strengthening the strategic support.     

Linking learner factors,teaching context,and engagement patterns with MOOC learning outcomes

Prior studies of MOOC learners have focused almost exclusively on behavioural and social aspects of engagement. This paper extends the scope of previous studies by adopting a multi-dimensional, person-centred approach to investigate learner engagement in MOOCs. An analysis of 1,452 self-administered survey responses uncovered three prototypical categories of MOOC learners based on patterns of behavioural, cognitive, emotional and social engagement: (a) “Individually Engaged” learners, (b) “Least Engaged” learners and (c) “Wholly Engaged” learners. The study revealed significant differences among the three cohorts of MOOC participants with respect to learner factors (gender, origin, motivation), teaching context (course level, course duration, form of the assessment) and learning outcomes (course completion, perceived quality of instruction). The results of this study suggest that adopting a multi-dimensional, person-centred approach can be useful for researchers and practitioners to classify MOOC learners into subpopulations, design effective educational interventions that best engage different types of learners, and provide support and scaffolding to individuals with idiosyncratic or problematic engagement patterns.     

The effects of assessment design on academic dishonesty,learner engagement,and certification rates in MOOCs

Background

Massive Open Online Courses (MOOCs) have touted the idea of democratizing education, but soon enough, this utopian idea collided with the reality of finding sustainable business models. In addition, the promise of harnessing interactive and social web technologies to promote meaningful learning was only partially successful. And finally, studies demonstrated that many learners exploit the anonymity and feedback to earn certificates unethically. Thus, establishing MOOC pedagogical models that balance open access, meaningful learning, and trustworthy assessment remains a challenge that is crucial for the field to achieve its goals.

Objectives

This study analysed the influence of an MOOC assessment model, denoted the Competency Exam (CE), on learner engagement, the level of cheating, and certification rates. At its core, this model separates learning from for-credit assessment, and it was introduced by the MITx Biology course team in 2016.

Methods

We applied a learning analytics methodology to the clickstream data of the verified learners (N = 559) from four consecutive runs of an Introductory Biology MOOC offered through edX. The analysis used novel algorithms for measuring the level of cheating and learner engagement, which were developed in the previous studies.

Results and Conclusions

On the positive side, the CE model reduced cheating and did not reduce learner engagement with the main learning materials – videos and formative assessment items. On the negative side, it led to procrastination, and certification rates were lower.

Implications

First, the results shed light on the fundamental connection between incentive design and learner behaviour. Second, the CE provides MOOC designers with an ‘analytically verified’ model to reduce cheating without compromising on open access. Third, our methodology provides a novel means for measuring cheating and learner engagement in MOOCs.

     

Analytics of learning tactics and strategies in an online learnersourcing environment

Background

The use of crowdsourcing in a pedagogically supported form to partner with learners in developing novel content is emerging as a viable approach for engaging students in higher-order learning at scale. However, how students behave in this form of crowdsourcing, referred to as learnersourcing, is still insufficiently explored.

Objectives

To contribute to filling this gap, this study explores how students engage with learnersourcing tasks across a range of course and assessment designs.

Methods

We conducted an exploratory study on trace data of 1279 students across three courses, originating from the use of a learnersourcing environment under different assessment designs. We employed a new methodology from the learning analytics (LA) field that aims to represent students' behaviour through two theoretically-derived latent constructs: learning tactics and the learning strategies built upon them.

Results

The study's results demonstrate students use different tactics and strategies, highlight the association of learnersourcing contexts with the identified learning tactics and strategies, indicate a significant association between the strategies and performance and contribute to the employed method's generalisability by applying it to a new context.

Implications

This study provides an example of how learning analytics methods can be employed towards the development of effective learnersourcing systems and, more broadly, technological educational solutions that support learner-centred and data-driven learning at scale. Findings should inform best practices for integrating learnersourcing activities into course design and shed light on the relevance of tactics and strategies to support teachers in making informed pedagogical decisions.     

Facilitating Students’ Interaction in MOOCs through Timeline-Anchored Discussion

ABSTRACT

Interaction is central to any learning experience. Currently, Massive Open Online Courses (MOOCs) rely on discussion forums as a primary means of interaction among learners and instructors. Threaded discussions of such forums help learners to hold an ongoing and recursive discussion over an extended period, but the lack of temporal and spatial contiguity makes it insufficient to deliver a smooth interaction experience. To facilitate such interaction, this study designed DanMOOC, a commenting tool that facilitates timeline-anchored discussion among MOOC learners and instructors. DanMOOC was the first to combine threaded discussions with Danmaku commenting, a video commenting feature that allows viewers of the same video to share comments at the top of the video screen. The design process followed an iterative process of user-centered design. The impact of the new design on learning was assessed empirically through a laboratory experiment comparing DanMOOC with the current MOOC system (video-based learning + forum). The results show that DanMOOC increases learners’ perceptions of social, teaching, and cognitive presence, engages learners more in discussion, and improves learners’ satisfaction with both the course and the platform.     

Mobile learning: Two case studies of supporting inquiry learning in informal and semiformal settings

Mobile technologies can support learning across different contexts as their portability enables them to be used by the learner in whichever context she or he is in. They can be particularly beneficial in informal and semiformal contexts where learners have more control over their learning goals and where motivation is often high. Inquiries in informal contexts are likely to be personally relevant in terms of topics of interest and capitalise on learners' location as learners decide what, where, when and whether to learn. There is considerable interest in how such benefits can be harnessed for more formal learning and one challenge is how to make inquiries personally relevant in such contexts. However, there is little literature that considers the structure needed to support informal and semiformal inquiry learning. This paper contributes to that literature by examining dimensions for researchers and designers to consider investigating or developing support for inquiries in informal or semiformal settings.The paper examines two case studies of inquiry learning in contrasting settings in order to understand more about learner control and how technology can support learners' inquiries. Case study one considers the use of web based software to support science inquiry learning by 14–15 year olds in a semiformal context, whilst the second case study reports on informal adult learners using their own mobile technologies to learn about landscape. These case studies are compared and contrasted in terms of the dimensions of learner control, location of learning, and the different support mechanisms for inquiry learning and a framework is proposed for considering these dimensions.     

Analysing successful massive open online courses using the community of inquiry model as perceived by students

This research examines the characteristics that contributed to the success of massive open online courses (MOOCs) in the fields of software, sciences, and management using data mining and semantic analysis together with content analysis. A total of 3,460 reviews regarding 5 different MOOCs that received a 5/5 grade were extracted from the CourseTalk website and analysed according to the community of inquiry model. It was found that, as well as in academic online courses, the characteristics that contributed to MOOCs' success were distributed between all 3 presence elements according to the community of inquiry model: teaching (36%), social (23%), and cognitive (36%; and technological [5%]). This is contrary to the perception that MOOCs mostly contain teaching presence elements. The four leading characteristics were teacher, exercise, atmosphere, and workload. Cluster analysis resulted in 5 types of students with similar presence element preferences. This shows that successful MOOCs enable students with different preferences to consume content and activities according to their individual preferences. These findings could be the base of future research on the subject of adapting MOOC activities and content to students' varied preferences, as well as further understanding the characteristics that contribute to successful MOOCs or other fully online courses.     

A study of building a resource-based learning environment with the inquiry learning approach: Knowledge of family trees

This study aims to provide teachers with ways and means to facilitate learners to develop nomenclature knowledge of family trees through the establishment of resource-based learning environments (RBLEs). It discusses the design of an RBLE in the classroom by selecting an appropriate context with the assistance of computer-mediated learning resources and tools and employing the inquiry learning approach as the pedagogy. This study reports on the creation of the RBLE within the learning context of family trees. The computer-mediated learning resources and tools comprise three components: an audio-visual database for guided and coupled inquiry, an interactive interface for conceptualising the nomenclature and a tool for learners to construct their own family trees. Scaffolds were designed for an inquiry mode of learning and teaching to support the use of the resources and tools in learning about family trees. The learning and teaching process, including the outcomes for learners, through the RBLE with inquiry learning approach are studied. The findings of an interview and a pre-test/post-test study indicate that the RBLE can assist learners to build knowledge of family trees. The role of teachers in such an environment is to guide and encourage learners to inquire during the learning process.     

深度学习视阈下MOOC学习者流失预测及干预研究

MOOC（Massive Open Online Courses）在为学习者提供优质课程的同时，低完成率成为影响其有效推广的重要因素。通过对edX开放数据集分析发现，学习者的逐渐流失是导致MOOC课程低完成率的因素之一，且学习行为与成绩之间存在复杂的相关性；基于线性回归和深度神经网络（Deep Neural Network，DNN）预测学习者的成绩，实验证明，DNN能够更好地拟合学习行为与成绩之间复杂的相关性，实现对成绩更加精准的预测，预警学习者流失；对预测的潜在流失学习者迭代进行个性化的教学干预，提高MOOC课程的完成率。     

基于深度学习的MOOCs辍学率预测方法

近年来大规模开放在线课程获得了较为广泛的关注。由于学习者学习方式不合理使得学习兴趣下降,学习效果不佳,MOOCs辍学率很高,针对这一问题,从学习者学习活动日志中自动抽取一段时间内连续特征,以学习者行为特征为自变量,建立MOOCs辍学预测模型。在KDD Cup 2015数据集上的实验表明,使用基于卷积神经网络的长短期记忆CNN_LSTM辍学预测模型,能够帮助MOOCs课程教师和设计者追踪课程学习者在不同时间步长的学习状态,从而动态监控不同阶段的辍学行为,模型的预测准确率高,这将为教师改进教学方法提供更合理的指导和建议。     

Virtual reality in problem-based learning contexts: Effects on the problem-solving performance,vocabulary acquisition and motivation of English language learners

Learning a foreign language requires interaction with language input while involved in a task. Given that problem-based learning (PBL) offers hands-on application in realistic contexts, and that virtual reality (VR) enables learners to interact with multiple modalities of information, this study examines how the integration of VR technology into PBL contexts affects students' motivation for, problem-solving during, and vocabulary acquisition in learning English as a foreign language (EFL). A total of 84 engineering majors who enrolled in a course of English for specific purposes were randomly assigned to either an experimental group or a control group. Students in the experimental group participated in a VR-assisted PBL context, in which they were to view a PBL scenario using VR technology and then to create VR videos about solving the given problems. Those in the control group participated in a PBL context without the use of VR technology for viewing and solving the identical scenario. After the intervention, all the students wrote a problem-solving analysis, took a vocabulary knowledge test, completed a learning motivation questionnaire, and participated in individual interviews. The results showed that the students in the experimental group significantly outperformed those in the control group in terms of vocabulary acquisition, and were more motivated to learn English related to their future careers, whereas there was no significant difference in the problem-solving performance of the two groups. Implications of these findings highlight the value of engaging EFL learners in immersive environments for contextualized learning through the integrated use of VR and PBL.     

Natural,practical and social contexts of e‐learning: a critical realist account for learning and technology

Much of the research in educational technology with a primary concern over how technology enhances learning has been criticized as privileging the immediate learning settings over the other dimensions of learners’ social life and the wider social and economic contexts in which learning and technology are located. The ability to develop a rich understanding of learning and technology in various contexts requires careful use of theory that might enable ‘looking beyond learning’. To meet this need, this paper proposes the use of a critical realistic account of learning and technology, in particular Margaret Archer's ‘three orders of reality’ and ‘personal Identity’. Drawing on an empirical case study for illustration, the paper argues that such an approach offers a ‘way out’ to unpack the relationship between learning and technology through a deep exploration of the interrelationship between individual practices of learning, with or without technology, and the bigger picture of how learning intervenes in other dimensions of social life and how other contexts come into play in learning settings.     

Gamification: A key determinant of massive open online course (MOOC) success

Massive open online courses (MOOCs), contribute significantly to individual empowerment because they can help people learn about a wide range of topics. To realize the full potential of MOOCs, we need to understand their factors of success, here defined as the use, user satisfaction, along the individual and organizational performance resulting from the user involvement. We propose a theoretical framework to identify the determinants of successful MOOCs, and empirically measure these factors in a real MOOC context. We put forward the role of gamification and suggest that, together with information system (IS) theory, gamification proved to play a crucial role in the success of MOOCs.     

Investigating the adoption of MOOCs: A technology–user–environment perspective

In spite of the potentials promised by MOOCs (massive open online courses), the adoption rate of MOOCs is still low, especially in developing countries. Research on the adoption of MOOCs in developing countries is also limited. To fill this research gap, this research aims to study the adoption of MOOCs by extending current research on innovation and technology adoption in order to promote the diffusion of MOOCs in developing countries. Here, a conceptual framework, technology–user–environment was proposed to identify influential drivers and barriers of MOOC adoption. Eight hundred and twenty‐seven individuals in a developing country, were invited to participate in a questionnaire survey. Data were analysed by hierarchical regression, and the results show that for the (a) technological characteristics, perceived usefulness has the most significant influence, followed by lack of accessibility and performance‐to‐cost value; for the (b)user factors, self‐regulation was the most significant predictor of MOOC adoption, and female learners showed higher willingness to adopt MOOCs; and for the (c) social environmental factors, tradition and social norms and lack of information about MOOCs were negatively associated with adoption of MOOCs. Implications and suggestions are discussed at the end of this paper.     

Do learners’ characteristics matter? An exploration of mobile-learning adoption in self-directed learning

This paper aims to identify individual characteristics that motivate learners to use mobile-learning. It sheds light on our current knowledge by a) examining an m-learning adoption model which accounts for learners’ characteristics (learning style and personal innovativeness) in addition to previously studied mobile platform characteristics and b) considering the context in which learning occurs (formal and informal). A framework has been introduced and empirically tested. Results suggest that individuals’ learning style and perceived playfulness influence m-learning usage in both learning situations; while performance expectancy and personal innovativeness are only influential in specific learning contexts. This study highlights the role of learners’ characteristics in m-learning adoption and emphasizes the importance of distinguishing between various types of m-learning. This multi-disciplinary research enriches m-learning literature and offers practical implications for educators using mobile technologies as well as developers of virtual learning platforms.     

Effects of anxiety levels on learning performance and gaming performance in digital game‐based learning

Anxiety plays an influential role in foreign language learning. However, a lack of attention was paid to examining the effects of anxiety levels on learning performance and gaming performance in digital game‐based learning. To this end, this study developed a game‐based English learning system and investigated how different levels of anxiety affected learners' learning performance and gaming performance. A quasi‐experiment was conducted in an elementary school. The results showed that high‐anxiety learners performed worse than low‐anxiety learners in speaking, word/sentence match, and overall learning performance. However, they performed similarly in listening performance. Moreover, the results showed that high‐ and low‐anxiety learners demonstrated a similar level of gaming performance. A subsequent analysis showed that significant correlations existed between learning performance and gaming performance for learners with high anxiety whereas such positive correlations were rarely found for learners with low anxiety, indicating that high‐anxiety learners' learning performance could be fostered by their gaming performance. The findings suggested that digital game‐based learning was particularly beneficial to high‐anxiety learners, whose gaming performance was a facilitative factor of their learning performance.