Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.     

Instructing students on effective sequences of examples and problems: Does self-regulated learning improve from knowing what works and why?

Nowadays, students often practice problem-solving skills in online learning environments with the help of examples and problems. This requires them to self-regulate their learning. It is questionable how novices self-regulate their learning from examples and problems and whether they need support. The present study investigated the open questions (1) to what extent students' (novices) task selections align with instructional design principles and (2) whether informing them about these principles would improve their task selections, learning outcomes, and motivation. Higher education students (N = 150) learned a problem-solving procedure by fixed sequences of examples and problems (FS-condition), or by self-regulated learning (SRL). The SRL participants selected tasks from a database, varying in format, complexity, and cover story, either with (ISRL-condition) or without (SRL-condition) watching a video detailing the instructional design principles. Students' task-selection patterns in both SRL conditions largely corresponded to the principles, although tasks were built up in complexity more often in the ISRL-condition than in the SRL-condition. Moreover, there was still room for improvement in students' task selections after solving practice problems. The video instruction helped students to better apply certain principles, but did not enhance learning and motivation. Finally, there were no test performance or motivational differences among conditions. Although these findings might suggest it is relatively ‘safe’ to allow students to independently start learning new problems-solving tasks using examples and problems, caution is warranted: It is unclear whether these findings generalize to other student populations, as the students participating in this study have had some experience with similar tasks or learning with examples. Moreover, as there was still room for improvement in students' task selections, follow-up research should investigate how we can further improve self-regulated learning from examples and practice problems.     

Analytics of learning tactics and strategies in an online learnersourcing environment

Background

The use of crowdsourcing in a pedagogically supported form to partner with learners in developing novel content is emerging as a viable approach for engaging students in higher-order learning at scale. However, how students behave in this form of crowdsourcing, referred to as learnersourcing, is still insufficiently explored.

Objectives

To contribute to filling this gap, this study explores how students engage with learnersourcing tasks across a range of course and assessment designs.

Methods

We conducted an exploratory study on trace data of 1279 students across three courses, originating from the use of a learnersourcing environment under different assessment designs. We employed a new methodology from the learning analytics (LA) field that aims to represent students' behaviour through two theoretically-derived latent constructs: learning tactics and the learning strategies built upon them.

Results

The study's results demonstrate students use different tactics and strategies, highlight the association of learnersourcing contexts with the identified learning tactics and strategies, indicate a significant association between the strategies and performance and contribute to the employed method's generalisability by applying it to a new context.

Implications

This study provides an example of how learning analytics methods can be employed towards the development of effective learnersourcing systems and, more broadly, technological educational solutions that support learner-centred and data-driven learning at scale. Findings should inform best practices for integrating learnersourcing activities into course design and shed light on the relevance of tactics and strategies to support teachers in making informed pedagogical decisions.     

Effects of an integrated concept mapping and web-based problem-solving approach on students' learning achievements,perceptions and cognitive loads

Although students could effectively search for web data with proper keywords and select web pages related to the studied core issue, however summarizing or organizing the retrieved information remains a difficult task for them. Concept mapping is known to be an effective knowledge construction tool for helping learners organize important concepts related to a core issue. To address the problem, an integrated concept mapping and web-based problem-solving environment, CM-Quest, has been developed; moreover, an experiment has been conducted to evaluate the effectiveness of the approach on students' learning performance, learning satisfaction and cognitive load in an elementary school social studies course. The results show that the concept map-integrated approach can significantly enhance the students' web-based problem-solving performance, although the students showed lower degrees of technology acceptance and learning satisfaction in comparison with the conventional web-based problem-solving approach. Moreover, it is found that the students in the concept mapping group revealed higher cognitive loads than those in the control group, which could be the factor contributing to the lower technology acceptance degree and learning satisfaction. As a consequence, it is concluded that the integrated concept mapping and web-based problem-solving approach is helpful to students in guiding them to learn in a more effective way. On the other hand, it remains an open issue to find a suitable way of integrating concept maps into the learning process without introducing too much extra cognitive load so as to promote students' acceptance degree of using technology for better learning.     

An exploration of students' strategy use in inquiry-based computer-supported collaborative learning

Abstract The aim of this study is to investigate students' use of cognitive learning strategies in inquiry-based computer-supported collaborative learning (CSCL). A process-oriented interview framework on cognitive activity, self-regulation and motivation, and a coding category for analysing cognitive learning strategies and cognitive self-regulation was developed. The students of an intervention group (n=18) participating in inquiry-based CSCL and a comparison group (n=8) were interviewed six to eight times during the 3 years of the study. The results derived from the mixed-method analysis of altogether 161 interviews were compared between the two groups. The results indicate that the students who participated in the inquiry-based CSCL activities reported deeper-level cognitive strategies such as monitoring, creating representations and sharing information collaboratively. The students of the comparison group reported more surface-level strategies such as memorization. However, the findings concerning the utility of CSCL inquiry on cognitive learning strategies were not uniformly positive. It was found that the students of the comparison group reported significantly more strategies under the category of content evaluation. Nevertheless, the results suggest that computer-supported inquiry-based learning can enhance the use of cognitive strategies that support learning.     

Students' engagement characteristics predict success and completion of online courses

This study examined students' engagement characteristics in online courses and their impact on academic achievements, trying to distinguish between course completers and noncompleters. Moreover, this research is intended to differentiate between those who pass the final exam and those who do not. Four online courses were examined with a similar pedagogical model (N_students = 646) using learning analytics methods. The results revealed significant differences between students who completed the courses and students who did not, in all 13 variables. Completers' learning activities were more than twice as high, except for writing in the forums. Course subject and ongoing task and assignment submissions predicted course completion, whereas, in addition to these variables, engagement with course materials and reading the forums predicted final exam success, as well. Thus, the prediction of success in final exam emphasized the significant importance of engagement in various activities in the online course.     

Efficiency of learning environment using GeoGebra when calculus contents are learned in collaborative groups

In this paper we present a modern approach of teaching mathematics based on the computer supported collaborative learning (CSCL) of calculus contents. The collaborative learning was used in calculus course at the University of Novi Sad, Serbia, for examining functions and drawing their graphs. In 2012 the authors decided to improve the collaborative learning introducing GeoGebra application. Small four member groups were formed by using Kagan's (1994) principles. Two groups of students, the experimental, and the control one were observed. The students in the experimental group learned with the help of GeoGebra, and the students in the control group learned without using GeoGebra.Comparison between those two groups of the first year calculus students, regarding their way of learning and the results achieved, is described below. Before the students' collaborative learning, they were tested with a pre-test and their knowledge necessary for examining functions was verified. The pre-test showed that there was no significant statistical difference between the experimental and the control group. The experimental group worked with the help of the computer and the control one without it. After the collaborative learning, the students were tested with a test (colloquium) and the results of the experimental group were significantly better than the results of students in the control group. At the end of the course the students did their exams (post-test), and the results of the experimental group were significantly better than the results of students in the control group.Some students from the experimental group had to answer questions in an interview related to the use of GeoGebra during their collaborative learning. In order to see the students' difficulties in solving problems, students in the experimental group were asked to cross out incorrect parts of solutions, not to erase them. The teachers reviewed the students' tasks done during the collaborative learning and after that the students who had corrected their mistakes were invited for an interview about using GeoGebra for overcoming their difficulties. Based on the students' results in the tests, answers in the questionnaire and in the interview, it can be concluded that GeoGebra has enabled an easier learning of this material. The GeoGebra package enables the students to check whether each step in the process of solving a task was correctly done or not. The results of our research show that GeoGebra can help those students having insufficient knowledge (necessary for solving those tasks) to improve it.We can say that our research shows that the students' learning achievement in examining functions and drawing their graphs is better when they use GeoGebra, working in collaborative groups than without using it. Also, GeoGebra enables creation of effective learning environment for examining functions and drawing their graphs.     

Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments

Online learning often requires learners to be self-directed and engaged. The present study examined students' self-regulatory behaviors in online video-based learning environments. Using an experimental design, this study investigated the effects of a newly designed enhanced video learning environment, which was designed to support or scaffold students' self-regulated or self-directed learning on students' learning behaviors and outcomes. In addition, correspondence between students' self-regulation strategies in traditional learning environments and observed self-regulated learning behaviors in the enhanced video environment were examined. A cross-sectional experimental research design with systematic random assignment of participants to either the control condition (common video) or the experimental condition (enhanced video) was utilized. Undergraduate and graduate students participated in the study (N = 80). Study results indicate that the newly designed enhanced video learning environment was a superior instructional tool than the common video learning environment in terms students' learning performance. In addition, there was correspondence between graduate students' self-reported self-regulation and observed self-regulation, with those high on seeking/learning information and managing their environment/behavior more likely to engage more in interactive note-taking.     

Completion strategy or emphasis manipulation? Task support for teaching information problem solving

While most students seem to solve information problems effortlessly, research shows that the cognitive skills for effective information problem solving are often underdeveloped. Students manage to find information and formulate solutions, but the quality of their process and product is questionable. It is therefore important to develop instruction for fostering these skills. In this research, a 2-h online intervention was presented to first-year university students with the goal to improve their information problem solving skills while investigating effects of different types of built-in task support. A training design containing completion tasks was compared to a design using emphasis manipulation. A third variant of the training combined both approaches. In two experiments, these conditions were compared to a control condition receiving conventional tasks without built-in task support. Results of both experiments show that students' information problem solving skills are underdeveloped, which underlines the necessity for formal training. While the intervention improved students’ skills, no differences were found between conditions. The authors hypothesize that the effective presentation of supportive information in the form of a modeling example at the start of the training caused a strong learning effect, which masked effects of task support. Limitations and directions for future research are presented.     

A contextual game-based learning approach to improving students' inquiry-based learning performance in social studies courses

Inquiry-based learning, an effective instructional strategy, can be in the form of a problem or task for triggering student engagement. However, how to situate students in meaningful inquiry activities remains to be settled, especially for social studies courses. In this study, a contextual educational computer game is developed to improve students' learning performance based on an inquiry-based learning strategy. An experiment has been conducted on an elementary school social studies course to evaluate the effects of the proposed approach on the inquiry-based learning performances of students with different learning styles. The experimental results indicate that the proposed approach effectively enhanced the students' learning effects in terms of their learning achievement, learning motivation, satisfaction degree and flow state. Furthermore, it is also found that the proposed approach benefited the “active” learning style students more than the “reflective” style students in terms of learning achievement. This suggests the need to provide additional supports to students with particular learning styles in the future.     

Students' online interactive patterns in augmented reality-based inquiry activities

Inquiry learning has been developing for years and many countries have incorporated inquiry learning into the scope of K-12 education. Educators have indicated the importance of engaging students in knowledge-sharing activities during the inquiry learning process. In this study, a location-based augmented reality (AR) environment with a five-step guiding mechanism is developed to guide students to share knowledge in inquiry learning activities. To evaluate the effectiveness of the proposed approach in terms of promoting the knowledge sharing behaviors of students, an experiment has been conducted in an elementary school natural science course. The participants were 57 fourth-grade students from an elementary school in Northern Taiwan, divided into an experimental group of 28 students who learned with the AR-based approach and a control group of 29 students who learned with the conventional in-class mobile learning approach. The students' learning behaviors, including their movements in the real-world environment and interactions with peers, were recorded. Accordingly, the learning patterns and interactions of the two groups were analyzed via lag-sequential analysis and quantitative content analysis. It was found that, in comparison with the conventional inquiry-based mobile learning activity, the AR-based inquiry learning activity is able to engage the students in more interactions for knowledge construction. The findings of this study provide guidance for helping teachers develop effective strategies and learning designs for conducting inquiry-based learning activities.     

Analysing the role of a pedagogical agent in psychological and cognitive preparatory activities

There is a lack of studies that examine the role of a pedagogical agent on student development in a specific learning situation that involves psychological and cognitive preparatory activities in high school settings. We examined the effectiveness of pedagogical agent (APT) cognitive and affective feedback on learner motivation and well-being. We applied an experimental research design, involving 45 fourth-year high school students, divided in experimental and control groups (APT vs. human tutor). We performed a quantitative analysis to collect and analyse data of students using our APT. APT cognitive feedback had a positive effect on students' motivation for learning by encouraging students' proposals and initiatives and arousing students' interest in the topic. In addition, APT affective feedback fostered an appropriate emotional climate and creative environment for learning by enhancing students' curiosity, creativity and confidence for carrying out the activity, while reducing students' negative emotions such as boredom and anger. This study provided us useful insights about the affective (and cognitive) competencies that a virtual affective pedagogical agent needs to have in order to support students' mental and emotional health throughout a learning situation. Yet, further research is needed to consolidate these findings and make APT more adaptive to different learning situations.     

Web-based undergraduate chemistry problem-solving: The interplay of task performance,domain knowledge and web-searching strategies

This study investigates the effect of Web-based Chemistry Problem-Solving, with the attributes of Web-searching and problem-solving scaffolds, on undergraduate students' problem-solving task performance. In addition, the nature and extent of Web-searching strategies students used and its correlation with task performance and domain knowledge also were examined. We recruited a total of 183 undergraduate students, all of whom are taking the freshman chemistry course, to participate in the study to solve three chemistry tasks across a semester. Mandated screen-capture software captured participants' on-screen Web-searching processes were recorded every five seconds. Results demonstrated that students' problem-solving performance was significantly improved from task 1 to 3, and students with more domain knowledge outperformed students with less domain knowledge. Students with higher problem-solving performance employed more existing knowledge and metacognitive Web-searching strategies; and students with lower problem-solving performance employed more cognitive Web-searching strategies. In addition, students' problem-solving performance was correlated with their domain knowledge, use of existing knowledge, and metacognitive Web-searching strategies. Moreover, students' use of existing knowledge was the major factor for predicting their problem-solving performance according to the regression model.     

A creative thinking approach to enhancing the web-based problem solving performance of university students

Along with the advancement of information and communication technology, researchers have pointed out the necessity and challenges of developing effective instructional strategies to enhance students' web-based problem-solving performance, which refers to the ability of investigating a series of related problems via searching for, abstracting and summarizing information on the web. In this study, a creative thinking strategy is proposed to cope with this problem. Moreover, an experiment was conducted on 80 freshmen from two classes of a university to evaluate the effectiveness of the proposed approach. The experimental results show that the proposed approach improved the students' web-based problem solving performance in comparison with the conventional approach in terms of “problem finding” and “idea finding.” Moreover, it was found that the proposed approach could improve the “fact finding” performance of the students with intuitive-type cognitive style. Accordingly, some implications and suggestions are given for educators who attempt to conduct web-based problem-solving activity.     

Using blogs to support learning during internship

Blogging has been recommended as a suitable tool for learning during internship due to its associated usefulness in collaborative learning, reflection, communication, and social support. In this study, blogging was incorporated into the internship activities of two discipline-specific groups of interns: information management (n = 53) and nursing (n = 28). In examining the behavior, perceptions and processes of blogging among interns from the two disciplines, a mixed-methods design was used to obtain quantitative and qualitative data through structured interviews and blogging entries. Results revealed that the interns engaged regularly in the writing and reading of their own blogs, and commented on others’ blog-writing. The interns perceived blogs to be useful during internship in providing an avenue for knowledge construction, problem solving, reflection, and communicating their emotions. Positive perceptions were not influenced by discipline background, frequency of use, or blogging platform. Qualitative analyses of blog contents indicated that the students engaged in cognitive, metacognitive-reflective, affective, and social-collaborative learning processes in blogging. Higher engagement was found in cognitive and metacognitive processes. Responses to open-ended probes suggest that pedagogical factors (e.g., grading system, supervision) may also have influenced students’ blogging behaviors and perceptions. Overall, this study offers evidence to support the use of blogging during internship as computer-based support for learning.     

Professional development to enhance teachers' practices in using information and communication technologies (ICTs) as cognitive tools: Lessons learned from a design-based research study

Technology integration in K-12 classrooms is usually overly teacher-centered and has insufficient impact on students' learning, especially in enhancing students' higher-order cognitive skills. The purpose of this project is to facilitate science teachers' use of information and communication technologies (ICTs) as cognitive tools to shift their practices from traditional teacher-centered methods to constructivist, student-centered ones. This paper describes the outcomes and lessons learned from an application of design-based research (DBR) in the implementation and refinement of a teacher professional development (PD) program that is a key component of the overall project. This DBR study involved 25 middle-school science teachers from 24 schools whose implementation of cognitive tools with their students in science classrooms and virtually through a social networking site were observed over four years. A mixed-methodology was utilized to examine the impact of the cognitive tools intervention on teachers' classroom practices and students' development of new literacy skills. Identifying reusable design principles related to technology integration was another focus of the DBR study. The results revealed teachers' positive changes in their classroom practices by gradually allowing students to take control over the use of technology, and positive impact on students' ICT skills and science learning. Design principles for future professional development programs aimed at preparing teachers to adopt a cognitive tools approach are described.     

The effects of computer simulation and animation (CSA) on students' cognitive processes: A comparative case study in an undergraduate engineering course

This study focuses on the investigation of the effects of computer simulation and animation (CSA) on students' cognitive processes in an undergraduate engineering course. The revised Bloom's taxonomy, which consists of six categories in the cognitive process domain, was employed in this study. Five of the six categories were investigated, including remember, understand, apply, analyze, and evaluate. Data were collected via a think‐aloud protocol involving two groups of student participants: One group learned a worked example problem with a CSA module, and the other group learned the same problem with traditional textbook‐style instruction. A new concept called frequency index was proposed for use in qualitative research that involves the quantitative comparison of the overall popularity of a particular mental activity performed by two groups of students. The results show that as compared to traditional textbook‐style instruction, CSA significantly increases students' activities in the understand category of the revised Bloom's taxonomy during learning and significantly increases students' activities in the understand, apply, analyse, and evaluate categories during subsequent problem‐solving. That learning via CSA has a profound impact on subsequent problem‐solving is attributed to intensive human–computer interactions built in the CSA learning module.     

Learning and engagement in a gamified course: Investigating the effects of student characteristics

The current study investigated college students' experiences of a gamified informatics course. We surveyed 139 students aged 18–31 years (M = 20 years, SD = 1.5) enrolled in an undergraduate informatics course focused on social networking technologies. Surveys were conducted at 3 time points during the course (beginning, middle, and end). Overall, we found positive trends with respect to students' perceptions of gamification's impact on their learning, achievement, and engagement in the course material. Although students who played and identified variously with recreational games were more alike than not, we did uncover one notable difference with respect to how students' gaming frequency impacted their engagement in the course. Nongamers expressed somewhat less motivation to do well in the course than frequent gamers. For all other measures of engagement, however, nongamers appeared to be equally engaged by the gamified format of the course as gamers. There were virtually no differences between male and female students' perceptions of gamification. This study contributes new insight into the conditions under which gamification succeeds or fails in educational settings. These insights will be useful to designers and instructors of gamified learning environments as they seek to engage and support a variety of learners.     

Measuring cognitive load in test items: static graphics versus animated graphics

The majority of multimedia learning studies focus on the use of graphics in learning process but very few of them examine the role of graphics in testing students' knowledge. This study investigates the use of static graphics versus animated graphics in a computer‐based English achievement test from a cognitive load theory perspective. Three hundred and three 7th‐grade students were randomly split into two groups and given the test questions either with static graphics or with animated graphics accompanied with text. Students' response time, response accuracy, self‐reported ratings on cognitive load and secondary task approach were used to measure their cognitive load. Findings revealed that animating graphics increased the response time and secondary task scores of the students but did not have any significant effect on their test success. Furthermore, no difference was observed in self‐reported cognitive loads. The relationship between the four different cognitive load measures was also examined in the study. No direct relation was found between self‐reported ratings and secondary task scores. On the other hand, self‐ratings and response accuracy were found to be more sensitive to intrinsic cognitive load, whereas response time and secondary task measures were found to be more sensitive to extraneous cognitive load in our testing environment.