Applying learning analytics dashboards based on process-oriented feedback to improve students' learning effectiveness

With the development of a technology-supported environment, it is plausible to provide rich process-oriented feedback in a timely manner. In this paper, we developed a learning analytics dashboard (LAD) based on process-oriented feedback in iTutor to offer learners their final scores, sub-scale reports, and corresponding suggestions on further learning content. We adopted a quasi-experimental design to investigate the effectiveness of the report on students' learning. Ninety-four freshman from two classes participated in this research. The two classes were divided into the LAD group and the original analytics report (OAR) based on a product-oriented feedback group. Before the experiment, all the students took the prior knowledge assessment. After a semester's instruction, all the students took the post-test of the summative assessment. Results indicated that students in the LAD group experienced better learning effectiveness than students in the OAR group. LAD based on process-oriented feedback was also effective in improving the skill learning effectiveness of the students with low-level prior knowledge.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.     

Exploring feedback and student characteristics relevant for personalizing feedback strategies

Personalized tutoring feedback is a powerful method that expert human tutors apply when helping students to optimize their learning. Thus, research on tutoring feedback strategies tailoring feedback according to important factors of the learning process has been recognized as a promising issue in the field of computer-based adaptive educational technologies. Our paper seeks to contribute to this area of research by addressing the following aspects: First, to investigate how students' gender, prior knowledge, and motivational characteristics relate to learning outcomes (knowledge gain and changes in motivation). Second, to investigate the impact of these student characteristics on how tutoring feedback strategies varying in content (procedural vs. conceptual) and specificity (concise hints vs. elaborated explanations) of tutoring feedback messages affect students' learning and motivation. Third, to explore the influence of the feedback parameters and student characteristics on students' immediate post-feedback behaviour (skipping vs. trying to accomplish a task, and failing vs. succeeding in providing a correct answer). To address these issues, detailed log-file analyses of an experimental study have been conducted. In this study, 124 sixth and seventh graders have been exposed to various tutoring feedback strategies while working on multi-trial error correction tasks in the domain of fraction arithmetic. The web-based intelligent learning environment ActiveMath was used to present the fraction tasks and trace students' progress and activities. The results reveal that gender is an important factor for feedback efficiency: Male students achieve significantly lower knowledge gains than female students under all tutoring feedback conditions (particularly, under feedback strategies starting with a conceptual hint). Moreover, perceived competence declines from pre- to post-test significantly more for boys than for girls. Yet, the decline in perceived competence is not accompanied by a decline in intrinsic motivation, which, instead, increases significantly from pre- to post-test. With regard to the post-feedback behaviour, the results indicate that students skip further attempts more frequently after conceptual than after procedural feedback messages.     

The impact of computer-mediated immediate feedback on developing oral presentation skills: An exploratory study in virtual reality

Feedback plays a central role in learning. Crucial to this is the nature and timing of the feedback. This experimental study explores the efficacy of immediate computer-mediated feedback within a virtual reality environment designed to facilitate the development of pre-university students' presentation skills. Two conditions were established to assess the efficacy: immediate computer-mediated feedback; a control condition of delayed expert-mediated feedback. Results showed improvement but no statistically relevant difference in performance gains between the two conditions, suggesting both can facilitate learning. Furthermore, students perceived the environment to be an effective and motivating platform in which to practise presentation skills. For educators seeking viable alternatives to face-to-face presentation practice and feedback, the finding that immediate computer-mediated feedback is potentially as effective in aiding presentation performance is crucial for two reasons: first, it expands practice opportunities for students; second, it could result in less pressure on resources, including time and staffing.     

Combined inner and outer loop feedback in an intelligent tutoring system for statistics in higher education

Intelligent tutoring systems (ITSs) can provide inner loop feedback about steps within tasks, and outer loop feedback about performance on multiple tasks. While research typically addresses these feedback types separately, many ITSs offer them simultaneously. This study evaluates the effects of providing combined inner and outer loop feedback on social sciences students' learning process and performance in a first-year university statistics course. In a 2 x 2 factorial design (elaborate inner loop vs. minimal inner loop and outer loop vs. no outer loop feedback) with 521 participants, the effects of both feedback types and their combination were assessed through multiple linear regression models. Results showed mixed effects, depending on students' prior knowledge and experience, and no overall effects on course performance. Students tended to use outer loop feedback less when also receiving elaborate inner loop feedback. We therefore recommend introducing feedback types one by one and offering them for substantial periods of time.     

Unpacking students' conceptualizations through haptic feedback

While it is clear that the use of computer simulations has a beneficial effect on learning when compared to instruction without computer simulations, there is still room for improvement to fully realize their benefits for learning. Haptic technologies can fulfill the educational potential of computer simulations by adding the sense of touch. Visuohaptic simulations may not only help students visualize these concepts, but they may also have the capability of enriching the learning experience and enhancing retention. To provide additional insights about how students conceptualize abstract and difficult concepts in science, this study proposes a sequencing approach. The research questions are: (1) what are undergraduate students' ways of conceptualizing electric fields through haptic feedback? And (2) what are undergraduate students' perceptions of using visuohaptic simulations for their learning of electric force concepts? Participants included nine undergraduate students who participated in a think aloud procedure. Data were analysed qualitatively using open coding followed by axial coding. The results suggest that students' conceptualized electric force concepts through embodied haptic experiences by inferring force–distance relationship, sign inference, shape of field and indirectly inferring the concept of electric potential. Students also perceived the value of using visual plus haptic simulations to help them understand and retain concepts.     

Effects of timing of formative feedback in computer-assisted learning environments

This study examines the effects of timing of corrective formative feedback on processing text information on question-answering. Undergraduate students read an expository text and answered questions in two attempts. Students were randomly assigned to a no feedback, immediate feedback and delayed feedback conditions. Students in the feedback conditions received feedback on the correctness of their answer after the first attempt and were informed about the right answer after the second attempt. Students were prompted to restudy the text after failing in their first attempt. However, students in the no feedback condition were just prompted to search the text. All students were tested on question-answering, corrective probability and a post-test cued-recall test. Results showed that: (a) feedback reduced the initial time reading the text; (b) feedback increased performance on question answering and cued-recall; (c) delayed feedback produced no advantages over immediate feedback. Theoretical and practical implications of these results are discussed.     

Improving summary writing through formative feedback in a technology-enhanced learning environment

Summary writing is a useful instructional tool for learning. However, summary writing is a challenge to many students. This mixed-method study examined the potential of the Student Mental Model Analyzer for Research and Teaching (SMART) system to help students produce summaries that reflect key concepts and relations in a text. SMART uses the students' summary to generate a multi-dimensional 3S (surface, structure, semantic) evaluation of the students' mental model. This model is then used to drive feedback to help students revise their summary. The current study is an initial investigation examining whether writing and revising in SMART improves students' summary quality. Students (n = 38) in a graduate-level online course used SMART for seven reading assignments. The 38 students submitted a total of 357 summaries in response to the seven readings. In 47 cases, students produced both an initial draft and a modified revision. These 47 cases were selected for analysis. In the quantitative phase, MANOVA results indicated that students' summaries improved along the 3S dimensions from initial draft to revision. In the qualitative phase, inspection of exemplar cases revealed how students' mental models changed towards more robust and cohesive knowledge structure for texts.     

A blended learning lecture delivery model for large and diverse undergraduate cohorts

A blended learning model was developed to enhance lecture delivery in a large, diverse introductory psychology unit, introducing the use of an online, personalized learning system for lecture preparation and using lecture time to extend students' understanding. Changes to the assessment included diagnostic, formative and summative online quizzes. Using hierarchical multiple regression to account for the variance associated with prior achievement and background knowledge, the results show that students who completed the online formative assessments had significantly higher scores on summative assessment tasks, and that scores were even higher for students who used these resources repeatedly. Changes to future implementations of the model are suggested to improve student engagement in formative assessment, and to facilitate lecturer's use of reports on student progress to focus and improve the quality of discussion in the face-to-face lectures.     

Interplay of prior knowledge,self‐regulation and motivation in complex multimedia learning environments

This study examined the direct and indirect effects of medical clerkship students' prior knowledge, self‐regulation and motivation on learning performance in complex multimedia learning environments. The data from 386 medical clerkship students from six medical schools were analysed using structural equation modeling. The structural model revealed that medical students' prior knowledge directly positively affected their learning outcome, self‐efficacy and performance approach goal orientation. The learners' self‐regulation had a significant positive direct effect on learning outcome. The learners' mastery goal orientation directly affected their learning outcome. Interestingly, inconsistent with our hypothesis, the learners' performance approach goal orientation showed a significant negative direct effect on learning outcome, and performance avoidance goal orientation had a significant positive effect on learning outcome. These findings help develop a more comprehensive understanding of the role of individual characteristics on learning performance of complex tasks in multimedia learning environments.     

Generic reflective feedback: An effective approach to developing clinical reasoning skills

Problem‐based learning can be an effective tool to develop clinical reasoning skills. However, it traditionally takes place in tutorial groups, giving students little flexibility in how and when they learn. This pilot study compared the effectiveness of generic reflective feedback (GRF) with tutorial‐based reflective feedback on the development of students' clinical reasoning skills. Case studies were placed in an interactive software package, with GRF for the nine patient cases (36 GRF forms in total). The GRF contained information about common mistakes and omissions made by students in previous years. By design, the GRF left some clinical questions unanswered in order to prompt student reflection, enhance the development of their clinical reasoning skills and give the 2010 students (n = 53) greater control over the way in which they engaged with the unit. Clinical reasoning skills development, assessed by responses to the 36 parts across the 9 cases and in the relevant section of the final exam, was compared with the 2009 tutorial cohort (n = 47 students). GRF enabled student progression through each of the nine online cases with minimal tutor intervention. The cohort receiving GRF performed as well as those students engaging in tutorials with individualized feedback. GRF was associated with a significant increase in student satisfaction and flexibility in how and when they learned. We conclude that GRF enabled online enhancement of a case‐based clinical reasoning unit. Further research on how GRF can optimize learning in different online settings is warranted.     

Computer-based feedback in linear algebra: Effects on transfer performance and motivation

Two studies investigated the effects on students' perceptions (Study 1) and learning and motivation (Study 2) of different levels of feedback in mathematical problems. In these problems, an error made in one step of the problem-solving procedure will carry over to the following steps and consequently to the final solution. Providing immediate feedback after an error is made could prevent such carry-over effects. Feedback given on all problem-solving steps was hypothesized to yield higher motivation and better learning than feedback on the final problem-solving step. Study 1 investigated students' perceptions of three feedback types: ‘on the final solution step’; ‘on all the solution steps at once’; and ‘on all the solution steps successively’. Feedback on all solutions steps was perceived by learners more positively than feedback on the final solution step. Study 2 investigated the learning and motivational effects of two types of feedback, namely, ‘feedback on the final solution step’ and ‘feedback on all the solution steps’. The hypotheses that feedback on all problem-solving steps would lead to more effective learning and higher motivation than feedback on the final solution step were confirmed. Our results support current efforts to implement step-wise feedback. The implications for further research and for the design of feedback are discussed.     

The comparisons of the influences of prior knowledge on two game-based learning systems

Game-based learning provides many benefits, such as enhancing the interaction with students and stimulating their learning motivation. Thus, it is popular to learners who have diverse characteristics. To this end, individual differences play an essential role. Among various individual differences, previous studies demonstrated that prior knowledge has great effects on game-based learning. However, such studies mainly considered a single game-based learning system. To address this issue, this study examined how prior knowledge affects students' reactions to two different types of game based learning systems, i.e., the Machinarium and the CSI: Web Adventures. The former delivers procedural knowledge while the latter provides declarative knowledge. The results from this study indicate that prior knowledge has positive impacts in the CSI group while it has negative impacts in the Machinarium group. These findings imply that prior knowledge is useful for the context of declarative knowledge while it may not be helpful for the context of procedural knowledge.     

E-portfolios supporting primary students' writing performance and peer feedback

E-portfolios show great promise for supporting students' writing performance and facilitating peer feedback, but empirical research of their use in primary education is limited. To address this gap, a yearlong study was conducted in a 4th grade primary class in Cyprus implementing e-portfolios (n = 20) to help students document their progress and send feedback to peers. A generic, open source weblog tool localized into Greek was used as an e-portfolio tool. Data sources included 176 student-essays, 1306 instances of peer feedback, students pre- and post-tests on writing performance, nine videotaped student interviews and one teacher interview. A paired samples t-test analysis showed that there was a statistically significant difference between students' pre-test (M = 62.6, SD = 9.82, n = 20) and post-test (M = 76.5, SD = 12.6, n = 20) on writing performance (t (19) = −8.03, p < .01). A repeated measures analysis of variance on e-portfolio artifacts showed that there were learning gains with respect to students' writing performance over time [F(4.01, 76.25) = 31.59, p < .01, η² = 0.89]. A qualitative analysis of students' comments showed that students provided more thorough peer feedback over time and became gradually more capable of providing corrective feedback. The qualitative analysis of student interviews provided evidence that students valued peer feedback in their portfolios. The analysis of the teacher interview showed that average and high-ability students benefited the most from peer feedback. This study provided evidence that e-portfolios can support the development of students' writing performance and peer feedback skills in the context of primary school essay writing. Instructional implications for portfolio implementation are offered.     

Hypermedia learning and prior knowledge: domain expertise vs. system expertise

Abstract Prior knowledge is often argued to be an important determinant in hypermedia learning, and may be thought of as including two important elements: domain expertise and system expertise. However, there has been a lack of research considering these issues together. In an attempt to address this shortcoming, this paper presents a study that examines how domain expertise and system expertise influence students' learning performance in, and perceptions of, a hypermedia system. The results indicate that participants with lower domain knowledge show a greater improvement in their learning performance than those with higher domain knowledge. Furthermore, those who enjoy using the Web more are likely to have positive perceptions of non-linear interaction. Discussions on how to accommodate the different needs of students with varying levels of prior knowledge are provided based on the results.     

Teaching strategic text review by computer and interaction with student characteristics

The effect of explaining the value of text review was studied. Students (n = 136) were randomly assigned to read a text passage displayed by computer with or without an explanation and in three presentation modes: required or optional review when answer to adjunct questions were incorrect, or reading the text without questions. Review groups learned more than those merely reading the text, and an interaction between student's prior knowledge and explanations indicated that explanation facilitated the learning of students with little familiarity with the material, while slightly impairing knowledgeable students' performance. The implications of these findings for using computers for such training and for ATI research are discussed.     

Providing feedback on computer-based algebra homework in middle-school classrooms

Homework is transforming at a rapid rate with continuous advances in educational technology. Computer-based homework, in particular, is gaining popularity across a range of schools, with little empirical evidence on how to optimize student learning. The current aim was to test the effects of different types of feedback on computer-based homework. In the study, middle school students completed a computer-based pretest, homework assignment, and posttest containing challenging algebraic problems. On the homework assignment, students were assigned to different feedback conditions. In Experiment 1 (N = 103), students received no feedback or correct-answer feedback after each problem. In Experiment 2 (N = 143), students received (1) no feedback, (2) correct-answer feedback, (3) try-again feedback, or (4) explanation feedback after each problem. For students with low prior knowledge, feedback resulted in better posttest performance than no feedback. However, students with high prior knowledge learned just as much whether they received feedback or not. Results suggest the provision of basic feedback on computer-based homework can benefit novice students’ mathematics learning.     

Formative computer-based feedback in the university classroom: Specific concept maps scaffold students' writing

Formative feedback can be regarded as a crucial scaffold for students' writing cohesive texts. However, especially in large lectures students rarely receive feedback on their writing product. Thus, computer-based feedback could be an alternative to provide formative feedback to students. However, it is less clear, how computer-based feedback should be designed to help students writing cohesive texts. For that purpose, we implemented three different computer-based feedback methods within an authentic large lecture class. We investigated effects of the format (outline versus concept map) and the specificity (specific versus general) of the feedback on students' perceived difficulty and the generation of cohesive texts. We found that specific concept map feedback was perceived as less difficult as compared to the general feedback or the specific outline feedback. Additionally, students who received specific feedback wrote explanatory texts that were more cohesive as compared to students with the general feedback. However, the format of the feedback (concept map versus outline) did not account for improvements of cohesion. Evidently, specific concept map feedback can be regarded as an efficient scaffold to provide cohesive explanations.     

F-TCKT：融合遗忘因素的深度时序卷积知识追踪模型

智慧教育中，对学生的知识水平进行追踪是很重要的技术之一。传统的深度知识追踪方法的主要关注点集中在循环神经网络(recurrent neural network, RNN)上，但RNN存在梯度消失或者梯度爆炸的问题，并且很多知识追踪方法没有考虑到学习过程中遗忘行为对结果的影响。针对以上问题，为了准确地预测学生的知识水平，提出了一种融合遗忘因素的深度时序卷积知识追踪模型(temporal convolutional knowledge tracking with forgetting, F-TCKT)。该模型引入了三个影响学生遗忘行为的因素，包括学习相同知识点的时间间隔、学习的时间间隔和同一知识点的学习次数。首先利用全连接网络计算得到表示学生遗忘程度的向量并与学生的答题记录进行拼接，然后使用梯度稳定的时间卷积网络(temporal convolutional network, TCN)和注意力机制预测学生下一次答题正误的概率。经实验验证，与传统方法相比，F-TCKT具有更好的预测性能。