The impact of learning design on student behaviour,satisfaction and performance: A cross-institutional comparison across 151 modules

Pedagogically informed designs of learning are increasingly of interest to researchers in blended and online learning, as learning design is shown to have an impact on student behaviour and outcomes. Although learning design is widely studied, often these studies are individual courses or programmes and few empirical studies have connected learning designs of a substantial number of courses with learning behaviour. In this study we linked 151 modules and 111.256 students with students' behaviour (<400 million minutes of online behaviour), satisfaction and performance at the Open University UK using multiple regression models. Our findings strongly indicate the importance of learning design in predicting and understanding Virtual Learning Environment behaviour and performance of students in blended and online environments. In line with proponents of social learning theories, our primary predictor for academic retention was the time learners spent on communication activities, controlling for various institutional and disciplinary factors. Where possible, appropriate and well designed communication tasks that align with the learning objectives of the course may be a way forward to enhance academic retention.     

Student engagement in course-based social networks: The impact of instructor credibility and use of communication

Social network sites provide the opportunity for building and maintaining online social network groups around a specific interest. Despite the increasing use of social networks in higher education, little previous research has studied their impacts on student's engagement and on their perceived educational outcomes. This research investigates the impact of instructors' self-disclosure and use of humor via course-based social networks as well as their credibility, and the moderating impact of time spent in these course-based social networks, on the students' engagement in course-based social networks. The research provides a theoretical viewpoint, supported by empirical evidence, on the impact of students' engagement in course-based social networks on their perceived educational outcomes. The findings suggest that instructors who create course-based online social networks to communicate with their students can increase their engagement, motivation, and satisfaction. We conclude the paper by suggesting the theoretical implications for the study and by providing strategies for instructors to adjust their activities in order to succeed in improving their students' engagement and educational outcomes.     

The relationship between frequency of Facebook use, participation in Facebook activities, and student engagement

Educators and others are interested in the effects of social media on college students, with a specific focus on the most popular social media website—Facebook. Two previous studies have examined the relationship between Facebook use and student engagement, a construct related to positive college outcomes. However, these studies were limited by their evaluation of Facebook usage and how they measured engagement. This paper fills a gap in the literature by using a large sample (N = 2368) of college students to examine the relationship between frequency of Facebook use, participation in Facebook activities, and student engagement. Student engagement was measured in three ways: a 19-item scale based on the National Survey of Student Engagement, time spent preparing for class, and time spent in co-curricular activities. Results indicate that Facebook use was significantly negatively predictive of engagement scale score and positively predictive of time spent in co-curricular activities. Additionally, some Facebook activities were positively predictive of the dependent variables, while others were negatively predictive.     

Investigating students' learning approaches,perceptions of online discussions,and students' online and academic performance

The main purpose of this study was to understand the relationships between students' approaches to learning, their perceptions of online discussions, students' contributions in asynchronous discussions, and their academic performance. Two sets of questionnaires were used for understanding students' approaches of learning and perceptions of online discussions. The online postings from seven weeks of discussions were coded into three major categories: Initiation, Elaborated Response (ER), and Response with Resources (RWR). The results showed, first, some aspects of students' perceptions influenced the numbers of ERs and RWRs. Secondly, students' contributions to Initiation messages and RWR significantly related to deep motivation and deep strategies; however, the numbers of these two types of messages were negatively correlated to surface strategies. Finally, cluster analysis revealed three distinct groups who scored significantly different in almost all aspects of approaches to learning and perceptions of online discussions. Students in the cluster who adopted deep approaches and scored highest in the perception scales outperformed students in the other two clusters, both in terms of the number of ER messages and academic performance. Pedagogical implications for teaching with online discussions are discussed in this study.     

Helping students help themselves: Generative learning strategies improve middle school students' self-regulation in a cognitive tutor

The current study investigated whether prompting students to engage in generative learning strategies improves students' subsequent judgments of learning and self-regulation. Seventy-eight middle school students in a pre-algebra class completed worksheets in between problem-solving sessions in a computer-based cognitive tutor. Some students were prompted to engage in a generative learning strategy (i.e., writing a summary or writing an explanation for a peer) followed by a judgment of learning (generative group), whereas other students were only asked to make a judgment of learning (control group). Results indicated non-significant levels of judgment accuracy in both groups; however, students in the generative group showed better-calibrated help-seeking behaviors when solving subsequent problems in the tutor. These results suggest that self-regulation can improve in the absence of accurate learning judgments, and that generative learning strategies can facilitate such an improvement. This may be especially true for younger students, who generally demonstrate lower metacognitive awareness.     

Using online assessments to stimulate learning strategies and achievement of learning goals

The main goals of this research are: (i) to explore the influence that announcement of certain type of online assessment has on students' learning strategies and (ii) to explore the influence of stimulated learning strategies on achievement levels that students exhibit during assessments. Research has been conducted by testing and surveying 351 students from higher education institutions. Results indicate that students' learning strategies can be influenced in a relatively short period of time by announcing various types of online assessments and that steering to more desirable deep learning strategies has positive impact on both formal and perceived levels of success in achieving the desired learning goals. These findings can be used to create a novel adaptive online assessment system that incorporates the elements of adaptivity within a series of assessments and uses post-assessment feedback to steer students’ learning strategies.     

Chinese undergraduate students' perceptions of mobile learning: Conceptions,learning profiles,and approaches

Close links between students' conceptions of and approaches to learning were established in the past research. However, only a few quantitative studies investigated this relationship particularly with regard to mobile learning (m‐learning). The correlation between learners' conceptions and approaches to m‐learning was analysed using a partial least squares analysis applied to data obtained from a sample of 971 undergraduate students in China. The results indicated that students' conceptions of m‐learning could be classified into reproductive, transitional, and constructive levels. Students may hold multiple m‐learning applications than a predominant one; hence, examining m‐learning as one monolithic entity may provide limited information. Latent profile analysis identified four learning profiles based on students' preferred m‐learning applications: passive, mixed, surface‐supportive, and high‐engagement.. Moreover, a general trend was observed, whereby students with reproductive and surface‐supportive learning profiles showed a tendency to adopt surface approaches, whereas those expressing constructive and mixed learning profiles were more inclined to adopt deep approaches. Interestingly, students with transitional conceptions and high‐engagement learning profiles tended to take both surface and deep approaches.     

Overcoming the novelty effect in online gamified learning systems: An empirical evaluation of student engagement and performance

Learners in the higher education context who engage with computer-based gamified learning systems often experience the novelty effect: a pattern of high activity during the gamified system's introduction followed by a drop in activity a few weeks later, once its novelty has worn off. We applied a two-tiered motivational, online gamified learning system over 2 years to a total number of 333 students. In a mixed methods research design, we used 3-year worth of longitudinal data (333 students for the treatment group and 175 in the control group) to assess students' engagement and performance in that period. Quantitative results established that students engaged and performed better in the gamified condition vis-à-vis the nongamified. Furthermore, students exhibited higher levels of engagement in the second year compared with the first year of the gamified condition. Our qualitative data suggest that students in the second year of the gamified delivery exhibited sustained engagement, overcoming the novelty effect. Thus, our main contribution is in suggesting ways of making the engagement meaningful and useful for the students, thus sustaining their engagement with computer-based gamified learning systems and overcoming the novelty effect.     

Factors influencing students' beliefs about the future in the context of tablet-based interactive classrooms

With the distribution of easy-to-use tablet computers, tablet-based interactive classrooms have become popular environments for innovative learning activities in recent years. However, little research has investigated the relationship between technologically enhanced learning environments and students' beliefs about the future and self-efficacy for learning. In this study, young students' perceptions of tablet-based interactive classrooms, beliefs about the future, and self-efficacy for learning in rural areas of Korea were examined after the students engaged in tablet-based interactive classrooms. To develop the theoretical framework, we created a structural research model of ease of use, usefulness, satisfaction, deepened experiences through tablet use, beliefs about the future, and self-efficacy in tablet-based interactive classrooms based on a partial least squares (PLS) method. The results indicate that (1) students in tablet-based interactive classrooms perceive frequent experiences with tablet-based instructions as easy and useful and (2) student satisfaction is significantly influenced by their perceptions of deeper learning experiences through tablet use, which are significantly influenced by their future expectations and self-efficacy for learning. This study provides relevant implications for educators who design learning activities for students in rural schools in tablet-based interactive classroom environments.     

From Moodle to Facebook: Exploring students' motivation and experiences in online communities

The present paper documents a qualitative study that examines the motivating and inhibiting factors that influenced students' engagement in online discussions via Moodle and Facebook. The data was collected through individual interviews with 14 pre-service teachers. Using the Activity Theory as a lens, the study reveals a set of factors concerning the technical tools, subjective perceptions, goals of online discussion, social presence within a community, rules for participation, and roles of the participants that affect their online engagement patterns. The findings of the study inform educators and software designers of how online discussions can be better promoted among students, and how a web-based environment more conducive to learning can be created.     

Measuring instant emotions based on facial expressions during computer-based assessment

Emotions are very important during learning and assessment procedures. However, measuring emotions is a very demanding task. Several tools have been developed and used for this purpose. In this paper, the efficiency of the FaceReader during a computer-based assessment (CBA) was evaluated. Instant measurements of the FaceReader were compared with the researchers’ estimations regarding students’ emotions. The observations took place in a properly designed room in real time. Statistical analysis showed that there are some differences between FaceReader’s and researchers’ estimations regarding Disgusted and Angry emotions. Results showed that FaceReader is capable of measuring emotions with an efficacy of over 87% during a CBA and that it could be successfully integrated into a computer-aided learning system for the purpose of emotion recognition. Moreover, this study provides useful results for the emotional states of students during CBA and learning procedures. This is actually the first time that student’s instant emotions were measured during a CBA, based on their facial expressions. Results showed that most of the time students were experiencing Neutral, Angry, and Sad emotions. Furthermore, gender analysis highlights differences between genders’ instant emotions.     

A contextual game-based learning approach to improving students' inquiry-based learning performance in social studies courses

Inquiry-based learning, an effective instructional strategy, can be in the form of a problem or task for triggering student engagement. However, how to situate students in meaningful inquiry activities remains to be settled, especially for social studies courses. In this study, a contextual educational computer game is developed to improve students' learning performance based on an inquiry-based learning strategy. An experiment has been conducted on an elementary school social studies course to evaluate the effects of the proposed approach on the inquiry-based learning performances of students with different learning styles. The experimental results indicate that the proposed approach effectively enhanced the students' learning effects in terms of their learning achievement, learning motivation, satisfaction degree and flow state. Furthermore, it is also found that the proposed approach benefited the “active” learning style students more than the “reflective” style students in terms of learning achievement. This suggests the need to provide additional supports to students with particular learning styles in the future.     

Students' engagement characteristics predict success and completion of online courses

This study examined students' engagement characteristics in online courses and their impact on academic achievements, trying to distinguish between course completers and noncompleters. Moreover, this research is intended to differentiate between those who pass the final exam and those who do not. Four online courses were examined with a similar pedagogical model (N_students = 646) using learning analytics methods. The results revealed significant differences between students who completed the courses and students who did not, in all 13 variables. Completers' learning activities were more than twice as high, except for writing in the forums. Course subject and ongoing task and assignment submissions predicted course completion, whereas, in addition to these variables, engagement with course materials and reading the forums predicted final exam success, as well. Thus, the prediction of success in final exam emphasized the significant importance of engagement in various activities in the online course.     

The impact of paper-based,computer-based and mobile-based self-assessment on students' science motivation and achievement

The present study focuses on the implementation of a self - assessment procedure in a Physics class, extended during a seven weeks period in a European secondary level school. The researchers used three modes of assessment based on paper and pencil, computer-web and mobile devices respectively. The aim of the study is to investigate the effect of each mode of assessment on students' motivation and achievement. Analysis of pre- and post-motivation tests revealed a more positive motivational orientation of students towards computers and mobile devices as assessment delivery mediums. Also, student evaluation implemented after the phase of the experimental procedure showed a significant increase in learning achievement for low-achieving students who participated in the mobile-based and computer-based assessment. The positive effects of computers and mobile devices on students' learning motivation suggest that they can be used as a promising alternative to paper and pencil assessment procedures.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.     

Facilitating student-driven constructing of learning environments using Web 2.0 personal learning environments

Web 2.0 Personal Learning Environments (PLEs) are becoming a promising area of development in e-Learning. While enhancing students' control over the entire learning process including constructing learning environment appears to be an essential objective of introducing Web 2.0 PLEs to education, there is little consensus on how to attain this objective. In this paper a theory-informed model to facilitate students' engagement in constructing their learning environment using Web 2.0 PLEs is proposed and evaluated. The model consists of four components: student's control dimensions, student-centric instructional approaches, the learning potential of Web 2.0 tools and services, and technology enhanced learning activities. This model is used to conduct a design-based research in the context of a first grade class in a secondary school in the Netherlands consisting of 29 students (18 girls and 11 boys, aged 11–13 year). The results suggest that the model can facilitate students' engagement in constructing their learning environment through influencing communication between teacher and students, involving students in adding tools, resources, and people into their learning environment, and enhancing their feeling of ownership over their learning environment.     

Shallow strategy development in a teachable agent environment designed to support self-regulated learning

To support self-regulated learning (SRL), computer-based learning environments (CBLEs) are often designed to be open-ended and multidimensional. These systems incorporate diverse features that allow students to enact and reveal their SRL strategies via the choices they make. However, research shows that students' use of such features is limited; students often neglect SRL-supportive tools in CBLEs. In this study, we examined middle school students' feature use and strategy development over time using a teachable agent system called Betty's Brain. Students learned about climate change and thermoregulation in two units spanning several weeks. Learning was assessed using a pretest–posttest design, and students' interactions with the system were logged. Results indicated that use of SRL-supportive tools was positively correlated with learning outcomes. However, promising strategy patterns weakened over time due to shallow strategy development, which also negatively impacted the efficacy of the system. Although students seemed to acquire one beneficial strategy, they did so at the cost of other beneficial strategies. Understanding this phenomenon may be a key avenue for future research on SRL-supportive CBLEs. We consider two hypotheses for explaining and perhaps reducing shallow strategy development: a student-centered hypothesis related to “gaming the system,” and a design-centered hypothesis regarding how students are scaffolded via the system.     

Promoting self-regulated learning in web-based learning environments

Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and evaluate authoring tools that support teachers and students in web-based learning and instruction. This paper presents how the authoring tools of the Study2000 project can implement psychologically sound measures to promote (a) active and elaborated learning activities and (b) meta-cognitive activities in a web-based learning environment. Furthermore, it describes a study involving 72 university students in the use of such a web-based learning environment in a self-regulated learning setting at the university level. Results show that students spent almost 70% of their study time with texts, 11% with learning tasks and 12% with the active and elaborated learning tools, whereas meta-cognitive aids where hardly used (<1%).     

Computer based testing using “digital ink”: Participatory design of a Tablet PC based assessment application for secondary education

In this paper, we identify key challenges faced by computer-based assessment (CBA) in secondary education and we put forward a framework of design considerations: design with the students and teachers, select the most appropriate media platform and plan an evolution rather than a revolution of prior practices. We present the CBA application ‘MyTest’ which was developed using the participatory methodology We!Design, with the collaboration of 31 students and teachers. The application is targeted for the Tablet PC platform, provides “digital ink” capabilities and supports both closed-type and open questions, facilitating the transfer of traditional assessment practices to CBA. Both students and teachers were excited about the design sessions, and they asserted that they would rely more on educational software designed using this approach. The comparison of the ‘MyTest’ application with an assessment application developed with the same participatory methodology and the involvement of 40 undergraduate students revealed dissimilar expectations and needs of high school and undergraduate students that are often disguised or misinterpreted. A pilot evaluation of the application in real learning conditions was conducted with 31 students using Tablet PCs and with 37 students using traditional PCs. Interestingly, the Tablet PC platform rendered the ‘MyTest’ application more useful and usable to the students, validating our claims.     

Sage,guide, both,or even more? An examination of instructor activity in online MBA courses

This study examined faculty characteristics and behaviors in 46 MBA courses conducted over a two-year period. We found that both formal instructor activities, referred to in the online learning literature as teaching presence, and informal instructor activities, known as immediacy behaviors, were positive predictors of student perceived learning and satisfaction with the educational delivery medium. We also found that instructor login intensity, the average amount of time spent per login session, was a negative predictor of perceived learning. Collectively, these findings suggest the need for instructors to structure and organize their courses beforehand so they can focus on efficient engagement with their students while the class is in session. Although teaching presence and instructor immediacy were significant predictors of delivery medium satisfaction, they explained only 6% of the variance. This finding should help instructors avoid taking unnecessary responsibility for students’ attitudes toward online learning. The paper concludes with a discussion of implications for training of online instructors and the appropriate use of multilevel analytical tools in online learning and education research.