Electronic delivery of lectures in the university environment: An empirical comparison of three delivery styles

The purpose of this study was to consider the efficacy and popularity of “Virtual Lectures” (text-based, structured electronic courseware with information presented in manageable “chunks”, interaction and multimedia) and “e-Lectures” (on-screen synchrony of PowerPoint slides and recorded voice) as alternatives to traditional lectures. We considered how three modes of delivery compare when increasingly deeper forms of learning are assessed and also student reaction to electronic delivery. Fifty-eight students in three groups took three topics of a human genetics module, one in each delivery style. Results indicated no overall greater efficacy of either delivery style when all question types were taken into account but significantly different delivery-specific results depending on which level of Bloom’s taxonomy was assessed. That is, overall, questions assessing knowledge consistently achieved the highest marks followed by analysis, comprehension, evaluation and application. Students receiving traditional lectures scored significantly lower marks for comprehension questions. Students receiving Virtual Lectures scored high for knowledge, comprehension and application but significantly lower for analysis and evaluation questions. The e-Lectures scored high for knowledge questions and were the median for all question types except application. Questionnaire analysis revealed a preference for traditional lectures over computer-based but nevertheless an appreciation of the advantages offered by them.     

The design of Cloudworks: Applying social networking practice to foster the exchange of learning and teaching ideas and designs

This paper describes a new social networking site, Cloudworks, which aims to provide a dynamic environment for finding, sharing and discussing learning and teaching ideas and designs. The paper begins by discussing the mismatch between the potential application of technologies in education and their actual use in practice. It considers some of the reasons for this and suggests ways in which this gap might be addressed. It goes on to outline the vision behind the development of Cloudworks, the phases of development and findings to date. It then contextualises this work theoretically drawing in particular on the notion of ‘social objects’ and a framework for sociality. The paper concludes with a discussion of the implications of this work and future research plans.     

A case study on using prediction markets as a rich environment for active learning

In this paper, prediction markets are presented as an innovative pedagogical tool which can be used to create a Rich Environment for Active Learning (REAL). Prediction markets are designed to make forecasts about specific future events by using a market mechanism to aggregate the information held by a large group of traders about that event into a single value. Prediction markets can be used to create decision scenarios which are linked to real-world events. The advantages of this approach in the cognitive and affective domains of learning are examined. The unique ability of prediction markets to enable active learning in large group teaching environments is explored. Building on this theoretical work, a detailed case study is presented describing how a prediction market can be deployed as a pedagogical tool in practice. Empirical evidence is presented exploring the effect prediction market participation has on learners in the cognitive domain.     

Effects of matching teaching strategy to thinking style on learner’s quality of reflection in an online learning environment

Reflection plays an important role in improving learning performance. This study, therefore, attempted to explore whether learners’ reflection levels can be improved if teaching strategies are adapted to fit with learners’ thinking styles in an online learning environment. Three teaching strategies, namely constructive, guiding, and inductive, were designed to match with three thinking styles, namely legislative, executive, and judicial respectively. An online reflection learning system was subsequently developed to reflect this scenario. An experiment was then conducted where the learners were classified into fit or non-fit group in order to analyze whether there was a good fit between the teaching strategies designed by the teacher and the thinking styles of learners. A total of 223 graduate and undergraduate students participated in the experiment. The results revealed that the reflection levels of the fit group had outperformed the non-fit group.     

Computer support for learning mathematics: A learning environment based on recreational learning objects

In this paper, we introduce an electronic collaborative learning environment based on Interactive Instructors of Recreational Mathematics (IIRM), establishing an alternative approach for motivating students towards mathematics. The IIRM are educational software components, specializing in mathematical concepts, presented through recreational mathematics, conceived as interactive, recreation-oriented learning objects, integrated within the environment. We present the architecture of the learning environment which integrates communication services that support the interaction processes of the learning community, through instant messaging, chat rooms, and multi-player math games. Through the environment’s interface of their personal workspace, students have access to several easy-to-use mechanisms that allows them to customize its content, its layout, and its appearance. At internal levels, the functionality of IIRM is enhanced with features supported by the environment infrastructure. We evaluated different aspects of the learning environment in three short, motivation-oriented math courses given to Mexican high-school students. The results indicate that the use of the IIRM-based electronic learning environment, positively affects student attitudes towards mathematics. We believe that this approach has the potential to promote the mathematics learning process, basically on its motivational aspects.     

How indirect supportive digital help during and after solving physics problems can improve problem-solving abilities

This study investigates the effectiveness of computer-delivered hints in relation to problem-solving abilities in two alternative indirect instruction schemes. In one instruction scheme, hints are available to students immediately after they are given a new problem to solve as well as after they have completed the problem. In the other scheme, hints are only available as worked out problems after students have finished their solution. The instruction schemes are supplied by means of a web-based program, Physhint, which supports the development of strategic knowledge [Pol, H. J., Harskamp, E. G., & Suhre, C. J. M. (2008). The effect of the timing of instructional support in a computer-supported problem-solving program for students in secondary physics education. Computers in Human Behavior, 24, 1156–1178]. This program supports novice problem solvers while undertaking physics problems concerned with forces by providing hints structured in accordance with Schoenfeld’s episodes [Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching (pp. 224–270). New York: McMillan Publishing].     

Performance analysis of GAME: A generic automated marking environment

This paper describes the Generic Automated Marking Environment (GAME) and provides a detailed analysis of its performance in assessing student programming projects and exercises. GAME has been designed to automatically assess programming assignments written in a variety of languages based on the “structure” of the source code and the correctness of the program’s output. Currently, the system is able to mark programs written in Java, C++ and the C language. To use the system, instructors are required to provide a simple “marking schema” for each given assessment item, which includes pertinent information such as the location of files and the model solution. In this research, GAME has been tested on a number of student programming exercises and assignments and its performance has been compared against that of a human marker. An in-depth statistical analysis of the comparison is presented, providing encouraging results and directions for employing GAME as a tool for teaching and learning.     

Complementary machine intelligence and human intelligence in virtual teaching assistant for tutoring program tracing

This study proposes a virtual teaching assistant (VTA) to share teacher tutoring tasks in helping students practice program tracing and proposes two mechanisms of complementing machine intelligence and human intelligence to develop the VTA. The first mechanism applies machine intelligence to extend human intelligence (teacher answers) to evaluate the correctness of student program tracing answers, to locate student errors, and to generate hints to indicate errors. The second mechanism applies machine intelligence to reuse human intelligence (previous hints that the teacher gave to other students in a similar error situation) to provide program-specific hints. Two evaluations were conducted with 85 and 64 participants, respectively. The evaluation results showed that the system helped above 89% of students correct their errors. The error-indicating hints generated by the first mechanism help students correct more than half of errors. Each teacher-generated hint was reused averagely three times by the second mechanism. The results also revealed that some error situations occurred frequently and occupied a major occurred percentage of student error situations. In sum, the VTA and these two mechanisms reduce teacher tutoring load and reduce the complexity of developing machine intelligence.     

A conversational intelligent tutoring system to automatically predict learning styles

This paper proposes a generic methodology and architecture for developing a novel conversational intelligent tutoring system (CITS) called Oscar that leads a tutoring conversation and dynamically predicts and adapts to a student’s learning style. Oscar aims to mimic a human tutor by implicitly modelling the learning style during tutoring, and personalising the tutorial to boost confidence and improve the effectiveness of the learning experience. Learners can intuitively explore and discuss topics in natural language, helping to establish a deeper understanding of the topic. The Oscar CITS methodology and architecture are independent of the learning styles model and tutoring subject domain. Oscar CITS was implemented using the Index of Learning Styles (ILS) model (Felder & Silverman, 1988) to deliver an SQL tutorial. Empirical studies involving real students have validated the prediction of learning styles in a real-world teaching/learning environment. The results showed that all learning styles in the ILS model were successfully predicted from a natural language tutoring conversation, with an accuracy of 61–100%. Participants also found Oscar’s tutoring helpful and achieved an average learning gain of 13%.     

Personalized multi-student improvement based on Bayesian cybernetics

This work presents innovative cybernetics (feedback) techniques based on Bayesian statistics for drawing questions from an Item Bank towards personalized multi-student improvement. A novel software tool, namely Module for Adaptive Assessment of Students (or, MAAS for short), implements the proposed (feedback) techniques. In conclusion, a pilot application to two Computer Science courses during a period of 4 years demonstrates the effectiveness of the proposed techniques. Statistical evidence strongly suggests that the proposed techniques can improve student performance. The benefits of automating a quicker delivery of University quality education to a large body of students can be substantial as discussed here.     

Generic service integration in adaptive learning experiences using IMS learning design

IMS Learning Design is a specification to capture the orchestration taking place in a learning scenario. This paper presents an extension called Generic Service Integration. This paradigm allows a bidirectional communication between the course engine in charge of the orchestration and conventional Web 2.0 tools. This communication allows the engine to configure external services so as to adjust their behaviour, and also retrieve information about their activity and use it to influence the orchestration process. This extension impacts all the phases of the life cycle of a learning activity: authoring, deployment and enactment. The paper analyses this impact and proposes appropriate adjustments. A pilot test was conducted and the obtained results show that learning experiences that adapt their flow-based on third party tools while maintaining interoperability, reusability and self-containment requirements are feasible.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.     

Authoring diagram-based CBA with CourseMarker

The CourseMarker system has been used to assess free-response computer based assessment (CBA) exercises since 1998. The aim of the studies reported here was to evaluate the feasibility and usefulness of developing and deploying diagram-based exercises using DATsys, an authoring environment for diagram-based CBA, together with CourseMarker. Postgraduate students constructed diagram-based exercises in four domains. The process of constructing the exercises was captured as an indicator of feasibility. The exercises were then used to assess two cohorts of undergraduate students. Instruments including system submission logs and student questionnaires were used to assess usefulness.Findings indicate that there is considerable potential for the assessment of free-response domains such as diagrams. Such an approach can help students as part of an iterative process of learning by allowing repeated submission of coursework, which may be most appropriate within a formative assessment context. The exercises are popular with students and demonstrate a gradual, though decelerating, increase in marks over subsequent submissions. The techniques are reliable, but further development allowing for alternative model solutions and assessment of the aesthetic appearance of diagrams would increase validity. Our techniques and findings are novel for CBA, and have implications for the increasingly important research area of formative assessment.     

Providing effective feedback, monitoring and evaluation to on-line collaborative learning discussions

Learning and knowledge building have become critical competences for people in the knowledge society era. In this paper, we propose a sociolinguistic dialogue model for understanding how learning evolves and how cognitive process is constructed in on-line discussions. The knowledge extracted from this model is used to assess participation behavior, knowledge building and performance. The ultimate purpose is to provide effective feedback, evaluation and monitoring to the discussion process. Seven hundred students from the Open University of Catalonia in Spain participated in this study. Results showed that learning and knowledge building may be greatly enhanced by presenting selected knowledge to learners as for their particular skills exhibited during interaction. In addition, this valuable provision of information is used as a meta cognitive tool for tutors and moderators for monitoring and evaluating the discussion process more conveniently. This contribution presents our conceptual model for interaction management as well as key design guidelines and evaluation results. Implications of this study are remarked and further research directions are proposed.