Improving summary writing through formative feedback in a technology-enhanced learning environment

Summary writing is a useful instructional tool for learning. However, summary writing is a challenge to many students. This mixed-method study examined the potential of the Student Mental Model Analyzer for Research and Teaching (SMART) system to help students produce summaries that reflect key concepts and relations in a text. SMART uses the students' summary to generate a multi-dimensional 3S (surface, structure, semantic) evaluation of the students' mental model. This model is then used to drive feedback to help students revise their summary. The current study is an initial investigation examining whether writing and revising in SMART improves students' summary quality. Students (n = 38) in a graduate-level online course used SMART for seven reading assignments. The 38 students submitted a total of 357 summaries in response to the seven readings. In 47 cases, students produced both an initial draft and a modified revision. These 47 cases were selected for analysis. In the quantitative phase, MANOVA results indicated that students' summaries improved along the 3S dimensions from initial draft to revision. In the qualitative phase, inspection of exemplar cases revealed how students' mental models changed towards more robust and cohesive knowledge structure for texts.