Reflective assessment for epistemic agency of academically low‐achieving students

This study aimed to examine the role and process of reflective assessment supported by the Knowledge Connections Analyzer in helping low academic achievers to develop epistemic agency in knowledge building. The participants were 33 ninth‐grade low achievers from a visual arts course in Hong Kong. A comparison class of 33 students, taught by the same teacher and studying the same topics in a regular knowledge‐building environment, also participated. Qualitative tracing of students' online discourse showed that reflective assessment can help low achievers develop high‐level epistemic agency. Qualitative analysis of the students' prompt sheets revealed that reflective assessment encouraged low achievers to set knowledge‐building goals, collectively and continuously analyse and reflect on their inquiry and ideas, and generate actions to address identified gaps, thus helping them engage in high‐level epistemic agency. The study results have important implications for designing technology‐rich environments that support learners and offer insights into how teachers can help learners develop epistemic agency.     

Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis

This contribution examines the methodological challenges involved in defining the collaborative knowledge-building processes occurring in asynchronous discussion and proposes an approach that could advance understanding of these processes. The written protocols that are available to the analyst provide an exact record of the instructional transactions at a given time in the online discussion. On the basis of a study of online discussion forums used in a higher education context, a model for the analysis of collaborative knowledge building in asynchronous discussion is presented. The model allows examination of the communication from the multiple perspectives of interaction, cognition and discourse analysis. The investigation was conducted using a qualitative case study approach and involved an in-depth examination of three cases. Content analysis of the discourse was done at a number of levels, focusing on the discussion forum itself, the discussion threads, the messages, and the exchanges and moves among the messages. As a result of correspondences found among the variables representing the different levels of the analysis, the most important being the relationship between type of interaction, phase of critical inquiry, and move in the exchange structure, it was possible to build a scheme for assessing knowledge building in asynchronous discussion groups. The scheme integrates the interactive, cognitive and discourse dimensions in computer-supported collaborative learning (CSCL). The study represents a merging of quantitative analysis within qualitative methodology and provides both an analytic and a holistic perspective on CSCL.     

Assessing students' conceptual understanding using an online three‐tier diagnostic test

When solving a mathematical problem, students who do not have sufficient conceptual understanding may perform poorly and exhibit misconceptions. This study was aimed to examine students' conceptual understanding and significant misconceptions when solving number sense‐related problems. An online three‐tier diagnostic test was administered to 125 fifth‐grade students with varied socio‐economic backgrounds in Hong Kong. Only 14.40% of the students exhibited high performance with high confidence, indicating that these students had a profound conceptual understanding of number sense. In addition, the majority of the students (66.40%) did not demonstrate number sense; these students exhibited several significant misconceptions and could solve the questions only by using a rule‐based method or guessing. Accordingly, most students performed unsatisfactorily on number sense‐related problems. This study is imperative in identifying early predictors and provides information for further compatible interventions in the teaching and learning of number sense in Hong Kong in particular and worldwide in general.     

Students assessing their own collaborative knowledge building

We describe the design of a knowledge-building environment and examine the role of knowledge-building portfolios in characterizing and scaffolding collaborative inquiry. Our goal is to examine collaborative knowledge building in the context of exploring the alignment of learning, collaboration, and assessment in computer forums. The key design principle involved turning over epistemic agency to students; guided by several knowledge-building principles, they were asked to identify clusters of computer notes that indicated knowledge-building episodes in the computer discourse. Three classes of 9th grade students in Hong Kong used Knowledge Forum in several conditions: Knowledge Forum only, Knowledge Forum with portfolios, and Knowledge Forum with portfolios and principles. Results showed: (1) Students working on portfolios guided by knowledge-building principles showed deeper inquiry and more conceptual understanding than their counterparts; (2) Students' knowledge-building discourse, reflected in portfolio scores, contributed to their domain understanding; and (3) Knowledge-building portfolios helped to assess and foster collective knowledge advances: A portfolio with multiple contributions from students is a group accomplishment that captures the distributed and progressive nature of knowledge building. Students extended their collective understanding by analyzing the discourse, and the portfolio scaffolded the complex interactions between individual and collective knowledge advancements. Computer-Supported Collaborative Learning (2006) 1(1):57-87 DOI 10.1007/s11412-006-6844-4 This paper was published without its complete corrections. This is a publisher and typesetter error. The online version of the original article can be found at:     

Students assessing their own collaborative knowledge building

We describe the design of a knowledge-building environment and examine the role of knowledge-building portfolios in characterizing and scaffolding collaborative inquiry. Our goal is to examine collaborative knowledge building in the context of exploring the alignment of learning, collaboration, and assessment in computer forums. The key design principle involved turning over epistemic agency to students; guided by several knowledge-building principles, they were asked to identify clusters of computer notes that indicated knowledge-building episodes in the computer discourse. Three classes of 9th grade students in Hong Kong used Knowledge Forum in several conditions: Knowledge Forum only, Knowledge Forum with portfolios, and Knowledge Forum with portfolios and principles. Results showed: (1) Students working on portfolios guided knowledge-building principles showed deeper inquiry and more conceptual understanding than their counterpart (2) Students' knowledge-building discourse, reflected in portfolio scores, contributed to their domain understanding; and (3) Knowledge-building portfolios helped to assess and foster collective knowledge advances: A portfolio with multiple contributions from students is a group accomplishment that captures the distributed and progressive nature of knowledge building. Students extended their collective understanding by analyzing the discourse, and the portfolio scaffolded the complex interactions between individual and collective knowledge advancements. An erratum to this article is available at .     

An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences

The use of e‐assessment methods raises important concerns regarding the reliability and validity of these methods. Potential threats to validity include mode effects and the possible influence of computer‐related attitudes. Although numerous studies have now investigated the validity of online assessments in noncourse‐based contexts, few studies have addressed this issue in an educational context. The present study helps fill this research gap by investigating whether university students' computer‐related attitudes and assessment mode preferences were related to performance on a course‐based online assessment task. Overall, students' attitudes and preferences bore no greater relationships to performance on the online than offline module assessment tasks. This provides support for the validity of course‐based online assessment methods and should help alleviate educators' concerns and encourage more widespread adoption of these methods, helping address the issue of their slow uptake to date. Suggestions for follow‐up studies to corroborate and extend the current findings are offered.     

Students' abilities to evaluate the credibility of online texts: The role of internet-specific epistemic justifications

Previous evaluation studies have rarely used authentic online texts and investigated upper secondary school students' use of evaluation criteria and deep reasoning. The associations between internet-specific epistemic justifications for knowing and credibility evaluation of online texts are not yet fully understood among adolescents. This study investigated upper secondary school students' (N = 372) abilities to evaluate self-selected authentic online texts and the role of internet-specific epistemic justifications in students' evaluation performance when solving a health-related information problem. Students selected three texts with Google Custom Search Engine and evaluated their credibility. Students' evaluation performance across the three texts was determined according to the different aspects evaluated (author, venue, intentions, evidence and corroboration) and the depth of their evaluations. Students also filled in the Internet-Specific Epistemic Justifications (ISEJ) inventory previously validated with pre-service teachers. The results revealed considerable differences in students' abilities to evaluate online texts. Students' beliefs in justification by authority and justification by multiple sources positively predicted their evaluation performance similarly in both topics. The findings suggest that the ISEJ inventory is also valid for upper secondary school students. Students should be explicitly taught to evaluate different credibility aspects and scaffolded to deeply engage with online information.     

Epistemic activities and collaborative learning: towards an analytical model for studying knowledge construction in networked learning settings

The aim of this paper is to provide an analytical model for studying knowledge creation among geographically dispersed students in an online postgraduate programme by using the concept of epistemic activity. Rooted in qualitative methodologies, it takes a case study approach which strengthens the grounding of theory on social learning by using triangulation of evidence from online discussions and seven semi‐structured interviews. The analytical model that emerges, distinguishes a set of moves that serve understanding negotiation of meaning and co‐construction of knowledge in online discussion. The findings of the study suggest that knowledge advancement processes can be understood as part of identifying key strategies in the form of online epistemic activities for postgraduate learners in a networked learning context. The paper offers a conceptual account for researching knowledge construction in higher education.     

Cognitive perturbation through dynamic modelling: a pedagogical approach to conceptual change in science

While simulations have widely been used to facilitate conceptual change in learning science, results indicate that significant disparity or gap between students' prior conceptions and scientific conceptions still exists. To bridge the gap, we argue that the applications of computer simulation in science education should be broadened to enable students to model their thoughts and to improve and advance their theories progressively. While computer simulations are often used to offer opportunities for students to explore scientific models, they do not give them the space to explore their own conceptions, and thus cannot effectively address the challenge of changing students' alternative conceptions. Findings from our recent empirical study reveal that, firstly, dynamic modelling using the environment WorldMaker 2000 in conjunction with the use of a cognitive perturbation strategy by the teacher was effective in helping students to migrate from their alternative conceptions towards a more scientifically inclined one; secondly, the pathways of conceptual change across groups were idiosyncratic and diverse. Respecting students' ideas seriously and providing cognitive perturbation at appropriate junctures of the inquiry process are found to be conducive to fostering conceptual change. In this paper, we will report on the details of the pedagogical approach adopted by the teacher and portray how students' conceptions change during the entire process of model building.     

Analysing computational thinking in collaborative programming: A quantitative ethnography approach

Computational thinking (CT), the ability to devise computational solutions for real‐life problems, has received growing attention from both educators and researchers. To better improve university students' CT competence, collaborative programming is regarded as an effective learning approach. However, how novice programmers develop CT competence through collaborative problem solving remains unclear. This study adopted an innovative approach, quantitative ethnography, to analyze the collaborative programming activities of a high‐performing and a low‐performing team. Both the discourse analysis and epistemic network models revealed that across concepts, practices, and identity, the high‐performing team exhibited CT that was systematic, whereas the CT of the low‐performing team was characterized by tinkering or guess‐and‐check approaches. However, the low‐performing group's CT development trajectory ultimately converged towards the high‐performing group's. This study thus improves understanding of how novices learn CT, and it illustrates a useful method for modeling CT based in authentic problem‐solving contexts.     

Enhancing middle school students' scientific learning and motivation through agent‐based learning

Facing students' decreasing motivation to pursue scientific study, schools and educators need to coordinate new technologies with pedagogical agents to effectively sustain or promote students' scientific learning and motivation to learn. Although the provision of pedagogical agents in student learning has been studied previously, it is not clear what benefits the strategy might offer with regard to student motivation. This study proposes an agent‐based mechanism that integrates problem‐solving and inquiry‐based instructions to help students better understand complex scientific concepts and to sustain their motivation to learn science. In this study, a quasi‐experiment was conducted to evaluate the performance and feasibility of our proposed mechanism. The results revealed that the agent‐based mechanism was effective and feasible for enhancing students' learning and motivation to learn. The mechanism was associated with increases in the acquisition of knowledge when compared with the control group. Its effect in promoting and sustaining students' motivation was also statistically significant. Detailed discussions of the findings are provided in this study.     

Enhancing skill in constructing scientific explanations using a structured argumentation scaffold in scientific inquiry

Constructing scientific explanations is necessary for students to engage in scientific inquiry. The purpose of this study is to investigate the influence of using a structured argumentation scaffold to enhance skill in constructing scientific explanations in the process of scientific inquiry. The proposed approach is designed to scaffold the following aspects of argumentation: the argumentation process, the explanation structuring, explanation construction, and explanation evaluation. A quasi-experiment was conducted to examine the effectiveness of the structured argumentation scaffold in developing skill in constructing scientific explanations and engaging in electronic dialogues. A web-based collaborative synchronous inquiry system, ASIS (Argumentative Scientific Inquiry System), was utilized to support students as they worked in groups to carry out inquiry tasks. Two intact sixth grade classes (n = 50) participated in the study. The data show that the ASIS with the structured argumentation scaffold helped students significantly improve their skills in constructing scientific explanations, make more dialogue moves for explanation and query, and use more of all four argument components. In addition, the use of warrants, one of the components of an argument, was found to be a critical variable in predicting students' competence with regard to constructing scientific explanations. The results provide references for further research and system development with regard to facilitating students' construction of scientific argumentation and explanations.     

Stop talking and type: comparing virtual and face‐to‐face mentoring in an epistemic game

Research has shown that computer games and other virtual environments can support significant learning gains because they allow young people to explore complex concepts in simulated form. However, in complex problem‐solving domains, complex thinking is learned not only by taking action, but also with the aid of mentors who provide guidance in the form of questions, instructions, advice, feedback and encouragement. In this study, we examine one context of such mentoring to understand the impact of replacing face‐to‐face interactions between mentors and students with virtual, chat‐based interactions. We use pre‐ and post‐measures of learning and a post‐measure of engagement, as well as epistemic network analysis (ENA), a novel quantitative method, to examine student and mentor discourse. Our results suggest that mentoring via online chat can be as effective as mentoring face‐to‐face in appropriately structured contexts more generally – and that ENA may be a useful tool for assessing student and mentor discourse in the context of learning interactions.     

Interplay of prior knowledge,self‐regulation and motivation in complex multimedia learning environments

This study examined the direct and indirect effects of medical clerkship students' prior knowledge, self‐regulation and motivation on learning performance in complex multimedia learning environments. The data from 386 medical clerkship students from six medical schools were analysed using structural equation modeling. The structural model revealed that medical students' prior knowledge directly positively affected their learning outcome, self‐efficacy and performance approach goal orientation. The learners' self‐regulation had a significant positive direct effect on learning outcome. The learners' mastery goal orientation directly affected their learning outcome. Interestingly, inconsistent with our hypothesis, the learners' performance approach goal orientation showed a significant negative direct effect on learning outcome, and performance avoidance goal orientation had a significant positive effect on learning outcome. These findings help develop a more comprehensive understanding of the role of individual characteristics on learning performance of complex tasks in multimedia learning environments.     

Supporting middle school students' online reading of scientific resources: moving beyond cursory,fragmented, and opportunistic reading

The abundant scientific resources on the Web provide great opportunities for students to expand their science learning, yet easy access to information does not ensure learning. Prior research has found that middle school students tend to read Web‐based scientific resources in a shallow, superficial manner. A software tool was designed to support middle school students in reading online scientific resources through three key strategies: making explicit a skim–read–summarize structure for online reading, using prompts to guide students' reading and foster articulation of thinking, and connecting reading to learning purposes. This study examined the differences between regular and guided online reading performed by eight pairs of sixth graders in a science inquiry project. The students' online reading processes and conversations were captured by a screen‐recording programme. Analysis of 60 h of screen videos showed that the students' online reading in the regular condition was cursory, fragmented, and opportunistic, while the structured online reading was more deliberate, thorough, and purposeful. Overall, the results suggest that middle school students' online reading of scientific resources needs to be guided.     

The effects of directive self‐explanation prompts to support active processing of multiple representations in a simulation‐based learning environment

Processing of multiple representations in multimedia learning environments is considered to help learners obtain a more complete overview of the domain and gain deeper knowledge. This is based on the idea that relating and translating different representations leads to reflection beyond the boundaries and details of the separate representations. To achieve this, the design of a learning environment should support learners in adequately processing multiple representations. In this study, we compared a scientific inquiry learning environment providing instructional support with directive self‐explanation prompts to relate and translate between representations with a scientific inquiry learning environment providing instructional support with general self‐explanation prompts. Learners who received the directive prompts outperformed the learners who received general prompts on test items assessing domain knowledge. These positive results did not stretch to transfer items and items measuring learners' capabilities to relate and translate representations in general. The results suggest that learner support should promote the active relation of representations and translation between them to foster domain knowledge, and that other forms of support (e.g. extended training) might be necessary to make learners more expert processors of multiple representations.     

Can structured representation enhance students' thinking skills for better understanding of E-learning content?

This paper proposes an e-learning model to assess the effects of online structured representation of content on learners' understanding. We designed a structured representation based on the theory of distributed cognition that provides seven segments for reading research articles. The study hypothesized that motivation, attention, and interactivity are essential factors that affect students' thinking skills for understanding e-learning content. To investigate and confirm the effect of these factors on the students' thinking skills, we designed a survey and analyzed the responses of 210 university students concerning the proposed structured representation. The results revealed that motivation, attention, and interactivity did contribute to the students' thinking skills. They also demonstrated that the structured representation helped students achieve an adequate level of thinking skills as they read research articles, which had a positive effect on their understanding. This finding demonstrates that structured representation has significant potential as a learning tool and that structure-based e-learning can influence students' metacognitive activities and facilitate their understanding.     

Developing a survey for assessing preferences in constructivist context‐aware ubiquitous learning environments

This study developed a survey to explore students' preferences in constructivist context‐aware ubiquitous learning environments. A constructivist context‐aware ubiquitous learning (u‐learning) environment survey (CULES) was developed, consisting of eight scales, including ease of use, continuity, relevance, adaptive content, multiple sources, timely guidance, student negotiation, and inquiry learning. The survey responses were gathered from 215 university students from five universities in Taiwan. The students all had actual experience of using u‐learning systems in u‐learning environments. Both exploratory and confirmatory factor analyses showed that the CULES had high reliability and validity. The structural model revealed that the provision of realistic and close‐to‐real‐life information could enhance students' preferences for timely guidance, student negotiation, and inquiry‐learning activities. In addition, the attainment of inquiry learning is quite challenging when designing u‐learning activities, as it involves the enhancement of the other CULES scales.