Implementation of Web-based dynamic assessment in facilitating junior high school students to learn mathematics

This research adopts the Graduated Prompting Assessment Module of the WATA system (GPAM-WATA) and applies it to the remedial teaching of junior high school mathematics. The theoretical basis of the development of GPAM-WATA is the idea of ‘graduated prompt approach’ proposed by Campione and Brown, 1985, Campione and Brown, 1987. In GPAM-WATA, when examinees fail to answer items correctly, they obtain instructional prompts (IPs) in a graduated way. This research developed the contents of IPs based on the mathematical problem-solving theory of Mayer (1992, pp. 458–460). A quasi-experimental design was adopted. Ninety-six junior high school seventh graders from three different classes participated in this research. The three classes were randomly divided into the GPAM-WATA group (n = 31), the N-WBT group (n = 31) and the PPT group (n = 34). All students received traditional mathematics instruction from the same teacher. After traditional mathematics instruction, all students took the pre-test of the summative assessment. The students in the three different groups then respectively received remedial teaching in the form of GPAM-WATA, normal Web-based test (N-WBT), and paper-and-pencil test (PPT). After the remedial teaching, all students took the post-test of the summative assessment. The results indicate that compared with other groups, performing remedial teaching using GPAM-WATA has significantly better effectiveness. Moreover, it is found that the IPs in GPAM-WATA are effective in remedial teaching for not only those students most lacking in different types of mathematical problem-solving knowledge but also all the other students.     

Web-based dynamic assessment: Taking assessment as teaching and learning strategy for improving students’ e-Learning effectiveness

This research combines the idea of cake format dynamic assessment defined by Sternberg and Grigorenko (2001) and the ‘graduated prompt approach’ proposed by 9 and 10 to develop a multiple-choice Web-based dynamic assessment system. This research adopts a quasi-experimental design to investigate the effectiveness of this Web-based dynamic assessment system (GPAM-WATA) and normal Web-based test (N-WBT). One hundred and sixteen sixth grade elementary students from four classes participated in this research. These four classes were randomly divided into the GPAM-WATA group and N-WBT group. Before e-Learning instruction, all the students took the prior knowledge assessment and the pre-test of the summative assessment. After 2-week e-Learning instruction, all the students took the post-test of the summative assessment. The research findings show that students in the GPAM-WATA group experience better e-Learning effectiveness than those in the N-WBT group. GPAM-WATA is also found to be effective in improving the e-Learning effectiveness of students with low-level prior knowledge.     

Web-based quiz-game-like formative assessment: Development and evaluation

This research aims to develop a multiple-choice Web-based quiz-game-like formative assessment system, named GAM-WATA. The unique design of ‘Ask-Hint Strategy’ turns the Web-based formative assessment into an online quiz game. ‘Ask-Hint Strategy’ is composed of ‘Prune Strategy’ and ‘Call-in Strategy’. ‘Prune Strategy’ removes one incorrect option and turns the original 4-option item into a 3-option one. ‘Call-in Strategy’ provides the rate at which other test takers choose each option when answering a question. This research also compares the effectiveness of three different types of formative assessment in an e-Learning environment: paper-and-pencil test (PPT), normal Web-based test (N-WBT) and GAM-WATA. In total, 165 fifth grade elementary students (from six classes) in central Taiwan participated in this research. The six classes of students were then divided into three groups and each group was randomly assigned one type of formative assessment. Overall results indicate that different types of formative assessment have significant impacts on e-Learning effectiveness and that the e-Learning effectiveness of the students in the GAM-WATA group appears to be better. Students in the GAM-WATA group more actively participate in Web-based formative assessment to do self-assessment than students in the N-WBT group. The effectiveness of formative assessment will not be significantly improved only by replacing the paper-and-pencil test with Web-based test. The strategies included in GAM-WATA are recommended to be taken into consideration when researchers design Web-based formative assessment systems in the future.     

Developing an assessment-centered e-Learning system for improving student learning effectiveness

This research used Web-based two-tier diagnostic assessment and Web-based dynamic assessment to develop an assessment-centered e-Learning system, named the ‘GPAM-WATA e-Learning system.’ This system consists of two major designs: (1) personalized dynamic assessment, meaning that the system automatically generates dynamic assessment for each learner based on the results of the pre-test of the two-tier diagnostic assessment; (2) personalized e-Learning material adaptive annotation, meaning that the system annotates the e-Learning materials each learner needs to enhance learning based on the results of the pre-test of the two-tier diagnostic assessment and dynamic assessment. This research adopts a quasi-experimental design, applying GPAM-WATA e-Learning system to remedial Mathematics teaching of the ‘Speed’ unit in an elementary school Mathematics course. 107 sixth-graders from four classes in an elementary school participated in this research (55 male and 52 female). With each class as a unit, they were divided into four different e-Learning models: (1) the personalized dynamic assessment and personalized e-Learning material adaptive annotation group (n = 26); (2) the personalized dynamic assessment and non-personalized e-Learning material adaptive annotation group (n = 28); (3) the non-personalized dynamic assessment and personalized e-Learning material adaptive annotation group (n = 26); and (4) the non-personalized dynamic assessment and non-personalized e-Learning material adaptive annotation group (n = 27). Before remedial teaching, all students took the prior knowledge assessment and the pre-test of the summative assessment and two-tier diagnostic assessment. Students then received remedial teaching and completed all teaching activities. After remedial teaching, all students took the post-test of the summative assessment and two-tier diagnostic assessment. It is found that compared to the e-Learning models without personalized dynamic assessment, e-Learning models with personalized dynamic assessment are significantly more effective in facilitating student learning achievement and improvement of misconceptions, especially for students with low-level prior knowledge. This research also finds that personalized e-Learning material adaptive annotation significantly affects the percentage of reading time students spend on the e-Learning materials they need to enhance learning. However, it does not appear to predict student learning achievement and improvement of misconceptions.     

Promoting self-regulated learning in web-based learning environments

Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and evaluate authoring tools that support teachers and students in web-based learning and instruction. This paper presents how the authoring tools of the Study2000 project can implement psychologically sound measures to promote (a) active and elaborated learning activities and (b) meta-cognitive activities in a web-based learning environment. Furthermore, it describes a study involving 72 university students in the use of such a web-based learning environment in a self-regulated learning setting at the university level. Results show that students spent almost 70% of their study time with texts, 11% with learning tasks and 12% with the active and elaborated learning tools, whereas meta-cognitive aids where hardly used (<1%).     

Learning styles and formative assessment strategy: enhancing student achievement in Web-based learning

Abstract The purpose of this research was to investigate the effects of formative assessment and learning style on student achievement in a Web-based learning environment. A quasi-experimental research design was used. Participants were 455 seventh grade students from 12 classes of six junior high schools. A Web-based course, named BioCAL, combining three different formative assessment strategies was developed. The formative assessment strategies included Formative Assessment Module of the Web-Based Assessment and Test Analysis system (FAM-WATA) (with six Web-based formative assessment strategies), Normal Module of Web-Based Assessment and Test Analysis system (N-WATA) (only with partial Web-based formative assessment strategy) and Paper and Pencil Test (PPT) (without Web-based formative assessment strategy). Subjects were tested using Kolb's Learning Style Inventory, and assigned randomly by class into three groups. Each group took Web-based courses using one of the formative assessment strategies. Pre- and post-achievement testing was carried out. A one-way ANCOVA analysis showed that both learning style and formative assessment strategy are significant factors affecting student achievement in a Web-based learning environment. However, there is no interaction between these two factors. A post hoc comparison showed that performances of the FAM-WATA group are higher than the N-WATA and PPT groups. Learners with a 'Diverger' learning style performed best followed by, 'Assimilator', 'Accommodator', and 'Converger', respectively. Finally, FAM-WATA group students are satisfied with six strategies of the FAM-WATA.     

What strategies are effective for formative assessment in an e‐learning environment?

Abstract  The web-based formative assessment developed in this research is named Formative Assessment Module of the Web-based Assessment and Test Analysis System (FAM-WATA). FAM-WATA is a multiple-choice web-based formative assessment module containing six effective strategies: 'repeat the test', 'correct answers are not given', 'query scores', 'ask questions', 'monitor answering history', and 'all pass and then reward'. This research explored the effectiveness of FAM-WATA, cognitive styles and e-learning, and student attitudes towards the six strategies of FAM-WATA. A total of 503 seventh-grade students in central Taiwan were valid in this research. Overall results indicated that students displayed a positive attitude towards the six strategies of FAM-WATA. In addition, results also showed that students in an e-learning environment equipped with FAM-WATA achieved better learning effectiveness, and that field independent students appeared to make better use of FAM-WATA strategies than field dependent students. This research concluded that FAM-WATA benefited student learning in an e-learning environment.     

Using computer supported collaborative learning strategies for helping students acquire self-regulated problem-solving skills in mathematics

The main objective of this paper is to investigate the effectiveness of a proposed computer-based instructional method in Primary Education for self-regulated problem solving. The proposed instructional method is based on Sternberg’s model of problem solving within an authentic context. It consists of three main phases: observation, collaboration and semi-structured guidance. The ultimate learning objective is to augment the autonomous problem-solving skills of primary school children. In our study the Synergo tool was used, which is a synchronous computer supported collaborative learning tool, as well as the Moodle learning management system. The context which frames the method is authentic, very close to a students’ realistic learning situation. The findings of this study advocate that students can increase their problem-solving skills in a relatively short period of time. At the same time, they can improve their approach to the solution of a given mathematical problem, performing significant signs of autonomy.     

Analyzing academic achievement of junior high school students by an improved rough set model

Rough set theory (RST) is an emerging technique used to deal with problems in data mining and knowledge acquisition. However, the RST approach has not been widely explored in the field of academic achievement. This investigation developed an improved RST (IMRST) model, which employs linear discriminant analysis to determine a reduct of RST, and analyzed the academic achievements of junior high school students in Taiwan. An interactive interface was created so that students could answer questions to predict their academic achievement and they could learn essential skills for improving their academic achievement. Empirical results showed that the IMRST model selects crucial information from data without predetermining factors and can provide accurate rates for inference rules. Hence, the developed IMRST model is a promising alternative for analyzing academic achievement data.     

Constructing and evaluating online goal-setting mechanisms in web-based portfolio assessment system for facilitating self-regulated learning

The purpose of the present study was to construct goal-setting mechanisms in a web-based portfolio assessment system (WBPAS), based on the self-regulated learning (SRL) process proposed by Zimmerman, and to examine effects of these mechanisms on SRL. The participants were two classes of 11th graders taking the website design class in a vocational high school. The participants were assigned randomly to either an experimental group (n = 40) learning with a WBPAS or a control group (n = 41) learning with a paper-based portfolio. The study results revealed the following: a) the quality of goal-setting mechanisms may facilitate SRL. b) Students setting learning goals with the WBPAS demonstrated significantly better SRL than students setting learning goals with the paper-based portfolio.     

Developing an effective support system for inquiry learning in a Web-based environment

Abstract Sixty-five teams of basic and secondary school students solved problem-solving tasks during a virtual hike in a Web-based inquiry-learning simulation 'Hiking Across Estonia'. This environment provided learners with all necessary background information for problem-solving and tools for carrying out experiments. There were 25 tasks in certain order about ecological and environmental issues. The teams were clustered according to the data about participants, the results of the pre- and post-test, and their achievement in problem-solving tasks of the virtual hike. Only two out of five clusters were regarded as effective in solving problems and analysing tables, graphs, figures, and photos. The others had difficulties in forming contextual or task and process awareness. A support system for increasing the effectiveness of inquiry learning and enhancing their development of analytical skills was developed on the basis of the strategies that the members of five clusters had used in solving the problems, their achievement in solving the tasks during the virtual hike and in the pre- and post-test, and the personal data about the teams. The support system contained different notes before solving the problems and changed sequence of the tasks on the virtual hike for some clusters. The usage of this system was evaluated in a second study with 60 teams. The comparison of two studies demonstrated significant effectiveness of the support system to both general problem-solving ability and analytical skills. The characteristics of each cluster and the influence of the support system are discussed in this paper.     

A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students

This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and peer-assessment; three teachers reviewed portfolios and evaluated learning performances. There were significant differences in the results of the three assessment methods, among which teacher-raters adopted the most rigorous scoring standards, while peer-raters tended to use the most lax standards. The results of self- and teacher-assessment were discovered to be consistent; however, consistency was not discovered between self- and peer-assessment as well as peer- and teacher-assessment. In analyzing their consistency with the end-of-course examination, teacher- and self-assessment demonstrated high consistency, whereas peer-assessment showed a low level of consistency.     

A Web-based model for developing assessment literacy of secondary in-service teachers

This research investigates the effect of a web-based model, named ‘Practicing, Reflecting, and Revising with Web-based Assessment and Test Analysis system (P2R-WATA) Assessment Literacy Development Model,’ on enhancing assessment knowledge and perspectives of secondary in-service teachers, and adopts a single group experimental research design. The WATA system provides teachers with personalized learning resources and situated environment to practice assembling, administering tests on-line, and appraising test-related statistical information to reflect and revise test items. The sample consisted of forty-seven secondary in-service mathematics and science teachers in a summer program for 36 h within six weeks. This research collects and analyzes quantitative data, including the pre-test and post-test of teachers’ knowledge and perspectives on assessment. The major results generally confirm the effectiveness of P2R-WATA model. Firstly, the assessment knowledge of the participants has improved after training, especially for teachers with low-level prior knowledge. Secondly, the findings also reveal that there is a significant improvement on teachers’ assessment perspectives.     

Impacts and preferences study for e-HO as a holistic learning environment complementary to e-Learning

We present in this study the significance and impacts of an innovative e-HO as a holistic and horizontal platform complementary to e-Learning to help realize transdisciplinary learning and foster integration of knowledge in higher education. A comprehensive investigation of a survey conducted among 647 university students for e-HO is presented with detailed studies on students’ preferences toward e-HO with respect to various demographic variables including gender, access frequency, grade level, discipline and access route concerning the five scales extracted in this particular survey. No significant gender difference surfaced, whereas access frequency turned out to be the most influential and decisive variable that has resulted in significant differences on all the scales involved, especially the characteristic ‘Leisure climate’. Further significant differences appeared on different scales with distinct natures for different demographic variables other than gender and access frequency, indicating the multidimensional characteristics of the whole endeavor and that care must be taken whenever a learning environment like this is constructed in higher education.     

Developing an efficient computational method that estimates the ability of students in a Web-based learning environment

This paper presents a computational method that can efficiently estimate the ability of students from the log files of a Web-based learning environment capturing their problem solving processes. The computational method developed in this study approximates the posterior distribution of the student’s ability obtained from the conventional Bayes Modal Estimation (BME) approach to a simple Gaussian function in order to reduce the amount of computations required in the subsequent ability update processes. To verify the correctness and usefulness of this method, the abilities of 407 college students who solved 61 physics problems in a Web-based learning environment were estimated from the log files of the learning environment. The reduced chi-squared statistic and Pearson’s chi-square test for the goodness of fit indicate that the estimated abilities were able to successfully explain the observed problem solving performance of students within error. The educational implications of estimating the ability of students in Web-based learning environments were also discussed.     

Using e-portfolio for learning goal setting to facilitate self-regulated learning of high school students

ABSTRACT

This study aims to explore the influence of using e-portfolio for learning goal setting on self-regulated learning. The subjects were 40 second-year students (18 males and 22 females) taking the ‘Webpage Design’ course of the Department of Information Technology in a vocational high school. The experimental teaching lasted for ten weeks, and the students used e-portfolio for learning goal setting. The t-test result showed that the students’ self-regulated learning after using e-portfolio for learning goal setting is better than that before goal setting. The regression result showed that goal setting performance significantly and positively influences self-regulated learning. In other words, having goal setting or better goal setting leads to better self-regulated learning. Finally, limitation and recommendations for future work are proposed.     

Reliability and validity of Web-based portfolio peer assessment: A case study for a senior high school’s students taking computer course

This study examined the reliability and validity of Web-based portfolio peer assessment. Participants were 72 second-grade students from a senior high school taking a computer course. The results indicated that: 1) there was a lack of consistency across various student raters on a portfolio, or inter-rater reliability; 2) two-thirds of the raters demonstrated inconsistency assessing different portfolios, i.e. inner-rater reliability; 3) peer-assessment scores were not consistent with teacher-assessment scores (criterion-related validity); 4) significant differences were found between peer-assessment scores and end-of-course examination scores, implying that Web-based portfolio peer assessment failed to reflect learning achievements (criterion-related validity). In short, Web-based portfolio peer assessment was not a reliable and valid method.     

Reflective behaviors under a web-based portfolio assessment environment for high school students in a computer course

This research attempted to categorize reflection in a Web-based portfolio assessment using the Chinese Word Segmenting System (CWSS). Another aim of this research was to explore reflective performance in which individual differences were further examined. Participants were 45 eight-grade students from a junior high school taking a computer course. The study results indicated that the words used most often in reflective journals fell into cognition and evaluation categories in comparison to emotion and memory. Based on lexical attributes, reflection was thus classified into cognition, evaluation and mix. Cognition was the most common type, and evaluation, the least; emotion and memory types failed to emerge. Although reflective journals tended to be short, the average scores on reflection were acceptably high, which implied it was the quality rather than the length of a reflective journal that students were primarily concerned about. In addition, significant group differences were detected in terms of word counts and reflection scores. The reviews of peer reflections were seldom fulfilled, and covered merely one-third of the peer work; there were significant group differences related to the number of reviews. The duration of peer reflection reviews was usually short, and again, significant differences were found across various duration groups.     

Exploring relations among college students’ prior knowledge,implicit theories of intelligence,and self-regulated learning in a hypermedia environment

Researchers and educators continue to explore how to assist students in the acquisition of conceptual understanding of complex science topics. While hypermedia learning environments (HLEs) afford unique opportunities to display multiple representations of these often abstract topics, students who do not engage in self-regulated learning (SRL) with HLEs often fail to achieve conceptual understanding. There is a lack of research regarding how student characteristics, such as prior knowledge and students’ implicit theory of intelligence (ITI), interact with SRL to influence academic performance. In this study, structural equation modeling was used to investigate these issues. It was found that prior knowledge and ITI were related to SRL and performance, and that SRL acted as a benevolent moderator, enhancing the positive effects of prior knowledge upon learning, and diminishing the negative effects of having a maladaptive ITI.     

A team-teaching model for practicing project-based learning in high school: Collaboration between computer and subject teachers

Project-based learning (PBL) is a highly effective means of motivating students to learn independently. However, training or encouraging teachers to practice PBL in their classrooms is challenging, especially if the educational system does not accommodate creative teaching practices. In particular, in a test-driven educational system, time constraints and an excess of teaching content makes it difficult to practice PBL at the high school level. This work presents a novel team-teaching model that is based on collaboration between subject teachers and the computer teacher to facilitate PBL in the classroom. A two-year experiment was conducted to study the feasibility of the proposed model, in which the school computer teacher conducts PBL with the 10th grade students in the first year, and subject teachers conduct PBL with the 11th grade students in the second year. Experimental results indicate that the proposed model is feasible in the given educational setting. No class time was lost, and the subject teacher successfully conducted PBL activities. Furthermore, a follow-up survey indicated that the students enjoyed the PBL activities in both classes.