On-line peer assessment and the role of the peer feedback: A study of high school computer course

The purposes of this study were to explore the effects and the validity of on-line peer assessment in high schools and to analyze effects of various types of peer feedback on students. The participating students, a total of 184 10th graders, developed their individual course projects by involving the on-line peer assessment learning activities in a computer course. The peer assessment activities consisted of three rounds, and each of the students acted as an author and a reviewer. Research data as evaluated from peers and experts indicated that students significantly improved their projects as involving the peer assessment activities. The scores determined by the learning peers were highly correlated with those marked by the experts, indicating that peer assessment in high school could be perceived as a valid assessment method. Moreover, this study also examined the relationships between the types of peer feedbacks in which students obtained from peer assessment and the subsequent performance of their projects. We categorized peer feedbacks into four types: Reinforcing, Didactic, Corrective and Suggestive. It was found that Reinforcing peer feedback was useful in helping students’ development of better projects; however, Didactic feedback and perhaps Corrective feedback provided by peers might play an unfavorable role for subsequent improvement of students’ projects. The Suggestive feedback may be helpful in the beginning of peer assessment activities; however, in the later parts of peer assessment, the effect of this type of feedback on learning might not be significant.     

Reflective behaviors under a web-based portfolio assessment environment for high school students in a computer course

This research attempted to categorize reflection in a Web-based portfolio assessment using the Chinese Word Segmenting System (CWSS). Another aim of this research was to explore reflective performance in which individual differences were further examined. Participants were 45 eight-grade students from a junior high school taking a computer course. The study results indicated that the words used most often in reflective journals fell into cognition and evaluation categories in comparison to emotion and memory. Based on lexical attributes, reflection was thus classified into cognition, evaluation and mix. Cognition was the most common type, and evaluation, the least; emotion and memory types failed to emerge. Although reflective journals tended to be short, the average scores on reflection were acceptably high, which implied it was the quality rather than the length of a reflective journal that students were primarily concerned about. In addition, significant group differences were detected in terms of word counts and reflection scores. The reviews of peer reflections were seldom fulfilled, and covered merely one-third of the peer work; there were significant group differences related to the number of reviews. The duration of peer reflection reviews was usually short, and again, significant differences were found across various duration groups.     

A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students

This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and peer-assessment; three teachers reviewed portfolios and evaluated learning performances. There were significant differences in the results of the three assessment methods, among which teacher-raters adopted the most rigorous scoring standards, while peer-raters tended to use the most lax standards. The results of self- and teacher-assessment were discovered to be consistent; however, consistency was not discovered between self- and peer-assessment as well as peer- and teacher-assessment. In analyzing their consistency with the end-of-course examination, teacher- and self-assessment demonstrated high consistency, whereas peer-assessment showed a low level of consistency.     

Web-based peer assessment: feedback for students with various thinking-styles

Abstract This study used aptitude treatment interaction design to examine how feedback formats (specific vs. holistic) and executive thinking styles (high vs. low) affect web-based peer assessment. An Internet-based (anonymous) peer-assessment system was developed and used by 58 computer science students who submitted assignments for peer review. The results indicated that while students with high executive thinking styles significantly improved over two rounds of peer assessment, low executive students did not improve through the cycles. In addition, high executive students contributed substantially better feedback than their low executive counterparts. In the second round of peer assessment, thinking style and feedback format interactively affected student learning. Low executive students receiving specific feedback significantly outperformed those receiving holistic feedback. In receiving holistic feedback, high executive thinkers outperformed their low executive counterparts. This study suggests that future web-based peer assessment adopts a specific feedback format for all students.     

Implementation of Web-based dynamic assessment in facilitating junior high school students to learn mathematics

This research adopts the Graduated Prompting Assessment Module of the WATA system (GPAM-WATA) and applies it to the remedial teaching of junior high school mathematics. The theoretical basis of the development of GPAM-WATA is the idea of ‘graduated prompt approach’ proposed by Campione and Brown, 1985, Campione and Brown, 1987. In GPAM-WATA, when examinees fail to answer items correctly, they obtain instructional prompts (IPs) in a graduated way. This research developed the contents of IPs based on the mathematical problem-solving theory of Mayer (1992, pp. 458–460). A quasi-experimental design was adopted. Ninety-six junior high school seventh graders from three different classes participated in this research. The three classes were randomly divided into the GPAM-WATA group (n = 31), the N-WBT group (n = 31) and the PPT group (n = 34). All students received traditional mathematics instruction from the same teacher. After traditional mathematics instruction, all students took the pre-test of the summative assessment. The students in the three different groups then respectively received remedial teaching in the form of GPAM-WATA, normal Web-based test (N-WBT), and paper-and-pencil test (PPT). After the remedial teaching, all students took the post-test of the summative assessment. The results indicate that compared with other groups, performing remedial teaching using GPAM-WATA has significantly better effectiveness. Moreover, it is found that the IPs in GPAM-WATA are effective in remedial teaching for not only those students most lacking in different types of mathematical problem-solving knowledge but also all the other students.     

Developing Web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-Learning environment

This research refers to the self-regulated learning strategies proposed by Pintrich (1999) in developing a multiple-choice Web-based assessment system, the Peer-Driven Assessment Module of the Web-based Assessment and Test Analysis system (PDA-WATA). The major purpose of PDA-WATA is to facilitate learner use of self-regulatory learning behaviors to perform self-regulated learning and in turn improve e-Learning effectiveness. PDA-WATA includes five main strategies: ‘Adding Answer Notes,’ ‘Stating Confidence,’ ‘Reading Peer Answer Notes,’ ‘Recommending Peer Answer Notes’ and ‘Querying Peers’ Recommendation on Personal Answer Notes’. Using these strategies, examinees are allowed to add answer notes to explain why they chose a certain option as the correct answer and state their confidence in their own answer and answer notes, for peers’ reference. In addition to reading peer answer notes, examinees can also recommend peer answer notes as valuable references. The recommendation information can also be queried by all examinees. Quasi-experimental design was adopted to understand the effectiveness of PDA-WATA in facilitating learner use of self-regulatory learning behaviors to perform self-regulated learning and in improving learner e-Learning effectiveness. Participants were 123 seventh-grade junior high school students from four classes. These four classes were randomly divided into the PDA-WATA group (n = 63) and the N-WBT group (n = 60). Before e-Learning instruction, all students took the pre-test of the Learning Process Inventory (LPI), used to understand how often learners use self-regulatory learning behaviors in the learning process, and the pre-test of the summative assessment. After a two-week e-Learning instruction, the students all took the post-test of the LPI and the summative assessment. Results indicate that students in the PDA-WATA group appear to be more willing to take the Web-based formative assessment than students in the N-WBT group. In addition, PDA-WATA appears to be significantly more effective than N-WBT in facilitating learner use of self-regulatory learning behaviors to perform self-regulated learning and in improving their e-Learning effectiveness. Moreover, this research also finds that in the PDA-WATA group, there is no significant difference between the learning effectiveness of students with a low level of self-regulated learning and students with a high level of self-regulated learning, but similar result cannot be found in the N-WBT group.     

A study on learning effect among different learning styles in a Web-based lab of science for elementary school students

The purpose of this study is to explore the learning effect related to different learning styles in a Web-based virtual science laboratory for elementary school students. The online virtual lab allows teachers to integrate information and communication technology (ICT) into science lessons. The results of this experimental teaching method demonstrate that: (a) students in the experimental group using the online virtual lab achieved better grades than those in the control group under traditional class instruction, (b) in the experimental group, grade achievements of students having different learning styles were not significantly different from each other leading us to conclude that the Web-based virtual learning environment is suitable for various learning styles, (c) students with the “accommodator” learning style made the most significant achievements in this study, the scores obtained by the experimental group being remarkably better than those in the control group, and (d) up to 75% of the students surveyed indicated that they preferred using the Web-based virtual lab to reading textbooks only. The results of the experimental teaching and the survey show the feasibility and effectiveness of the Web-based learning environment being studied. It encourages further development of the Web-based virtual lab.     

Web-based dynamic assessment: Taking assessment as teaching and learning strategy for improving students’ e-Learning effectiveness

This research combines the idea of cake format dynamic assessment defined by Sternberg and Grigorenko (2001) and the ‘graduated prompt approach’ proposed by 9 and 10 to develop a multiple-choice Web-based dynamic assessment system. This research adopts a quasi-experimental design to investigate the effectiveness of this Web-based dynamic assessment system (GPAM-WATA) and normal Web-based test (N-WBT). One hundred and sixteen sixth grade elementary students from four classes participated in this research. These four classes were randomly divided into the GPAM-WATA group and N-WBT group. Before e-Learning instruction, all the students took the prior knowledge assessment and the pre-test of the summative assessment. After 2-week e-Learning instruction, all the students took the post-test of the summative assessment. The research findings show that students in the GPAM-WATA group experience better e-Learning effectiveness than those in the N-WBT group. GPAM-WATA is also found to be effective in improving the e-Learning effectiveness of students with low-level prior knowledge.     

A new fuzzy peer assessment methodology for cooperative learning of students

In this paper, a new fuzzy peer assessment methodology that considers vagueness and imprecision of words used throughout the evaluation process in a cooperative learning environment is proposed. Instead of numerals, words are used in the evaluation process, in order to provide greater flexibility. The proposed methodology is a synthesis of perceptual computing (Per-C) and a fuzzy ranking algorithm. Per-C is adopted because it allows uncertainties of words to be considered in the evaluation process. Meanwhile, the fuzzy ranking algorithm is deployed to obtain appropriate performance indices that reflect a student's contribution in a group, and subsequently rank the student accordingly. A case study to demonstrate the effectiveness of the proposed methodology is described. Implications of the results are analyzed and discussed. The outcomes clearly demonstrate that the proposed fuzzy peer assessment methodology can be deployed as an effective evaluation tool for cooperative learning of students.     

On-line peer assessment and the role of the peer feedback: A study of high school computer course

The purposes of this study were to explore the effects and the validity of on-line peer assessment in high schools and to analyze effects of various types of peer feedback on students. The participating students, a total of 184 10th graders, developed their individual course projects by involving the on-line peer assessment learning activities in a computer course. The peer assessment activities consisted of three rounds, and each of the students acted as an author and a reviewer. Research data as evaluated from peers and experts indicated that students significantly improved their projects as involving the peer assessment activities. The scores determined by the learning peers were highly correlated with those marked by the experts, indicating that peer assessment in high school could be perceived as a valid assessment method. Moreover, this study also examined the relationships between the types of peer feedbacks in which students obtained from peer assessment and the subsequent performance of their projects. We categorized peer feedbacks into four types: Reinforcing, Didactic, Corrective and Suggestive. It was found that Reinforcing peer feedback was useful in helping students’ development of better projects; however, Didactic feedback and perhaps Corrective feedback provided by peers might play an unfavorable role for subsequent improvement of students’ projects. The Suggestive feedback may be helpful in the beginning of peer assessment activities; however, in the later parts of peer assessment, the effect of this type of feedback on learning might not be significant.     

Improving student performance in a first-year geography course: Examining the importance of computer-assisted formative assessment

The main objective of this paper is to examine the effectiveness of computer-assisted formative assessment in a large, first-year undergraduate geography course. In particular, the paper evaluates the impact of computer-assisted multiple-choice practice tests on student performance in the course as well as student opinions of this type of formative assessment in two academic years (2008 and 2009). The multiple-choice questions included in the formative assessment vary in their level of difficulty and range from those that focus on knowledge and comprehension to those that focus on application and analysis. While the use of the computer-assisted practice tests is completely voluntary over 50 percent of students used them. Feedback questionnaires from both academic years reveals that students are overwhelmingly positive with over 95 percent indicating that the computer-assisted practice tests assist them in identifying their strengths and weaknesses and help them prepare for in-class midterms and final exams. Statistical analysis of in-class performance on midterms shows that students who use the computer-assisted practice quizzes earn significantly higher grades (i.e., equivalent to three letter grades) than those students who do not. The results of the research demonstrate that computer-assisted formative assessment (in this case practice tests) has a positive impact on student performance.     

The efficacy of a web-based gambling intervention program for high school students: A preliminary randomized study

Early onset in adolescent gambling involvement can be a precipitator of later gambling problems. The aim of the present study was to test the preliminary efficacy of a web-based gambling intervention program for students within a high school-based setting. Students attending a high school in Italy (N = 168) participated in the present study (58% male – age, M = 15.01; SD = 0.60). Twelve classes were randomly assigned to one of two conditions: intervention (N = 6; 95 students) and control group (N = 6; 73 students). Both groups received personalized feedback and then the intervention group received online training (interactive activities) for three weeks. At a two-month follow-up, students in the intervention group reported a reduction in gambling problems relative to those in the control group. However, there were no differences in gambling frequency, gambling expenditure, and attitudes toward the profitability of gambling between the two groups. In addition, frequent gamblers (i.e., those that gambled at least once a week at baseline) showed reductions in gambling problems and gambling frequency post-intervention. Frequent gamblers that only received personalized feedback showed significantly less realistic attitudes toward the profitability of gambling post-intervention. The present study is the first controlled study to test the preliminary efficacy of a web-based gambling intervention program for students within a high school-based setting. The results indicate that a brief web-based intervention delivered in the school setting may be a potentially promising strategy for a low-threshold, low-cost, preventive tool for at-risk gambling high school students.     

Developing a diagnosis system of work‐related capabilities for students: A computer‐assisted assessment

A gap exists between students' employment needs and higher education offerings. Thus, developing the capability to meet the learning needs of students in supporting their future aspirations should be facilitated. To bridge this gap in practice, this study uses multiple methods (i.e., nominal group technique and instructional systems development) and robust tests (e.g., reliability and validity) to develop a diagnosis system of work‐related capabilities. The research participants are twofold. At the early stage, this study invites three professors and three senior Human Resources managers to join the panel and help this study develop required capabilities. At the design and implementation stages, the research participants are freshman students at a famous university in Taiwan. This system enables college students to evaluate their capability scores by comparing themselves with the performances of prior students. The diagnosis results not only help students make learning plans during their university education, but also assist educators in adjusting the structure of the curriculum. Future studies are invited to use or adapt the assessment of work‐related capabilities, as well as proposed system procedure and architecture this study developed.     

A comparative study of different machine learning methods for landslide susceptibility assessment: A case study of Uttarakhand area (India)

Landslide susceptibility assessment of Uttarakhand area of India has been done by applying five machine learning methods namely Support Vector Machines (SVM), Logistic Regression (LR), Fisher's Linear Discriminant Analysis (FLDA), Bayesian Network (BN), and Naïve Bayes (NB). Performance of these methods has been evaluated using the ROC curve and statistical index based methods. Analysis and comparison of the results show that all five landslide models performed well for landslide susceptibility assessment (AUC = 0.910–0.950). However, it has been observed that the SVM model (AUC = 0.950) has the best performance in comparison to other landslide models, followed by the LR model (AUC = 0.922), the FLDA model (AUC = 0.921), the BN model (AUC = 0.915), and the NB model (AUC = 0.910), respectively.     

Reliability and validity of self-reported assessment of exposure and outcome variables for manual lifting tasks: a preliminary investigation

The reliability and validity of self-reported assessment of exposure and outcome variables were examined for manual lifting activities among ten physiotherapists. In this study, the participants evaluated the effects of five lifting variables on perceived effort, twice separated by a one-week period. One hundred and sixty-two lifting conditions were evaluated by each subject. The exposure and outcome lifting variables were described in linguistic terms. Intra-class correlation coefficient (ICC(1,1)) analysis revealed a mean value of 0.62 for all lifting activities. The self-reported assessment was cross-validated with the NIOSH lifting index by mapping the linguistic variables into numerical ranges. Moderate correlations (r = 0.54 and 0.53, p<0.01) were obtained between perceived physical exertion/perceived risk and lifting index. The findings of this study provide preliminary indications that human-based methodologies may be further explored on experienced workers.     

An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing

This mixed-method case study examined the potential of computer-assisted, math game making activities in facilitating design-based math learning for school children. Sixty-four middle school children participated in Scratch-based, math game making activities. Data were collected via activity and conversation observation, artifact analysis, interviewing, and survey. The study findings indicated that participants developed significantly more positive dispositions toward mathematics after computer game making. The study also found that experience-driven game design processes helped to activate children's reflection on everyday mathematical experiences. Mathematical thinking and content experience were intertwined within the process of computer game authoring. On the other hand, children designers were involved in game-world and story crafting more than mathematical representation. And it was still challenging for them to perform computer game coding with abstract reasoning.     

Unethical computer using behavior scale: A study of reliability and validity on Turkish university students

This study was carried out in a Turkish university with 216 undergraduate students of computer technology as respondents. The study aimed to develop a scale (UECUBS) to determine the unethical computer use behavior. A factor analysis of the related items revealed that the factors were can be divided under five headings; intellectual property, social impact, safety and quality, net integrity and information integrity.     

Mobile‐Based micro‐Learning and Assessment: Impact on learning performance and motivation of high school students

Mobile‐based micro‐learning has gained a lot of attention lately, especially for work‐based and corporate training. It combines features of mobile learning and micro‐learning to deliver small learning units and short‐term learning activities. The current study uses the lens of the Self‐Determination Theory of motivation and proposes a series of Mobile‐Based micro‐Learning and Assessment (MBmLA) homework activities to improve high school students' motivation and learning performance in science. An experiment was conducted to evaluate the effectiveness of the proposed approach. One hundred and eight students of a senior‐level high school in Europe were randomly assigned into either a control condition (conventional paper‐based homework approach) or an experimental (MBmLA approach) condition. The study carried out for a period of 5 weeks. From the experimental results, it was found that, in comparison to the conventional paper‐based approach, the proposed MBmLA approach enhanced students' basic psychological needs of self‐perceived autonomy, competence, and relatedness and improved students' exam performance in terms of factual knowledge. Moreover, students self‐reported greater learning satisfaction with the mobile‐based microassessment and micro‐learning homework tasks. Implications on educational practices as well as future research are discussed.     

A case study of the in-class use of a video game for teaching high school history

This study examines the case of a sophomore high school history class where Making History, a video game designed with educational purposes in mind, is used in the classroom to teach about World War II. Data was gathered using observation, focus group and individual interviews, and document analysis. The high school was a rural school located in a small town in the Midwestern United States. The teacher had been teaching with the game for several years and spent one school week teaching World War II, with students playing the game in class for three days of that week. The purpose of this study was to understand teacher and student experiences with and perspectives on the in-class use of an educational video game. Results showed that the use of the video game resulted in a shift from a traditional teacher-centered learning environment to a student-centered environment where the students were much more active and engaged. Also, the teacher had evolved implementation strategies based on his past experiences using the game to maximize the focus on learning.     

A case study of online instructional collaborative discussion activities for problem-solving using situated scenarios: An examination of content and behavior cluster analysis

In some higher education courses that focus on case studies, teachers can provide situated scenarios (such as business bottlenecks and medical cases) and problem-solving discussion tasks for students to promote their cognitive skills. There is limited research on the content, performance, and behavioral patterns of teaching using online discussions and integrated situated scenarios. This case study empirically explored the learning process of adopting collaborative online instructional discussion activities for the purpose of problem-solving using situated scenarios in a higher education course. Thirty-two university students carried out problem-solving activities on case scenarios and problem-solving tasks assigned by the teacher on the discussion forum. Two forms of instructional activities were considered: 1) problem-solving in a given scenario and 2) problem-solving when learners play roles in a given scenario. The activities mentioned above were each implemented for a week.