How the views of faculty can inform undergraduate Web-based research: Implications for academic writing

The steady increase in the use of Web sites as sources in undergraduate research-based papers has raised concerns about the suitability of these reference materials for citation and about the ability of undergraduates to distinguish between appropriate and inappropriate sources in academic writing. While informal means of evaluating Web sites are in existence, there is a need for Web-source assessment criteria that not only focus on the use of these particular sources in introductory undergraduate writing programs but also closely match the requirements of faculty within a specific academic field. This paper identifies elements of a prototypical rating instrument for students in Humanities courses based on the results of a three-part survey of faculty members in a Humanities Department (N = 31) and highlights the divergence between criteria that faculty found crucial and those that many undergraduate students appear to be using in their Web-based research.     

Annotations and the collaborative digital library: Effects of an aligned annotation interface on student argumentation and reading strategies

Recent research on annotation interfaces provides provocative evidence that anchored, annotation-based discussion environments may lead to better conversations about a text. However, annotation interfaces raise complicated tradeoffs regarding screen real estate and positioning. It is argued that solving this screen real estate problem requires limiting the number of annotations displayed to users. In order to understand which annotations have the most learning value for students, this paper presents two complementary studies examining the effects of annotations on students performing a reading-to-write task. The first study used think-aloud protocols and a within-subjects methodology, finding that annotations appeared to provoke students to reflect more critically upon the primary text. This effect was particularly strong when students encountered pairs of annotations presenting different viewpoints on the same section of text. Student interviews suggested that annotations were most helpful when they caused the reader to consider and weigh conflicting viewpoints. The second study used a between-subjects methodology and a more naturalistic task to provide complementary evidence that annotations encourage more reflective responses to a text. This study found that students who received annotated materials both perceived themselves and were perceived by instructors as less reliant on unreflective summary strategies than students who received the same content but in a different format. These findings indicate that the learning value of an annotation lies in its ability to provoke students to consider and weigh new perspectives on the primary text. When selected effectively, annotations provide a critical scaffolding that can support students’ critical thinking and argumentation activities. Collaborative digital libraries and applications for the Web 2.0 should be designed with this learning framework in mind.     

Individual and team annotation effects on students’ reading comprehension,critical thinking,and meta-cognitive skills

Many students enter college without the needed skills to be successful. Colleges and universities are seeking instructional interventions to address these needs. Various classes are leveraging web-based social media to provide new instructional technologies that will help students learn. This paper reports on two studies related to the potential of online social annotation for improving teaching and learning in second-semester Freshman English classes. The approach, referred to as the Social Annotation Model-Learning System (SAM-LS), combines various instructional strategies, team-based learning, and a social annotation computer-supported collaborative learning tool, HyLighter, to increase student engagement with selected essays and with classmates. SAM-LS stimulates students to actively monitor their thoughts and compare them to both peers and the instructor (or domain experts). Study 1 showed no significant difference between the SAM-LS approach and a control; however, results appear to be related to confounding factors. Study 2 showed that students achieve better outcomes on measures of reading comprehension and meta-cognitive skill, but not critical thinking, when SAM-LS activities include small team collaborations. The two studies suggest future directions for research and development of SAM-LS and the HyLighter tool.     

A Web 2.0-based collaborative annotation system for enhancing knowledge sharing in collaborative learning environments

A limitation of current Web-based collaborative learning is the restricted ability of students to create and share individual annotations with annotated documents. Applying Web 2.0 collaborative annotation systems and analyzing students’ annotation behavior has attracted attention to improve collaborative learning. This study designed a personalized annotation management system 2.0 (PAMS 2.0) for managing, sharing, and reusing individual and collaborative annotations as well as providing a shared mechanism for discussion about shared annotations among multiple users.     

Effects of disease and leader type on moderators in online support groups

This study examines the effects of illness (cancer and Parkinson’s [PD]) on three moderators, the expression of positive and negative emotions and cognitive mechanisms. Each illness makes its own unique demands on patients and may influence change mechanisms. Similarly, we are asking, what effects do the type of leadership have on mediators that have previously been linked to positive outcomes. Four types of groups were studied, professional, the wellness community (TWC) chat mixed cancer (N groups = 4) and TWC chat PD patient groups (N groups = 6). The two peer groups were bulletin boards for colorectal cancer (N groups = 1) and PD BBs (N groups = 6). The design was a 2×2, disease and leader type. Computer-based text analysis, the linguistic inquiry and word count assessed the dependent variables. The results of the Manova found that; disease is P = NS, leader type, P = .00, interaction, P = .00. The interaction between disease and leader type is statistically significant, indicating that disease in combination with whether the leader is a peer or professional effects the expression of emotions and cognitive mechanisms.     

Performance measures in courses using computer-aided personalized system of instruction

Archived data from four courses taught with computer-aided personalized system of instruction (CAPSI) – an online, self-paced, instructional program – were used to explore the relationship between objectively rescored final exam grades, peer reviewing, and progress rate – i.e., the rate at which students completed unit tests. There was a strong positive correlation (r = .68, p < .01) between rate of progress and the amount of peer reviewing students did. This was predictable because peer reviewers had to be further along in the course than the students whose unit tests they reviewed. Students who completed all the units tended to obtain higher final exam scores than those who did not. For students who completed all the units there was little difference between the final exam performance of those who had a high progress rate and those who had a low progress rate. Considering all students together there was a moderate correlation between progress rate and final exam performance (r = .36, p < .01). In addition, there was a moderate positive correlation (r = .33, p < .01) between the amount of peer reviewing students did and their performance on the final exam. This correlation was substantially reduced (r = .13, p < .20) when rate of progress was partialed out. Thus, overall, students who progressed more rapidly through the course did more peer reviewing and learned more as measured by final exam performance. Interestingly, there were students who showed good learning without much participation in course related behaviors such as completing unit tests and peer reviewing. It is concluded that the CAPSI instructional program provides a good learning environment for students who utilize its components but it also accommodates other learning styles. In addition, there may be a distinction between students who complete all assigned units and those who do not regardless of their tendencies to procrastinate.     

Construction of an instrument to measure student information and communication technology skills,experience and attitudes to e-learning

Over the past 20 years self-report measures of healthcare students’ information and communication technology skills have been developed with limited validation. Furthermore, measures of student experience of e-learning emerged but were not repeatedly used with diverse populations. A psychometric approach with five phases was used to develop and test a new self-report measure of skills and experience with information and communication technology and attitudes to computers in education. Phase 1: Literature review and identification of key items. Phase 2: Development and refinement of items with expert panel (n = 16) and students (n = 3) to establish face and content validity. Phase 3: Pilot testing of draft instrument with graduate pre-registration nursing students (n = 60) to assess administration procedures and acceptability of the instrument. Phase 4: Test–retest with further sample of graduate pre-registration nursing students (n = 70) tested stability and internal consistency. Phase 5: Main study with pre-registration nursing students (n = 458), further testing of internal consistency. The instrument proved to have moderate test–retest stability and the sub-scales had acceptable internal consistency. When used with a larger, more diverse population the psychometric properties were more variable. Further work is needed to refine the instrument with specific reference to possible cultural and linguistic response patterns and technological advances.     

Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms

Collaboratively annotating digital texts allows learners to add valued information, share ideas, and create knowledge. However, excessive annotations and poor-quality annotations in a digital text may cause information overload and divert attention from the main content. The increased cognitive load ultimately reduces the effectiveness of collaborative annotations in promoting reading comprehension. Thus, this work develops a web-based collaborative reading annotation system (WCRAS-TQAFM) with two quality annotation filtering mechanisms—high-grade and master annotation filters—to promote the reading performance of learners. Ninety-seven students from three classes of a senior high school in Taiwan were invited to participate in an 80-min reading activity in which individual readers use WCRAS with or without annotation filters. Analytical results indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Also, the Cohen’s kappa statistics was used for assessing whether the annotation selected by the high-grade annotation filter is in agreement with the annotations selected by a domain expert. The statistic results indicate that the proposed high-grade annotation filter is valid to some degree. Finally, neither of the proposed quality annotation filtering approaches significantly reduces cognitive load.     

The Role of Inference in the Temporal Annotation and Analysis of Text

In this paper we argue for the importance of doing inference over the information expressed by the annotations of temporally annotated corpora. We describe the process of inferential closure which can be applied to determine the full temporal content that follows from an annotation. We illustrate the importance of temporal inference and temporal closure in relation to three tasks, which are: (a) the comparison of different temporal annotations, (b) facilitating the manual annotation process needed to create temporally annotated corpora and (c) empirical investigations done over temporally annotated data.     

The effects of video on cognitive load and social presence in multimedia-learning

Two studies examined the use of video in multimedia learning environments. In Study 1, participants (N = 26) viewed one of two versions of a computer-based multimedia presentation: video, which included a video of a lecture with synchronized slides, or no video, which included the slides but only an audio narration of the lecture. Learning, cognitive load and social presence were assessed, but a significant difference was found only for cognitive load, with video experiencing greater cognitive load, t (24) = 2.45, p < .05. In Study 2, students (N = 25) were randomly assigned to either video or no video condition. Background knowledge and visual/verbal learning preference were assessed before viewing the presentation, and learning, cognitive load, and social presence were assessed after viewing. No significant differences were found for learning or social presence. However, a significant visual/verbal learning preference by condition interaction was found for cognitive load, F (1,21) = 4.51, p < .05: low visual-preference students experienced greater cognitive load in the video condition, while high visual-preference students experienced greater cognitive load in the no video condition.     

Cross-language transfer of semantic annotation via targeted crowdsourcing: task design and evaluation

Modern data-driven spoken language systems (SLS) require manual semantic annotation for training spoken language understanding parsers. Multilingual porting of SLS demands significant manual effort and language resources, as this manual annotation has to be replicated. Crowdsourcing is an accessible and cost-effective alternative to traditional methods of collecting and annotating data. The application of crowdsourcing to simple tasks has been well investigated. However, complex tasks, like cross-language semantic annotation transfer, may generate low judgment agreement and/or poor performance. The most serious issue in cross-language porting is the absence of reference annotations in the target language; thus, crowd quality control and the evaluation of the collected annotations is difficult. In this paper we investigate targeted crowdsourcing for semantic annotation transfer that delegates to crowds a complex task such as segmenting and labeling of concepts taken from a domain ontology; and evaluation using source language annotation. To test the applicability and effectiveness of the crowdsourced annotation transfer we have considered the case of close and distant language pairs: Italian–Spanish and Italian–Greek. The corpora annotated via crowdsourcing are evaluated against source and target language expert annotations. We demonstrate that the two evaluation references (source and target) highly correlate with each other; thus, drastically reduce the need for the target language reference annotations.     

Learning science via animated movies: Its effect on students’ thinking and motivation

Some researchers claim that animations may hinder students’ meaningful learning or evoke misunderstandings. In order to examine these assertions, our study investigated the effect of animated movies on students’ learning outcomes and motivation to learn. Applying the quantitative methodology, two pre- and post-questionnaires were administered: Science thinking skills and Motivation to learn science. Students’ overall achievement in science was examined by their report card scores. The research population (N = 1335) was divided into experimental (N = 926) and control (N = 409) groups from 11 elementary schools. Findings indicated that the use of animated movies promoted students’ explanation ability and their understanding of scientific concepts. Findings also indicated that students who studied science with the use of animated movies developed higher motivation to learn science, in terms of: self-efficacy, interest and enjoyment, connection to daily life, and importance to their future, compared to the control students. Following the definition of multimedia, the students who study with the use of animated movies, applied all three learning styles: visual, auditory and kinesthetic. The use of multimedia and the fact that the students were engaged in exploring new concepts, that were relevant to their daily life experiences, can explain the positive results.     

The effect of functional roles on perceived group efficiency during computer-supported collaborative learning: a matter of triangulation

In this article, the effect of functional roles on group performance and collaboration during computer-supported collaborative learning (CSCL) is investigated. Especially the need for triangulating multiple methods is emphasised: Likert-scale evaluation questions, quantitative content analysis of e-mail communication and qualitative analysis of open-ended questions were used. A comparison of 41 questionnaire observations, distributed over 13 groups in two research conditions – groups with prescribed functional roles (n = 7, N = 18) and nonrole groups (n = 6, N = 23) – revealed no main effect for performance (grade). Principal axis factoring of the Likert-scales revealed a latent variable that was interpreted as perceived group efficiency (PGE). Multilevel modelling (MLM) yielded a positive marginal effect of PGE. Most groups in the role condition report a higher degree of PGE than nonrole groups. Content analysis of e-mail communication of all groups in both conditions (role n = 7, N = 25; nonrole n = 6, N = 26) revealed that students in role groups contribute more ‘coordination’ focussed statements. Finally, results from cross case matrices of student responses to open-ended questions support the observed marginal effect that most role groups report a higher degree of PGE than nonrole groups.     

The Use of Embeddings to Provide a Clean Separation of Term and Annotation for Higher Order Rippling

Rippling is a proof search guidance technique with particular application to proof by mathematical induction. It is based on a concept of annotating the differences between two terms. In its original formulation this annotation was only appropriate to first-order formulae. We use a notion of embedding to adapt these annotations appropriately for higher-order syntax. This representation simplifies the theory of annotated terms, no longer requiring special substitution and unification theorems. A key feature of the representation is that it provides a clean separation of the term and the annotation. We illustrate this with selected examples using our implementation of these ideas in λClam.     

On the impact of knowledge extraction and aggregation on crowdsourced annotation of visual artworks

Cultural heritage institutions more and more provide online access to their collections. Collections containing visual artworks need detailed and thorough annotations of the represented visual objects (e.g. plants or animals) to enable human access and retrieval. To make these suitable for access and retrieval, visual artworks need detailed and thorough annotations of the visual classes. Crowdsourcing has proven a viable tool to cater for the pitfalls of automatic annotation techniques. However, differently from traditional photographic image annotation, the artwork annotation task requires workers to possess the knowledge and skills needed to identify and recognise the occurrences of visual classes. The extent to which crowdsourcing can be effectively applied for artwork annotation is still an open research question. Based on a real-life case study from Rijksmuseum Amsterdam, this paper investigates the performance of a crowd of workers drawn from the CrowdFlower platform. Our contributions include a detailed analysis of crowd annotations based on two annotation configurations and a comparison of these crowd annotations with the ones from trusted annotators. In this study we apply a novel method for the automatic aggregation of local (i.e. bounding box) annotations, and we study how different knowledge extraction and aggregation configurations affect the identification and recognition aspects of artwork annotation. Our work sheds new light on the process of crowdsourcing artwork annotations, and shows how techniques that are effective for photographic image annotation cannot be straightforwardly applied to artwork annotation, thus paving the way for new research in the area.     

Does cognitive load moderate the seductive details effect? A multimedia study

Several studies have shown that adding seductive details to instructional materials has a detrimental effect on learning. However, other studies have shown non-significant findings. The present study uses cognitive load theory as a theoretical framework to explain these controversial results in seductive details research. Using a 2 × 2 experimental design we asked a group of high-school students (N = 100) to learn about biology with a multimedia environment that manipulated the presence of seductive details (with vs. without) and the modality of the verbal information (high load, on-screen text vs. low load, narration). The findings showed that students’ learning performance was significantly higher when seductive details were presented under the low load condition (narration) as compared to all other conditions. The theoretical implications for understanding the effects of non-redundant and interesting, but irrelevant learning material are discussed and future research directions are presented.     

Eras: Improving the quality control in the annotation process for Natural Language Processing tasks

The increasing amount of valuable, unstructured textual information poses a major challenge to extract value from those texts. We need to use NLP (Natural Language Processing) techniques, most of which rely on manually annotating a large corpus of text for its development and evaluation. Creating a large annotated corpus is laborious and requires suitable computational support. There are many annotation tools available, but their main weaknesses are the absence of data management features for quality control and the need for a commercial license. As the quality of the data used to train an NLP model directly affects the quality of the results, the quality control of the annotations is essential. In this paper, we introduce ERAS, a novel web-based text annotation tool developed to facilitate and manage the process of text annotation. ERAS includes not only the key features of current mainstream annotation systems but also other features necessary to improve the curation process, such as the inter-annotator agreement, self-agreement and annotation log visualization, for annotation quality control. ERAS also implements a series of features to improve the customization of the user’s annotation workflow, such as: random document selection, re-annotation stages, and warm-up annotations. We conducted two empirical studies to evaluate the tool’s support to text annotation, and the results suggest that the tool not only meets the basic needs of the annotation task but also has some important advantages over the other tools evaluated in the studies. ERAS is freely available at https://github.com/grosmanjs/eras.     

Presenting a model of predicting computer anxiety in terms of epistemological beliefs and achievement goals

The aim of this study was to prepare a model for computer anxiety through investigating the relationship of achievement goals and epistemological beliefs with computer anxiety. In order to fulfill this, 375 undergraduate students (218 female and 157 male) from the University of Tehran were chosen through relative class sampling. They completed a questionnaire composed of an achievement goals scale, an epistemological beliefs questionnaire, and a computer anxiety scale. The results showed that mastery and performance-avoidance goals directly and epistemological beliefs indirectly, i.e.; through the mediating role of achievement goals, can significantly (p < 0.01) predict computer anxiety.     

Enhancement of spatial thinking with Virtual Spaces 1.0

Developing a software environment to enhance 3D geometric proficiency demands the consideration of theoretical views of the learning process. Simultaneously, this effort requires taking into account the range of tools that technology offers, as well as their limitations. In this paper, we report on the design of Virtual Spaces 1.0 software, a program that exercises the user’s abilities to build spatial images and to manipulate them. This paper also reports on a study that aimed to assess whether those abilities affected achievements in the spatial thinking of 10th graders who worked with the software. Additionally, we investigated whether self-regulating questions can improve the effect of exercising with Virtual Spaces 1.0. The sample was 192 students, who were randomly assigned to four groups, two of which used Virtual Spaces 1.0 (Group 1 with virtual reality and self-regulating questions N = 52, Group 2 with virtual realty only N = 52) and the other two the non-Virtual Spaces 1.0 (Group 3 self-regulating questions only N = 45, Group 4 non-treatment group N = 45). The results suggest that spatial thinking was enhanced by exercising with Virtual Spaces 1.0 and asking self-regulating questions. In addition, it was found that the self-regulating questions make the use of virtual reality more efficient, and that the influence of self-regulating questions is especially manifested in tasks that make use of high order skills.     

The effects of conceptual change texts accompanied with animations on overcoming 11th grade students’ alternative conceptions of chemical bonding

This paper aims to determine the effect of conceptual change texts accompanied with computer animations on 11th grade students’ understanding and alternative conceptions related to chemical bonding. One experimental group (EG; N = 28) and one comparison group (CG; N = 30) were used in the study. While the comparison group taught traditional instruction, the experimental group received conceptual change text accompanied with computer animations instruction. Chemical bonding achievement test was applied as pre-test, post-test and delayed test to collect data. The results of the study indicated that while there is no statistically significant difference between groups in pre-test, performance of EG students is greater than the CG ones in post-test and delayed test. And also, the EG students are better in remediating their alternative conceptions related to chemical bonding. Based on the study, it is concluded that conceptual change texts combined with computer animations can be effective instructional tools to improve students’ conceptual understanding of chemical concepts.