The benefits of more electronic screen space on students’ retention of material in classroom lectures

Many lecture halls today have two or more screens to be used by instructors for lectures with computer-supported visual aids. Typically, this additional screen real estate is not used to display additional information; rather a single stream of information is projected on all screens. We describe a controlled laboratory study that empirically assesses the effect on students learning of using the increased classroom screen real estate to project an additional stream of information. We measured how well participants learned from a two-stream presentation compared to a one-stream presentation duplicated on both screens. Data indicate that using extra screen real estate can indeed improve learning. In particular, learning was most improved when pertinent prior information was shown alongside currently explained information. There is also evidence that visual comparisons were improved with parallel viewing using extra screen real estate. Subjective data gathered from participants showed a strong preference for learning with two streams of content over a regular one-stream presentation.     

Extracting relevance and affect information from physiological text annotation

We present physiological text annotation, which refers to the practice of associating physiological responses to text content in order to infer characteristics of the user information needs and affective responses. Text annotation is a laborious task, and implicit feedback has been studied as a way to collect annotations without requiring any explicit action from the user. Previous work has explored behavioral signals, such as clicks or dwell time to automatically infer annotations, and physiological signals have mostly been explored for image or video content. We report on two experiments in which physiological text annotation is studied first to (1) indicate perceived relevance and then to (2) indicate affective responses of the users. The first experiment tackles the user’s perception of relevance of an information item, which is fundamental towards revealing the user’s information needs. The second experiment is then aimed at revealing the user’s affective responses towards a -relevant- text document. Results show that physiological user signals are associated with relevance and affect. In particular, electrodermal activity was found to be different when users read relevant content than when they read irrelevant content and was found to be lower when reading texts with negative emotional content than when reading texts with neutral content. Together, the experiments show that physiological text annotation can provide valuable implicit inputs for personalized systems. We discuss how our findings help design personalized systems that can annotate digital content using human physiology without the need for any explicit user interaction.     

A Web 2.0-based collaborative annotation system for enhancing knowledge sharing in collaborative learning environments

A limitation of current Web-based collaborative learning is the restricted ability of students to create and share individual annotations with annotated documents. Applying Web 2.0 collaborative annotation systems and analyzing students’ annotation behavior has attracted attention to improve collaborative learning. This study designed a personalized annotation management system 2.0 (PAMS 2.0) for managing, sharing, and reusing individual and collaborative annotations as well as providing a shared mechanism for discussion about shared annotations among multiple users.     

基于图学习的自动图像标注

自动图像标注是图像检索任务中重要而具有挑战性的工作.文中首先讨论并解释了自动图像标注问题,通过总结现有的研究工作,提出了一种基于图学习的图像标注框架.在该框架下,图像标注被分为两个阶段来完成,即基本图像标注与图像标注改善.其中,前者是通过以图像间相似性为依据的图学习过程来提供图像的初始标注,而后者是通过以词汇间语义相关性为依据的图学习过程来改善前者取得的标注结果.该框架主要涉及到图像与文本词汇两种媒体的内部和相互之间的各种关系的估计问题.基于此,作者又给出了针对上述各子问题的改进方法,并将它们综合起来实现了有效的图像标注.最后,通过Corel图像集与网络数据集上一系列实验结果,验证了该模型框架及所提出解决方案的有效性.     

From paper-and-pen annotations to artefact-based mobile learning

Making annotations on books and learning materials is part of students' everyday life. Although there are many computer-based annotation systems, many people prefer to print digital documents and make annotations on physical paper. We introduce a classification scheme for computer-based annotation systems and provide an overview of different systems using this scheme. During our investigations, we identified several shortcomings in existing annotation systems: limitation to single document formats, no capturing of relations to reflect lateral reading, loss of context information an annotation was created in and limited sharing capabilities among collaborative users. To overcome these shortcomings, we introduce the concepts of Anchor Points , Artefacts and Spaces . Anchor Points define locations in arbitrary documents. Artefacts capture annotations on multiple Anchor Points in several documents, allowing students to map their current working context that typically contains several open documents and a relationship between them. Spaces are repositories for the learning materials and annotations that are automatically replicated among collaborative mobile users.     

Understanding metacomprehension accuracy within video annotation systems

While much research has been conducted on annotations and note-taking in the context of learning from static text and images, the influence of annotating instructional video upon metacognitive monitoring and learning remains unclear. This is an important issue because video annotation systems are gaining popularity in the practice of online and blended learning, but the conditions for effective implementation of such systems are still not well understood. This study explored the influence of video annotation conditions upon metacomprehension accuracy and learning performance with a group of 81 undergraduate students of various majors. Findings suggest that video annotation systems designed for simultaneous notetaking may have a deleterious effect upon metacognitive monitoring in general and metacomprehension in particular. Text-based strategies used to improve metacomprehension accuracy such as a delay in the production of a keyword to summarize the essence of an instructional topic do not appear to impact metacognitive performance in the context of video annotation. Interestingly, participants in the control condition (without annotation) performed as well in both learning performance and metacomprehension accuracy as their counterparts. These findings have implications for the design of video annotation systems and learner best practices in the use of video annotation, particularly in online and blended learning formats.     

Evaluation of a Geospatial Annotation Tool for Unmanned Vehicle Specialist Interface

Map-based interfaces have been developed to support collaborative control of unmanned vehicles (i.e., robots). Annotation on the map (or geospatial annotation) has been proposed as an effective way to support team collaboration; however, there is a lack of research focused on the design of geospatial annotation tools to promote usability and task performance. The utility of location reference in geospatial annotations for communication and information sharing is the focus of this article. Two annotation tool designs were developed. The annotation contents were directly anchored on the map in the first design, whereas in the second design annotations were summarized in a separate panel on the interface. Evaluation participants followed instructions from a simulated team leader and assigned unmanned vehicles to different tasks for two simulated scenarios that include searching for victims and collecting hazardous materials samples. The results demonstrate the potential of using geospatial annotations to enrich communication and support map-based unmanned vehicle control. Participants appreciated the direct location reference feature of the first design and had generally shorter response time, but felt that the second design provided better usability and lower task workload. These results suggest that the user experience depends on the manner of obtaining information from the annotation tools, and the integration of the tool with user's task flow and other interface components, such as the map display. The presented results can be used as a basis for designing geospatial annotation tools for team collaboration that better fit user needs and preferences.     

A study of multimedia annotation of Web-based materials

Web-based learning has become an important way to enhance learning and teaching, offering many learning opportunities. A limitation of current Web-based learning is the restricted ability of students to personalize and annotate the learning materials. Providing personalized tools and analyzing some types of learning behavior, such as students’ annotation, has attracted attention as a means to enhance Web-based learning. We describe a Web-based tool for creating and sharing annotations and investigate the effect on learning of its use with college students. First, an annotation tool was designed and implemented for the research. Second, learning support mechanisms, including full and group annotation sharing, were developed to promote students’ motivation for annotation. Lastly, experiments with individual and shared annotation were conducted and the results show that the influence of annotation on learning performance becomes stronger with the use of sharing mechanisms. We conclude that there is value in further study of collaborative learning through shared annotation.     

Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms

Collaboratively annotating digital texts allows learners to add valued information, share ideas, and create knowledge. However, excessive annotations and poor-quality annotations in a digital text may cause information overload and divert attention from the main content. The increased cognitive load ultimately reduces the effectiveness of collaborative annotations in promoting reading comprehension. Thus, this work develops a web-based collaborative reading annotation system (WCRAS-TQAFM) with two quality annotation filtering mechanisms—high-grade and master annotation filters—to promote the reading performance of learners. Ninety-seven students from three classes of a senior high school in Taiwan were invited to participate in an 80-min reading activity in which individual readers use WCRAS with or without annotation filters. Analytical results indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Also, the Cohen’s kappa statistics was used for assessing whether the annotation selected by the high-grade annotation filter is in agreement with the annotations selected by a domain expert. The statistic results indicate that the proposed high-grade annotation filter is valid to some degree. Finally, neither of the proposed quality annotation filtering approaches significantly reduces cognitive load.     

Semantic Annotation and Retrieval of Music and Sound Effects

We present a computer audition system that can both annotate novel audio tracks with semantically meaningful words and retrieve relevant tracks from a database of unlabeled audio content given a text-based query. We consider the related tasks of content-based audio annotation and retrieval as one supervised multiclass, multilabel problem in which we model the joint probability of acoustic features and words. We collect a data set of 1700 human-generated annotations that describe 500 Western popular music tracks. For each word in a vocabulary, we use this data to train a Gaussian mixture model (GMM) over an audio feature space. We estimate the parameters of the model using the weighted mixture hierarchies expectation maximization algorithm. This algorithm is more scalable to large data sets and produces better density estimates than standard parameter estimation techniques. The quality of the music annotations produced by our system is comparable with the performance of humans on the same task. Our ldquoquery-by-textrdquo system can retrieve appropriate songs for a large number of musically relevant words. We also show that our audition system is general by learning a model that can annotate and retrieve sound effects.     

On the impact of knowledge extraction and aggregation on crowdsourced annotation of visual artworks

Cultural heritage institutions more and more provide online access to their collections. Collections containing visual artworks need detailed and thorough annotations of the represented visual objects (e.g. plants or animals) to enable human access and retrieval. To make these suitable for access and retrieval, visual artworks need detailed and thorough annotations of the visual classes. Crowdsourcing has proven a viable tool to cater for the pitfalls of automatic annotation techniques. However, differently from traditional photographic image annotation, the artwork annotation task requires workers to possess the knowledge and skills needed to identify and recognise the occurrences of visual classes. The extent to which crowdsourcing can be effectively applied for artwork annotation is still an open research question. Based on a real-life case study from Rijksmuseum Amsterdam, this paper investigates the performance of a crowd of workers drawn from the CrowdFlower platform. Our contributions include a detailed analysis of crowd annotations based on two annotation configurations and a comparison of these crowd annotations with the ones from trusted annotators. In this study we apply a novel method for the automatic aggregation of local (i.e. bounding box) annotations, and we study how different knowledge extraction and aggregation configurations affect the identification and recognition aspects of artwork annotation. Our work sheds new light on the process of crowdsourcing artwork annotations, and shows how techniques that are effective for photographic image annotation cannot be straightforwardly applied to artwork annotation, thus paving the way for new research in the area.     

Part and Attribute Discovery from Relative Annotations

Part and attribute based representations are widely used to support high-level search and retrieval applications. However, learning computer vision models for automatically extracting these from images requires significant effort in the form of part and attribute labels and annotations. We propose an annotation framework based on comparisons between pairs of instances within a set, which aims to reduce the overhead in manually specifying the set of part and attribute labels. Our comparisons are based on intuitive properties such as correspondences and differences, which are applicable to a wide range of categories. Moreover, they require few category specific instructions and lead to simple annotation interfaces compared to traditional approaches. On a number of visual categories we show that our framework can use noisy annotations collected via “crowdsourcing” to discover semantic parts useful for detection and parsing, as well as attributes suitable for fine-grained recognition.     

Hypertext annotation: Effects of presentation formats and learner proficiency on reading comprehension and vocabulary learning in foreign languages

This study extends current knowledge by exploring the effect of different annotation formats, namely in-text annotation, glossary annotation, and pop-up annotation, on hypertext reading comprehension in a foreign language and vocabulary acquisition across student proficiencies. User attitudes toward the annotation presentation were also investigated. Data were collected from 83 non-English-majored university students in Taiwan in a 4-week period. Each week participants read 3 passages, each with different annotation formats as a treatment condition and one passage without annotation as a control condition. Posttests of reading comprehension and vocabulary recognition followed each passage. The results indicate that, for reading comprehension, the in-text format led to the lowest performance among all types of annotation, including the control condition. The best performance was observed in the condition where annotations were presented in the pop-up format. No interaction effect between format and proficiency was detected. For vocabulary acquisition, reading passages with hypermedia annotations significantly benefit vocabulary learning for participants of medium and high proficiencies compared with the control condition. No significant differences were found among the 3 formats. The beneficial effect, however, did not extend to low-proficiency participants. Participant feedback revealed a positive attitude toward annotations. Among the 3 annotation formats, the glossary type was considered the least preferred type by participants. Findings of the research provide insights on the design and instruction for online reading.     

The influence of screen size and text layout on the study of text

This study investigates the effects of screen size (12 inch versus 15 inch) and text layout (well structured and ill structured) on the learning of text presented on the monitor of a personal computer. Two aspects of learning are assessed. A summary and a multiple-choice test are employed to measure the amount of information retained. Efficacy of learning is assessed by learning time and by cognitive effort, as measured by the performance on a secondary task. The results indicate that neither screen size nor text layout has a significant influence on the required cognitive effort or on the amount of information acquired. There is, however, a significant (main) effect of screen size on learning time: subjects using a 15 inch screen need less learning time than subjects using a 12 inch screen, with no difference in learning performance. It is suggested that more efficient integration processes in constructing the semantic representation are responsible for this reduction in learning time. Implications for future research are discussed.     

Eras: Improving the quality control in the annotation process for Natural Language Processing tasks

The increasing amount of valuable, unstructured textual information poses a major challenge to extract value from those texts. We need to use NLP (Natural Language Processing) techniques, most of which rely on manually annotating a large corpus of text for its development and evaluation. Creating a large annotated corpus is laborious and requires suitable computational support. There are many annotation tools available, but their main weaknesses are the absence of data management features for quality control and the need for a commercial license. As the quality of the data used to train an NLP model directly affects the quality of the results, the quality control of the annotations is essential. In this paper, we introduce ERAS, a novel web-based text annotation tool developed to facilitate and manage the process of text annotation. ERAS includes not only the key features of current mainstream annotation systems but also other features necessary to improve the curation process, such as the inter-annotator agreement, self-agreement and annotation log visualization, for annotation quality control. ERAS also implements a series of features to improve the customization of the user’s annotation workflow, such as: random document selection, re-annotation stages, and warm-up annotations. We conducted two empirical studies to evaluate the tool’s support to text annotation, and the results suggest that the tool not only meets the basic needs of the annotation task but also has some important advantages over the other tools evaluated in the studies. ERAS is freely available at https://github.com/grosmanjs/eras.     

Retracted: Examining students' learning,perceived learning and motivation in social annotation-based translation activities

This study was conducted to examine how a collaborative social annotation tool can be used to support collaborative learning in translation instruction for EFL students. Participants were 100 undergraduate students who were English majors in a southeast university in China. An experiment with crossover design was carried out to examine the participants' learning and motivation in the social annotation-supported condition and the traditional instruction condition. The results showed that students received higher scores on their translation work and rated their perceived learning higher when they completed the translation assignments in the social annotation-support condition than in the traditional instruction condition. In addition, participants reported that they were more motivated when they completed the translation assignments in the social annotation-supported condition, though participants with different levels of language proficiency might be motivated by different reasons.     

Ease of annotation in proof-reading tasks

Two models of proof-reading tasks are explored by varying the procedures for annotating a text. One model assumes that the processes of detecting errors, recording annotations and resuming proof-reading are sets of serial processes. The other model assumes that annotation processes may overlap with reading the text. Performance when proof-reading a vertically displayed text (as on a CRT) and recording the errors on a separate sheet was compared with reading a horizontal text (as on a desk top) and recording annotations in the margins. The data supported the serial model and showed that variations in annotation procedures can yield differences in proof-reading speed of comparable magnitude to those found in an earlier study where CRT and printout displays were contrasted and the differences were attributed to legibility factors. The implications of these findings and this model of proof-reading are related to the wider issues of using electronic texts.     

Access by content to handwritten archive documents: generic document recognition method and platform for annotations

This paper presents annotations needed for handwritten archive document retrieval by content. We propose two complementary ways of producing these annotations: automatically by using document image analysis and collectively by using the Internet and manual input by users. A platform for managing these annotations is presented as well as examples of automatic annotations on civil status registers, military forms (tested on 165,000 pages) and naturalization decrees, using a generic method for structured document recognition and handwriting recognition on names. Examples of collective annotations built on automatic annotations are also given. This platform is already open to the public in the reading room of the new building of the Archives départementales des Yvelines and on the Internet. About 1,450,000 images of civil status registers are available for collective annotation as well as 105,000 pages of military forms with automatic annotation of handwritten names.     

Notes and annotations as information resources in a social networking platform

Taking notes and annotations contributes in the learning process. Many platforms are developed as Computer Supported Collaborative Learning (CSCL) thanks to advancements in new technologies. A common limitation of these platforms is the restricted ability to share/retrieve notes and annotations (Su, Yang, Hwang, & Zhang, 2010a). This is because the annotations in these platforms are disconnected from the information system and they are only accessible in the annotation system. As a result, the annotations could not be indexed as any other information resources (e.g., a document). This means that the annotations are not accessible/visible like other resources. In this paper, we present an original semantic model in which notes and annotations are modeled as information resources. The semantic model is used within the MEMORAe web platform. Then we detail an experiment of collaborative learning made within a university course using the MEMORAe web platform. The feedback of this experiment shows us that students are satisfied with the use of the MEMORAe web platform for helping them to index and retrieve notes and annotations as any information resources.