Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms

Collaboratively annotating digital texts allows learners to add valued information, share ideas, and create knowledge. However, excessive annotations and poor-quality annotations in a digital text may cause information overload and divert attention from the main content. The increased cognitive load ultimately reduces the effectiveness of collaborative annotations in promoting reading comprehension. Thus, this work develops a web-based collaborative reading annotation system (WCRAS-TQAFM) with two quality annotation filtering mechanisms—high-grade and master annotation filters—to promote the reading performance of learners. Ninety-seven students from three classes of a senior high school in Taiwan were invited to participate in an 80-min reading activity in which individual readers use WCRAS with or without annotation filters. Analytical results indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Also, the Cohen’s kappa statistics was used for assessing whether the annotation selected by the high-grade annotation filter is in agreement with the annotations selected by a domain expert. The statistic results indicate that the proposed high-grade annotation filter is valid to some degree. Finally, neither of the proposed quality annotation filtering approaches significantly reduces cognitive load.     

The impact of reading fluency level on interactive information retrieval

Given an information need and the corresponding set of documents retrieved, it is known that user assessments for such documents differ from one user to another. One frequent reason that is put forward is the discordance between text complexity and user reading fluency. We explore this relationship from three different dimensions: quantitative features, subjective-assessed difficulty, and reader/text factors. In order to evaluate quantitative features, we wondered whether it is possible to find differences between documents that are evaluated by the user and those that are ignored according to the complexity of the document. Secondly, a task related to the evaluation of the relevance of short texts is proposed. For this end, users evaluated the relevance of these short texts by answering 20 queries. Documents complexity and relevance assessments were done previously by some human experts. Then, the relationship between participants assessments, experts assessments and document complexity is studied. Finally, a third experimentation was performed under the prism of neuro-Information Retrieval: while the participants were monitored with an electroencephalogram (EEG) headset, we tried to find a correlation among EEG signal, text difficulty and the level of comprehension of texts being read during the EEG recording. In light of the results obtained, we found some weak evidence showing that users responded to queries according to text complexity and user’s reading fluency. For the second and third group of experiments, we administered a sub-test from the Woodcock Reading Mastery Test to ensure that participants had a roughly average reading fluency. Nevertheless, we think that additional variables should be studied in the future in order to achieve a sound explanation of the interaction between text complexity and user profile.

     

Comparative analysis of relevance feedback methods based on two user studies

Rigorous analysis of user interest in web documents is essential for the development of recommender systems. This paper investigates the relationship between the implicit parameters and user explicit rating during their search and reading tasks. The objective of this paper is therefore three-fold: firstly, the paper identifies the implicit parameters which are statistically correlated with the user explicit rating through user study 1. These parameters are used to develop a predictive model which can be used to represent users’ perceived relevance of documents. Secondly, it investigates the reliability and validity of the predictive model by comparing it with eye gaze during a reading task through user study 2. Our findings suggest that there is no significant difference between the predictive model based on implicit indicators and eye gaze within the context examined. Thirdly, we measured the consistency of user explicit rating in both studies and found significant consistency in user explicit rating of document relevance and interest level which further validates the predictive model. We envisage that the results presented in this paper can help to develop recommender and personalised systems for recommending documents to users based on their previous interaction with the system.     

Instance based personalized multi-form image browsing and retrieval

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.     

Learning personal preferences on online newspaper articles from user behaviors

This paper discusses methods by which user preferences for WWW-based newspaper articles can be learned from user behaviors. Two modes of inference were compared in an experiment: one using explicit feedback and the other using implicit feedback. In the explicit feedback mode, the users score all articles according to their relevance. In the implicit feedback mode, the user reads articles by performing scrolling and enlarging operations, and the system infers from the operations how much the user was interested in each article. Our newspaper on the WWW, called ANATAGONOMY, has a learning engine and a scoring engine on the server. The system users read daily news articles by using a WWW browser in which there is an interaction agent that monitors the user behaviors. The learning engine on the server infers user preferences from the interaction agent, and the scoring engine scores new articles and creates personalized newspaper pages based on the extracted user profiles. In an experiment, the system was able to personalize the newspaper to some extent when using only implicit feedback when some parameters were properly set, but the personalization was not as precise as it was when explicit feedback was used. By mixing explicit feedback with implicit feedback, the system could personalize newspapers quickly and precisely without requiring too much effort on the part of the users. User preferences can also be used to construct information retrieval agents or even to create cyberspace communities of the users that have similar interests. We think that the proposed technique for learning user preferences greatly enhances the value of the WWW.     

Ease of annotation in proof-reading tasks

Two models of proof-reading tasks are explored by varying the procedures for annotating a text. One model assumes that the processes of detecting errors, recording annotations and resuming proof-reading are sets of serial processes. The other model assumes that annotation processes may overlap with reading the text. Performance when proof-reading a vertically displayed text (as on a CRT) and recording the errors on a separate sheet was compared with reading a horizontal text (as on a desk top) and recording annotations in the margins. The data supported the serial model and showed that variations in annotation procedures can yield differences in proof-reading speed of comparable magnitude to those found in an earlier study where CRT and printout displays were contrasted and the differences were attributed to legibility factors. The implications of these findings and this model of proof-reading are related to the wider issues of using electronic texts.     

Constructing structure maps of multiple on-line texts

We propose that in many contexts of text use, people need to consult a mental representation of the mapping between the content of documents and their structure. We report three experiments that investigate the construction and use of such ‘structure maps.’ In each experiment people read multiple on-line texts on the same topic, and then searched for specific pieces of information in those texts. Search performance was compared with people who had not read the texts. People who had read multiple texts were, to some extent, able to recall where information was in the texts as shown by the locations in which they first searched (Experiments 1 and 2) or the number of pages opened during a search (Experiment 3). We also found that readers of multiple texts were able to find facts in those texts faster than were people who had not read the texts, and that this speedup was not a simple effect of faster reading while scanning for facts (Experiments 1 and 2) or of greater familiarity with the general topic (Experiment 3). These incidental effects of reading occurred whether or not participants were warned before reading that they would have subsequently to search the texts and were not compromised by transformations in the appearance of text (double column to single column) that disrupted the positions of facts on pages (Experiment 2). We conclude that readers spontaneously construct structure maps of multiple electronic texts, even when their reading goal stresses abstraction of meaning across sources. Structure maps likely play a vital role in many aspects of text use, such as re-reading and knowledge updating, so that their support is an important consideration in the design of on-line texts.     

融合用户评分与显隐兴趣相似度的协同过滤推荐算法

协同过滤算法是推荐系统中使用最广泛的算法,其核心是利用某兴趣爱好相似的群体来为用户推荐感兴趣的信息。传统的协同过滤算法利用用户-项目评分矩阵计算相似度,通过相似度寻找用户的相似群体来进行推荐,但是由于其评分矩阵的稀疏性问题,对相似度的计算不够准确,这间接导致推荐系统的质量下降。为了缓解数据稀疏性对相似度计算的影响并提高推荐质量,提出了一种融合用户评分与用户显隐兴趣的相似度计算方法。该方法首先利用用户-项目评分矩阵计算用户评分相似度;然后根据用户基本属性与用户-项目评分矩阵得出项目隐性属性;之后综合项目类别属性、项目隐性属性、用户-项目评分矩阵和用户评分时间,得到用户显隐兴趣相似度;最后融合用户评分相似度和用户显隐兴趣相似度得到用户相似度,并以此相似度寻找用户的相似群体以进行推荐。在数据集Movielens上的实验结果表明,相比传统算法中仅使用单一的评分矩阵来计算相似度,提出的新相似度计算方法不仅能够更加准确地寻找到用户的相似群体,而且还能够提供更好的推荐质量。     

Eras: Improving the quality control in the annotation process for Natural Language Processing tasks

The increasing amount of valuable, unstructured textual information poses a major challenge to extract value from those texts. We need to use NLP (Natural Language Processing) techniques, most of which rely on manually annotating a large corpus of text for its development and evaluation. Creating a large annotated corpus is laborious and requires suitable computational support. There are many annotation tools available, but their main weaknesses are the absence of data management features for quality control and the need for a commercial license. As the quality of the data used to train an NLP model directly affects the quality of the results, the quality control of the annotations is essential. In this paper, we introduce ERAS, a novel web-based text annotation tool developed to facilitate and manage the process of text annotation. ERAS includes not only the key features of current mainstream annotation systems but also other features necessary to improve the curation process, such as the inter-annotator agreement, self-agreement and annotation log visualization, for annotation quality control. ERAS also implements a series of features to improve the customization of the user’s annotation workflow, such as: random document selection, re-annotation stages, and warm-up annotations. We conducted two empirical studies to evaluate the tool’s support to text annotation, and the results suggest that the tool not only meets the basic needs of the annotation task but also has some important advantages over the other tools evaluated in the studies. ERAS is freely available at https://github.com/grosmanjs/eras.     

融合评分矩阵和评论文本深度神经网络推荐模型

针对个性化推荐中用户评分矩阵数据集稀疏,用户和项目描述信息未充分利用的问题,提出融合评分矩阵和评论文本的深度神经网络推荐模型(deep neural network recommendation model,DeepRec)。首先将通过数据预处理得到的用户偏好特征和项目属性特征的文本集合分别输入到卷积神经网络进行训练,得到用户和项目的深层次非线性特征,同时将评分矩阵输入多层感知机得到用户偏好隐表示,并对两种模型提取的用户偏好隐表示进行融合;其次利用多层感知机建模用户和项目隐表示对用户进行个性化推荐;最后基于三组数据集以均方根误差为评估指标进行对比实验。结果表明DeepRec的预测误差更低,有效提高了推荐的精准度。     

Comprehension and navigation of networked hypertexts

This study aims to investigate secondary school students' reading comprehension and navigation of networked hypertexts with and without a graphic overview compared to linear digital texts. Additionally, it was studied whether prior knowledge, vocabulary, verbal, and visual working memory moderated the relation between text design and comprehension. Therefore, 80 first‐year secondary school students read both a linear text and a networked hypertext with and without a graphical overview. Logfiles registered their navigation. After reading the text, students answered textbased multiple choice questions and drew mindmaps to assess their structural knowledge of each text content. It was found that both textbased and structural knowledge were lower after reading a networked hypertext than a linear text, especially in students with lower levels of vocabulary. Students took generally more time to read the hypertext than the linear text. We concluded that networked hypertexts are more challenging to read than linear texts and that students may benefit from explicit training on how to read hypertexts.     

Hypermedia annotation presentation: The effect of location and type on the EFL learners’ achievement in reading comprehension and vocabulary acquisition

This study investigated the effect of gloss presentation in different text locations whilst participants read EFL texts in a hypermedia environment. Seventy-eight undergraduate EFL learners read and summarized seven texts and completed a vocabulary assessment. The number of propositions recalled in each summary was recorded. The data suggest that reading passages with hypermedia annotations significantly benefits passage comprehension and vocabulary (compared to reading passages with no annotations). The best performance was observed in the condition where glosses were placed after the glossed word. The study also reported large observed score mean differences for the definition gloss type of 3–5-words.     

Annotations and the collaborative digital library: Effects of an aligned annotation interface on student argumentation and reading strategies

Recent research on annotation interfaces provides provocative evidence that anchored, annotation-based discussion environments may lead to better conversations about a text. However, annotation interfaces raise complicated tradeoffs regarding screen real estate and positioning. It is argued that solving this screen real estate problem requires limiting the number of annotations displayed to users. In order to understand which annotations have the most learning value for students, this paper presents two complementary studies examining the effects of annotations on students performing a reading-to-write task. The first study used think-aloud protocols and a within-subjects methodology, finding that annotations appeared to provoke students to reflect more critically upon the primary text. This effect was particularly strong when students encountered pairs of annotations presenting different viewpoints on the same section of text. Student interviews suggested that annotations were most helpful when they caused the reader to consider and weigh conflicting viewpoints. The second study used a between-subjects methodology and a more naturalistic task to provide complementary evidence that annotations encourage more reflective responses to a text. This study found that students who received annotated materials both perceived themselves and were perceived by instructors as less reliant on unreflective summary strategies than students who received the same content but in a different format. These findings indicate that the learning value of an annotation lies in its ability to provoke students to consider and weigh new perspectives on the primary text. When selected effectively, annotations provide a critical scaffolding that can support students’ critical thinking and argumentation activities. Collaborative digital libraries and applications for the Web 2.0 should be designed with this learning framework in mind.     

基于贝叶斯后验概率和非合作博弈的推荐算法

针对传统协同过滤推荐算法推荐精度较低等问题,提出一种基于贝叶斯后验概率预测和非合作博弈的个性化推荐算法。采用文件主题模型求取用户与其参加过的所有社交活动的主题分布,利用隐含主题概率分布表征用户兴趣度,根据信任传递机制求取用户的直接信任和间接信任,形成用户间的信任度;将用户的兴趣度和信任度等隐式特征赋予合理的先验分布,利用贝叶斯后验概率预测隐式特征后的显式反馈;依据显式反馈将推荐结果转化为非合作博弈中用户效益最大化的纳什均衡求解。仿真对比实验表明,与其他三种推荐算法相比该算法的查准率至少提高了3.13%,查全率至少提高了2.62%。     

新的基于日志分析的用户个性化模型

作为个性化服务技术的核心,用户模型的质量关系到个性化服务的质量。目前的用户模型大多只考虑用户的显式信息或隐式信息,很少同时考虑两者,使得检索质量不如人意。提出了一种新的基于日志分析的用户个性化模型,结合了传统的显式建模和隐式建模的优点,把显式个性化信息和隐式个性化信息通过两层树状结构结合起来。模型同时考虑了用户历史信息的长短期划分,以及检索系统返回结果的顺序和用户对结果页面的点击顺序。实验结果表明,基于该用户模型的个性化检索效果与原有检索系统的检索效果相比有显著提高。     

CASAM: collaborative human-machine annotation of multimedia

The CASAM multimedia annotation system implements a model of cooperative annotation between a human annotator and automated components. The aim is that they work asynchronously but together. The system focuses upon the areas where automated recognition and reasoning are most effective and the user is able to work in the areas where their unique skills are required. The system’s reasoning is influenced by the annotations provided by the user and, similarly, the user can see the system’s work and modify and, implicitly, direct it. The CASAM system interacts with the user by providing a window onto the current state of annotation, and by generating requests for information which are important for the final annotation or to constrain its reasoning. The user can modify the annotation, respond to requests and also add their own annotations. The objective is that the human annotator’s time is used more effectively and that the result is an annotation that is both of higher quality and produced more quickly. This can be especially important in circumstances where the annotator has a very restricted amount of time in which to annotate the document. In this paper we describe our prototype system. We expand upon the techniques used for automatically analysing the multimedia document, for reasoning over the annotations generated and for the generation of an effective interaction with the end-user. We also present the results of evaluations undertaken with media professionals in order to validate the approach and gain feedback to drive further research.     

A novel approach of making better recommendations by revealing hidden desires and information curation for users of internet of things

One of the most significant disadvantages of the Internet of Things (IoT) is the overload of information. More information makes it harder to find valuable information. Recommendation systems identify the most suitable items for a given user. The recommended result is only valid if the system users know what they want, and clearly and explicitly convey their needs to the system. Because the role of a recommendation system is to calculate the similarity between the given request and each item, and to rank the similarity, the requests and identity of items should be clear to obtain correct results. However, in most situations in which recommendations are made, requests are implicit and ambiguous. A good recommendation system should make a reliable list of items, even with ambiguous requests. This paper proposes a model of generating recommendations for implicit requests. The model employs two methods that reveals the desire of the requestor and uses content curation with a customized layout to display the recommendations. The first method for revealing the requestor’s desire is to specify the implicit request by combining the user’s customized preference with the collective intelligence. The second method for employing content curation is to arrange the recommendation for users to accept spontaneously. To persuade users, the recommendations are transformed into a layout based on a personalized cognitive bias. Through these processes, reliable and beneficial recommendations can be provided to any user even if their requests are implicit or unclear.

     

A framework for implicit human-centered image tagging inspired by attributed affect

In this paper, a framework for implicit human-centered tagging is presented. The proposed framework draws its inspiration from the psychologically established process of attribution. The latter strives to explain affect-related changes observed during an individual’s participation in an emotional episode, by bestowing the corresponding affect changing properties on a selected perceived stimulus. Our framework tries to reverse-engineer this attribution process. By monitoring the annotator’s focus of attention through gaze-tracking, we identify the stimulus attributed as the cause for the observed change in core affect. The latter is analyzed from the user’s facial expressions. Experimental results attained by a lightweight, cost-efficient application based on the proposed framework show promising accuracy in both the assessment of topical relevance and direct annotation scenarios. These results are especially encouraging given the fact that the behavioral analyzers used to obtain user affective response and eye gaze lack the level of sophistication and high cost usually encountered in the related literature.     

一种自适应语义Web检索方法

基于内容的语义Web检索只考虑内容本身,没有考虑用户的不同,不能准确反映用户需求。为此,提出一个自适应语义Web检索框架,对于Web中文文档,借助HowNet知识库给出一种本体学习方法,通过提取用户客观、显式和隐式信息建立用户信息库,并设计用户初始查询本体和个性化查询本体构建算法,从而实现用户的自适应检索。实验结果表明,该方法具有较高的检索效率。     

Personalized web-document filtering using reinforcement learning

Document filtering is increasingly deployed in Web environments to reduce information overload of users. We formulate online information filtering as a reinforcement learning problem, i.e., TD(0). The goal is to learn user profiles that best represent information needs and thus maximize the expected value of user relevance feedback. A method is then presented that acquires reinforcement signals automatically by estimating user's implicit feedback from direct observations of browsing behaviors. This "learning by observation" approach is contrasted with conventional relevance feedback methods which require explicit user feedbacks. Field tests have been performed that involved 10 users reading a total of 18,750 HTML documents during 45 days. Compared to the existing document filtering techniques, the proposed learning method showed superior performance in information quality and adaptation speed to user preferences in online filtering.