Annotations for streaming video on the Web: system design and usage studies

Streaming video on the World Wide Web is being widely deployed, and workplace training and distance education are key applications. The ability to annotate video on the Web can provide significant added value in these and other areas. Written and spoken annotations can provide `in context' personal notes and can enable asynchronous collaboration among groups of users. With annotations, users are no longer limited to viewing content passively on the Web, but are free to add and share commentary and links, thus transforming the Web into an interactive medium. We discuss design considerations in constructing a collaborative video annotation system, and we introduce our prototype, called MRAS. We present preliminary data on the use of Web-based annotations for personal note-taking and for sharing notes in a distance education scenario. Users showed a strong preference for MRAS over pen-and-paper for taking notes, despite taking longer to do so. They also indicated that they would make more comments and questions with MRAS than in a `live' situation, and that sharing added substantial value.     

Self-traits and motivations as antecedents of digital media flow and addiction: The Internet,mobile phones,and video games

In the context of a new paradigm of the media environment, this study aimed to explore levels of media consumption, self-traits and individual motivations as antecedents of media flow and addiction. Focusing specifically on the use of three prominent digital media forms—the Internet, video games, and mobile phones—this study identified four psychological factors—self-esteem, self-efficacy, self-control and dispositional media use motives—and examined the influence of each on media users’ experiences of flow and levels of addiction for each of the mediums considered. The results indicated that self-control most significantly affected both users’ flow and addiction in relation to their use of the Internet, video games, and mobile phones. In addition, two dimensions of dispositional media use motives, namely pastime and self-presence, exhibited significant associations with flow and addiction for users’ consumption of two or all three media types. As expected, the amount of time spent using media was significantly related to the flow and addiction associated with all digital media usage.     

Developing a collaborative model of research using digital video

Digital video can play an important role in reconfiguring educational research methodology that challenges the perceived distinction between teachers and academics – the former normally positioned as ‘users’ of research that has been designed, interpreted and theorised by the latter, usually the ‘producers’ or academics [Triggs, P., & John, P. (2004). From transaction to transformation: ICT, professional development and the formation of communities of practice Journal of Computer Assisted Learning Special Issue 20(6) 426–439]. In this paper, we report on a collaborative project between university researchers and four teachers within the Primary and Secondary Sectors that has sought to develop new ways of asking questions about the classroom experience through the use of digital video. By each contributing their expertise and experience, teacher practitioners and researchers work in partnership to interpret and jointly construct new knowledge, understanding and classroom practice. The first part of this paper reports on the methodology, examining the role of digital video in facilitating research collaborations, while the second part provides an account of one of the participating teacher’s experiences of the process.     

Understanding metacomprehension accuracy within video annotation systems

While much research has been conducted on annotations and note-taking in the context of learning from static text and images, the influence of annotating instructional video upon metacognitive monitoring and learning remains unclear. This is an important issue because video annotation systems are gaining popularity in the practice of online and blended learning, but the conditions for effective implementation of such systems are still not well understood. This study explored the influence of video annotation conditions upon metacomprehension accuracy and learning performance with a group of 81 undergraduate students of various majors. Findings suggest that video annotation systems designed for simultaneous notetaking may have a deleterious effect upon metacognitive monitoring in general and metacomprehension in particular. Text-based strategies used to improve metacomprehension accuracy such as a delay in the production of a keyword to summarize the essence of an instructional topic do not appear to impact metacognitive performance in the context of video annotation. Interestingly, participants in the control condition (without annotation) performed as well in both learning performance and metacomprehension accuracy as their counterparts. These findings have implications for the design of video annotation systems and learner best practices in the use of video annotation, particularly in online and blended learning formats.     

A Web 2.0-based collaborative annotation system for enhancing knowledge sharing in collaborative learning environments

A limitation of current Web-based collaborative learning is the restricted ability of students to create and share individual annotations with annotated documents. Applying Web 2.0 collaborative annotation systems and analyzing students’ annotation behavior has attracted attention to improve collaborative learning. This study designed a personalized annotation management system 2.0 (PAMS 2.0) for managing, sharing, and reusing individual and collaborative annotations as well as providing a shared mechanism for discussion about shared annotations among multiple users.     

An innovative web-based collaborative platform for video annotation

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms’ evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system’s usability.     

轻量化Transformer目标跟踪数据标注算法

目的 基于深度模型的跟踪算法往往需要大规模的高质量标注训练数据集,而人工逐帧标注视频数据会耗费大量的人力及时间成本。本文提出一个基于Transformer模型的轻量化视频标注算法（Transformer-based label network,TLNet）,实现对大规模稀疏标注视频数据集的高效逐帧标注。方法 该算法通过Transformer模型来处理时序的目标外观和运动信息,并融合前反向的跟踪结果。其中质量评估子网络用于筛选跟踪失败帧,进行人工标注;回归子网络则对剩余帧的初始标注进行优化,输出更精确的目标框标注。该算法具有强泛化性,能够与具体跟踪算法解耦,应用现有的任意轻量化跟踪算法,实现高效的视频自动标注。结果 在2个大规模跟踪数据集上生成标注。对于LaSOT （large-scale single object tracking）数据集,自动标注过程仅需约43 h,与真实标注的平均重叠率（mean intersection over union,mIoU）由0.824提升至0.871。对于TrackingNet数据集,本文使用自动标注重新训练3种跟踪算法,并在3个数据集上测试跟踪性能,使用本文标注训练的模型在跟踪性能上超过使用TrackingNet原始标注训练的模型。结论 本文算法TLNet能够挖掘时序的目标外观和运动信息,对前反向跟踪结果进行帧级的质量评估并进一步优化目标框。该方法与具体跟踪算法解耦,具有强泛化性,并能节省超过90%的人工标注成本,高效地生成高质量的视频标注。     

Physical Versus Digital Sticky Notes in Collaborative Ideation

In this paper, we compare the use of physical and digital sticky notes in collaborative ideation. Inspired by a case study in a design company, we focus on a collaborative ideation task, which is often part of pair-wise brainstorming in design. For comparison and to focus on the different materiality, we developed a digital sticky notes setup designed to be as close to the physical setup as possible, not adding any advanced digital features, even though technology has reached a stage where more sophisticated use of digital sticky notes on digital boards is possible. In this paper, we present a study of ideation among pairs of experienced sticky note users. The ideation sessions were video recorded and analyzed to focus on how collaboration is supported across the two setups. Based on quantitative analyses of the participants’ interactions with the artefacts, talking patterns, position and attention during the sessions, we qualify how the differences and similarities between the 2 setups have an impact on note handling, ideation techniques, group dynamics and socio-spatial configuration, e.g. the use of the room, the boards and tables. We conclude that, while the physical setup seems more appropriate for creating notes and posting notes, the digital setup invites more note interaction. Nevertheless, we did not find significant differences in the ideation outcome (e.g., number of notes created) or how participants collaborated between the 2 setups. Hence, we argue that collaborative ideation can successfully be supported in a digital setup as well. Consequently, we believe that the next step in a technological setup is not an either or, but should bring the best of the two worlds together.     

On designing usable and secure recognition-based graphical authentication mechanisms

In this article we present the development of a new, web-based, graphical authentication mechanism called ImagePass. The authentication mechanism introduces a novel feature based on one-time passwords that increases the security of the system without compromising its usability. Regarding usability, we explore the users’ perception of recognition-based, graphical authentication mechanisms in a web environment. Specifically, we investigate whether the memorability of recognition-based authentication keys is influenced by image content. We also examine how the frequency of use affects the usability of the system and whether user training via mnemonic instructions improves the graphical password recognition rate. The design and development process of the proposed system began with a study that assessed how the users remember abstract, face or single-object images, and showed that single-object images have a higher memorability rate. We then proceeded with the design and development of a recognition-based graphical authentication mechanism, ImagePass, which uses single-objects as the image content and follows usable security guidelines. To conclude the research, in a follow-up study we evaluated the performance of 151 participants under different conditions. We discovered that the frequency of use had a great impact on users’ performance, while the users’ gender had a limited task-specific effect. In contrast, user training through mnemonic instructions showed no differences in the users’ authentication metrics. However, a post-study, focus-group analysis revealed that these instructions greatly influenced the users’ perception for memorability and the usability of the graphical authentication. In general, the results of these studies suggest that single-object graphical authentication can be a complementary replacement for traditional passwords, especially in ubiquitous environments and mobile devices.     

Zodiac: A history-based interactive video authoring system

Easy-to-use audio/video authoring tools play a crucial role in moving multimedia software from research curiosity to mainstream applications. However, research in multimedia authoring systems has rarely been documented in the literature. This paper describes the design and implementation of an interactive video authoring system called Zodiac, which employs an innovative edit history abstraction to support several unique editing features not found in existing commercial and research video editing systems. Zodiac provides users a conceptually clean and semantically powerful branching history model of edit operations to organize the authoring process, and to navigate among versions of authored documents. In addition, by analyzing the edit history, Zodiac is able to reliably detect a composed video stream's shot and scene boundaries, which facilitates interactive video browsing. Zodiac also features a video object annotation capability that allows users to associate annotations to moving objects in a video sequence. The annotations themselves could be text, image, audio, or video. Zodiac is built on top of MMFS, a file system specifically designed for interactive multimedia development environments, and implements an internal buffer manager that supports transparent lossless compression/decompression. Shot/scene detection, video object annotation, and buffer management all exploit the edit history information for performance optimization.     

Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms

Collaboratively annotating digital texts allows learners to add valued information, share ideas, and create knowledge. However, excessive annotations and poor-quality annotations in a digital text may cause information overload and divert attention from the main content. The increased cognitive load ultimately reduces the effectiveness of collaborative annotations in promoting reading comprehension. Thus, this work develops a web-based collaborative reading annotation system (WCRAS-TQAFM) with two quality annotation filtering mechanisms—high-grade and master annotation filters—to promote the reading performance of learners. Ninety-seven students from three classes of a senior high school in Taiwan were invited to participate in an 80-min reading activity in which individual readers use WCRAS with or without annotation filters. Analytical results indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Also, the Cohen’s kappa statistics was used for assessing whether the annotation selected by the high-grade annotation filter is in agreement with the annotations selected by a domain expert. The statistic results indicate that the proposed high-grade annotation filter is valid to some degree. Finally, neither of the proposed quality annotation filtering approaches significantly reduces cognitive load.     

Notes and annotations as information resources in a social networking platform

Taking notes and annotations contributes in the learning process. Many platforms are developed as Computer Supported Collaborative Learning (CSCL) thanks to advancements in new technologies. A common limitation of these platforms is the restricted ability to share/retrieve notes and annotations (Su, Yang, Hwang, & Zhang, 2010a). This is because the annotations in these platforms are disconnected from the information system and they are only accessible in the annotation system. As a result, the annotations could not be indexed as any other information resources (e.g., a document). This means that the annotations are not accessible/visible like other resources. In this paper, we present an original semantic model in which notes and annotations are modeled as information resources. The semantic model is used within the MEMORAe web platform. Then we detail an experiment of collaborative learning made within a university course using the MEMORAe web platform. The feedback of this experiment shows us that students are satisfied with the use of the MEMORAe web platform for helping them to index and retrieve notes and annotations as any information resources.     

From paper-and-pen annotations to artefact-based mobile learning

Making annotations on books and learning materials is part of students' everyday life. Although there are many computer-based annotation systems, many people prefer to print digital documents and make annotations on physical paper. We introduce a classification scheme for computer-based annotation systems and provide an overview of different systems using this scheme. During our investigations, we identified several shortcomings in existing annotation systems: limitation to single document formats, no capturing of relations to reflect lateral reading, loss of context information an annotation was created in and limited sharing capabilities among collaborative users. To overcome these shortcomings, we introduce the concepts of Anchor Points , Artefacts and Spaces . Anchor Points define locations in arbitrary documents. Artefacts capture annotations on multiple Anchor Points in several documents, allowing students to map their current working context that typically contains several open documents and a relationship between them. Spaces are repositories for the learning materials and annotations that are automatically replicated among collaborative mobile users.     

Towards hierarchical context: unfolding visual community potential for interactive video retrieval

Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users’ access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.     

Touching annotations: A visual metaphor for navigation of annotation in digital documents

Direct touch manipulation interactions with technology are now commonplace and significant interest is building around their use in the culture and heritage domain. Such interactions can give people the opportunity to explore materials and artefacts in ways that would otherwise be unavailable. These are often heavily annotated and can be linked to a large array of related digital content, thus enriching the experience for the user. Research has addressed issues of how to present digital documents and their related annotations but at present it is unclear what the optimal interaction approach to navigating these annotations in a touch display context might be.In this paper we investigate the role of two alternative approaches to support the navigation of annotations in digitised documents in the context of a touch interface. Through a control study we demonstrate that, whilst the navigation paradigm displays a significant interaction with the type of annotations task performed, there is no discernible advantage of using a natural visual metaphor for annotation in this context. This suggests that design of digital document annotation navigation tools should account for the context and navigation tasks being considered.     

Interactive multi-user video retrieval systems

In this paper we present two interactive multi-user systems for video search and browsing. The first is composed by web applications which allows multiuser interaction in a distributed environment; such applications are based on the Rich Internet Application paradigm, designed to obtain the levels of responsiveness and interactivity typical of a desktop application. The second system implements a multi-user collaborative application within a single location, exploiting multi-touch devices. Both systems use the same backend, based on a service oriented architecture (SOA) that provides services for automatic and manual annotation, and an ontology-based video search and browsing engine. Ontology-based browsing let users to inspect the content of video collections; user queries are expanded through ontology reasoning. User-centered field trials of the systems, conducted to assess the user experience and satisfaction, have shown that the approach followed to design these interfaces is extremely appreciated by professional archivists and people working on multimedia.     

Collaborative knowledge building with shared video representations

Online video has become established as a fundamental part of the fabric of the web; widely used by people for information sharing, learning and entertainment. We report results from a design study that explored how people interact to create shared multi-path video representations in a social video environment. The participants created multiple versions of a video by providing alternative and interchangeable scenes that formed different paths through the video content. This multi-path video approach was designed to circumvent limitations of traditionally linear video for use as a shared representation in collaborative knowledge building activities. The article describes how people created video resources in collaborative activities in two different settings. We discuss different modes of working that were observed and outline the specific challenges of using the video medium as shared representation. Finally we demonstrate how an analysis of collaborative dimensions of the shared multi-path video representation can be applied to discuss the design space and to raise the discourse about the usefulness of these representations in knowledge building environments.     

Evaluation of a Geospatial Annotation Tool for Unmanned Vehicle Specialist Interface

Map-based interfaces have been developed to support collaborative control of unmanned vehicles (i.e., robots). Annotation on the map (or geospatial annotation) has been proposed as an effective way to support team collaboration; however, there is a lack of research focused on the design of geospatial annotation tools to promote usability and task performance. The utility of location reference in geospatial annotations for communication and information sharing is the focus of this article. Two annotation tool designs were developed. The annotation contents were directly anchored on the map in the first design, whereas in the second design annotations were summarized in a separate panel on the interface. Evaluation participants followed instructions from a simulated team leader and assigned unmanned vehicles to different tasks for two simulated scenarios that include searching for victims and collecting hazardous materials samples. The results demonstrate the potential of using geospatial annotations to enrich communication and support map-based unmanned vehicle control. Participants appreciated the direct location reference feature of the first design and had generally shorter response time, but felt that the second design provided better usability and lower task workload. These results suggest that the user experience depends on the manner of obtaining information from the annotation tools, and the integration of the tool with user's task flow and other interface components, such as the map display. The presented results can be used as a basis for designing geospatial annotation tools for team collaboration that better fit user needs and preferences.     

Remote Collaboration Over Video Data: Towards Real-Time e-Social Science

The design of distributed systems to support collaboration among groups of scientists raises new networking challenges that grid middleware developers are addressing. This field of development work, ‘e-Science’, is increasingly recognising the critical need of understanding the ordinary day-to-day work of doing research to inform design. We have investigated one particular area of collaborative social scientific work – the analysis of video data. Based on interviews and observational studies, we discuss current practices of social scientific work with digital video in three areas: Preparation for collaboration; Control of data and application; and Annotation configurations and techniques. For each, we describe how these requirements feature in our design of a distributed video analysis system as part of the MiMeG project: our security policy and distribution; the design of the control system; and providing freeform annotation over data. Finally, we review our design in light of initial use of the software between project partners; and discuss how we might transform the spatial configuration of the system to support annotation behaviour.     

Product information visualization and augmentation in collaborative design

In this paper, a collaborative system for product information visualization and augmentation is presented. The developed system allows the users, who can be remotely distributed, to view a product model, which is a geometric representation of the product, from different perspectives. They can choose to view design and product history, such as previous modification processes and feature information of the product independently. The product models displayed to the users are immediately updated after any design modifications have been made to the CAD model. Product features being discussed can be highlighted to draw the users’ attention. In addition, modifications can be displayed dynamically for the users to evaluate the design effect. The product history document module in the system provides a user-friendly interface for retrieving design records. After a specific record has been chosen, the related product model is displayed, and it can be aligned with the current product model for the ease of comparison and evaluation. The feature information of the product is displayed using virtual “floating” annotations linked to the related features. A user interface to enter annotations is provided, and the annotations entered by different users can be shared in real time. A cluster-based greedy algorithm is implemented to avoid overlapping annotations in the field of view.