Moving-Object Detection, Association, and Selection in Home Videos

Due to the prevalence of digital video camcorders, home videos have become an important part of life-logs of personal experiences. To enable efficient video parsing, a critical step is to automatically extract objects, events and scene characteristics present in videos. This paper addresses the problem of extracting objects from home videos. Automatic detection of objects is a classical yet difficult vision problem, particularly for videos with complex scenes and unrestricted domains. Compared with edited and surveillant videos, home videos captured in uncontrolled environment are usually coupled with several notable features such as shaking artifacts, irregular motions, and arbitrary settings. These characteristics have actually prohibited the effective parsing of semantic video content using conventional vision analysis. In this paper, we propose a new approach to automatically locate multiple objects in home videos, by taking into account of how and when to initialize objects. Previous approaches mostly consider the problem of how but not when due to the efficiency or real-time requirements. In home-video indexing, online processing is optional. By considering when, some difficult problems can be alleviated, and most importantly, enlightens the possibility of parsing semantic video objects. In our proposed approach, the how part is formulated as an object detection and association problem, while the when part is a saliency measurement to determine the best few locations to start multiple object initialization     

Older people’s production and appropriation of digital videos: an ethnographic study

While most of today’s children, young people, and adults are both consumers and producers of digital content, very little is known about older people as digital content creators. Drawing on a three-year ethnographic study, this paper reports on the digital video production and appropriation of approximately 200 older people (aged 60–85). They generated 320 videos over the course of the study. We show their motivations for engaging in digital video production, discuss their planned video making, and highlight their creativity while editing videos. We show the different meanings they ascribed to digital videos in their social appropriation of these objects, the meaningful strategies they adopted to share videos, and the impact on their perceived wellbeing. Furthermore, we outline the solutions the participants developed to overcome or cope with interaction issues they faced over time. We argue that the results portray older people as active and creative makers of digital videos with current video capturing, editing, and sharing technologies. We contend that this portrayal both encourages us to re-consider how older people should be seen within human–computer interaction and helps to frame future research/design activities that bridge the grey digital divide.     

“I plan to be a 10”: Online literacy and lesbian,gay, bisexual,and transgender students

Lesbian, gay, bisexual, and transgender (LGBT) people use computer mediated communication (CMC) in a variety of ways as they come to understand their sexual identity and begin to identify with larger communities. A Web-based survey gathered 75 responses from Web users who supplied information about their online experiences as LGBT people. They report using online resources to serve many functions in the coming out process: They gather information, try out new ways of self-expression, and an find audience of like-minded people with whom they interact. Online resources were reported as most useful for obtaining basic information, for offering the ability to express oneself on LGBT issues and as a LGBT person, and for connecting with a larger LGBT community. The ways people find, use, and interact with these resources are examples of online literacy, a set of skills that may parallel traditional academic literacies but which may be used in different ways and to different ends.     

Instructor presence in instructional video: Effects on visual attention,recall, and perceived learning

In an effort to enhance instruction and reach more students, educators design engaging online learning experiences, often in the form of online videos. While many instructional videos feature a picture-inpicture view of instructor, it is not clear how instructor presence influences learners' visual attention and what it contributes to learning and affect. Given this knowledge gap, this study explored the impact of instructor presence on learning, visual attention, and perceived learning in mathematics instructional videos of varying content difficulty. Thirty-six participants each viewed two 10-min-long mathematics videos (easy and difficult topics), with instructor either present or absent. Findings suggest that instructor attracted considerable visual attention, particularly when learners viewed the video on an easy topic. Although no significant difference in learning transfer was found for either topic, participants' recall of information from the video was better for easy topic when instructor was present. Finally, instructor presence positively influenced participants' perceived learning and satisfaction for both topics and led to a lower level of self-reported mental effort for difficult topic.     

Documenting and sharing software knowledge using screencasts

Screencasts are used to capture a developer’s screen while they narrate how a piece of software works or how the software can be extended. They have recently become a popular alternative to traditional text-based documentation. This paper describes our investigation into how developers produce and share developer-focused screencasts. In this study, we identified and analyzed a set of development screencasts from YouTube to explore what kinds of software knowledge are shared in video walkthroughs of code and what techniques are used for sharing software knowledge. We also interviewed YouTube screencast producers to understand their motivations for creating screencasts as well as to discover the challenges they face while producing code-focused videos. Finally, we compared YouTube screencasts to videos hosted on the professional RailsCasts website to better understand the differences and practices of this more curated ecosystem with the YouTube platform. Our three-phase study showed that video is a useful medium for communicating program knowledge between developers and that developers build their online persona and reputation by sharing videos through social channels. These findings led to a number of best practices for future screencast creators.     

A Review of “College Libraries and Student Culture: What We Now Know”

This article discusses a student-guided Instagram program for first-year English composition classes at the University of Montevallo during Fall 2013. The activity engaged aspects of critical pedagogy by placing value on students’ prior experiences as creators and users of information and encouraging a classroom environment guided by student input. Since the program was meant for students in their first semester, it was also designed to be fun and social. The Instagram program helped students encounter the unfamiliar academic library environment through the familiar medium of social media, alleviating library anxiety as they produced online content representing their experience.     

Of flags: Online queer identities, writing classrooms, and action horizons

In this article I argue that lesbian, gay, bisexual, transgender (LGBT) and queer pedagogies have been too exclusively focused on identity. The problem with these identity-based pedagogies is that they cannot fully account for actual LGBT identities, due in part to a continued multiplication of these identities online. I use LGBT pride flags to illustrate the ways in which these identities have proliferated, presenting a theoretical limit for current understandings of queer and LGBT pedagogy. As an alternative, I introduce the notion of the action horizon, which forms a pedagogy that encourages students to imagine themselves as actors in the public sphere, shaping policies and confronting complex real-world problems. The various LGBT pride flags then serve as material in three examples—an online asynchronous discussion, writing in a computer classroom, and paper assignments—all of which prompt students to enact the action horizon. Despite the possibilities for real-world engagement offered by the action horizon, success with this pedagogy can be limited in terms of LGBT issues, and so I end with some consideration of the implications and limitations of this kind of teaching.     

基于联盟链的音乐版权保护与交易系统

网络技术的快速发展促进了文字、图片、视频、音乐等数字内容的快速传播,这使数字内容的侵权变得更加容易。针对目前数据化音乐版权管理存在的确权难、耗时长、维权难等问题,结合区块链的去中心化和不可窜改的特点,使用Hyperledger Fabric平台设计并实现了一个基于联盟链的数字音乐版权保护与交易系统。采用Shazam算法提取能证明音乐的原创性的音乐特征指纹,将音乐特征指纹存储在星际文件系统(IPFS)上,IPFS返回的哈希地址存储在区块链中,在弥补区块链扩容难、存储空间昂贵的缺点的同时达到音乐作品的原创性证明,并利用音频水印技术的鲁棒性和隐蔽性,为创作者维权提供可信凭证。仿真实验结果表明,数字音乐版权保护与交易系统中音乐版权登记用时增幅约为0.238 s,每首音乐的特征指纹在IPFS上所占空间约为192 KB,并且区块链系统能够保持较高的吞吐量,具有良好的性能和较高的安全性。     

SHIATSU: tagging and retrieving videos without worries

The dramatic growth of video content over modern media channels (such as the Internet and mobile phone platforms) directs the interest of media broadcasters towards the topics of video retrieval and content browsing. Several video retrieval systems benefit from the use of semantic indexing based on content, since it allows an intuitive categorization of videos. However, indexing is usually performed through manual annotation, thus introducing potential problems such as ambiguity, lack of information, and non-relevance of index terms. In this paper, we present SHIATSU, a complete system for video retrieval which is based on the (semi-)automatic hierarchical semantic annotation of videos exploiting the analysis of visual content; videos can then be searched by means of attached tags and/or visual features. We experimentally evaluate the performance of SHIATSU on two different real video benchmarks, proving its accuracy and efficiency.     

A Survey of MPEG-1 Audio,Video and Semantic Analysis Techniques

Digital audio & video data have become an integral part of multimedia information systems. To reduce storage and bandwidth requirements, they are commonly stored in a compressed format, such as MPEG-1. Increasing amounts of MPEG encoded audio and video documents are available online and in proprietary collections. In order to effectively utilise them, we need tools and techniques to automatically analyse, segment, and classify MPEG video content. Several techniques have been developed both in the audio and visual domain to analyse videos. This paper presents a survey of audio and visual analysis techniques on MPEG-1 encoded media that are useful in supporting a variety of video applications. Although audio and visual feature analyses have been carried out extensively, they become useful to applications only when they convey a semantic meaning of the video content. Therefore, we also present a survey of works that provide semantic analysis on MPEG-1 encoded videos.     

Storyboard-based accurate automatic summary video editing system

The recent popularity of smart mobile devices has led to a significant increase in the needs of multimedia services. Finding new more efficient methods for automatic classification and retrieval of a large number of multimedia files will significantly reduce manpower costs. However, most current video content analysis methods adopt low-level features to analyze video frame by frame, and need to improve high-level semantic analysis on a number of issues. Hence, this study presents a storyboard-based accurate automatic summary video editing system that uses storyboard information, such as character dialogue, narration, caption, background music and shot changes, to enable accurate video content retrieval and automatic render summary videos. The proposed system can be applied to the course video trailer and the commercial video trailer for quick preview video content or suitable viewing configuration for smart mobile devices. Consequently, the audience can quickly understand the whole video story and the video editors can substantially reduce the time taken to publish videos.     

Integrated Mining of Visual Features, Speech Features, and Frequent Patterns for Semantic Video Annotation

To support effective multimedia information retrieval, video annotation has become an important topic in video content analysis. Existing video annotation methods put the focus on either the analysis of low-level features or simple semantic concepts, and they cannot reduce the gap between low-level features and high-level concepts. In this paper, we propose an innovative method for semantic video annotation through integrated mining of visual features, speech features, and frequent semantic patterns existing in the video. The proposed method mainly consists of two main phases: 1) Construction of four kinds of predictive annotation models, namely speech-association, visual-association, visual-sequential, and statistical models from annotated videos. 2) Fusion of these models for annotating un-annotated videos automatically. The main advantage of the proposed method lies in that all visual features, speech features, and semantic patterns are considered simultaneously. Moreover, the utilization of high-level rules can effectively complement the insufficiency of statistics-based methods in dealing with complex and broad keyword identification in video annotation. Through empirical evaluation on NIST TRECVID video datasets, the proposed approach is shown to enhance the performance of annotation substantially in terms of precision, recall, and F-measure.     

Content-based query processing for video databases

This paper presents a query processing strategy for the content-based video query language named CVQL. By CVQL, users can flexibly specify query predicates by the spatial and temporal relationships of the content objects. The query processing strategy evaluates the predicates and returns qualified videos or frames as results. Before the evaluation of the predicates, a preprocessing is performed to avoid unnecessary accessing of videos which are impossible to be the answers. The preprocessing checks the existence of the content objects specified in the predicates to eliminate unqualified videos. For the evaluation of the predicates, an M-index is designed based on the analysis of the behaviors of the content objects. The M-index is employed to avoid frame-by-frame evaluation of the predicates. Experimental results are presented to illustrate the performance of this approach     

Extracting content from instructional videos by statistical modelling and classification

This paper presents a robust approach to extracting content from instructional videos for handwritten recognition, indexing and retrieval, and other e-learning applications. For the instructional videos of chalkboard presentations, retrieving the handwritten content (e.g., characters, drawings, figures) on boards is the first and prerequisite step towards further exploration of instructional video content. However, content extraction in instructional videos is still challenging due to video noise, non-uniformity of the color in board regions, light condition changes in a video session, camera movements, and unavoidable occlusions by instructors. To solve this problem, we first segment video frames into multiple regions and estimate the parameters of the board regions based on statistical analysis of the pixels in dominant regions. Then we accurately separate the board regions from irrelevant regions using a probabilistic classifier. Finally, we combine top-hat morphological processing with a gradient-based adaptive thresholding technique to retrieve content pixels from the board regions. Evaluation of the content extraction results on four full-length instructional videos shows the high performance of the proposed method. The extraction of content text facilitates the research on full exploitation of instructional videos, such as content enhancement, indexing, and retrieval.

Lecture Video Enhancement and Editing by Integrating Posture, Gesture, and Text

This paper describes a novel framework for automatic lecture video editing by gesture, posture, and video text recognition. In content analysis, the trajectory of hand movement is tracked and the intentional gestures are automatically extracted for recognition. In addition, head pose is estimated through overcoming the difficulties due to the complex lighting conditions in classrooms. The aim of recognition is to characterize the flow of lecturing with a series of regional focuses depicted by human postures and gestures. The regions of interest (ROIs) in videos are semantically structured with text recognition and the aid of external documents. By tracing the flow of lecturing, a finite state machine (FSM) which incorporates the gestures, postures, ROIs, general editing rules and constraints, is proposed to edit videos with novel views. The FSM is designed to generate appropriate simulated camera motion and cutting effects that suit the pace of a presenter's gestures and postures. To remedy the undesirable visual effects due to poor lighting conditions, we also propose approaches to automatically enhance the visibility and readability of slides and whiteboard images in the edited videos     

Interaction and Outcomes in Collaborative Storytelling Systems: a Framework,a Field Study,and a Model

In the last decades, advances in interactive information technologies have facilitated collaborative fiction writing, which has become widespread and large-scale. This paper proposes a framework to analyze collaborative storytelling systems, made of a set of parameters divided into six conceptual areas. Four of them relate to the systems and two (process and output) to the results of the collaboration. Through this framework we can study more precisely these different factors of the systems, their interplay, and how they impact the creators’ performance. We also present a controlled extended-duration field study on collaborative storytelling, and we use this framework to comparatively analyze these observations and other relevant experiences in the field of co-creation of shared narrative spaces. As a result, we propose a human-information interaction model for collaborative narrative systems, intended to better support co-creation and address the barriers of this kind of systems turning them into new opportunities for collaboration.     

A two-step approach to see-through bad weather for surveillance video quality enhancement

Adverse weather conditions such as snow, fog or heavy rain greatly reduce the visual quality of outdoor surveillance videos. Video quality enhancement can improve the visual quality of surveillance videos providing clearer images with more details to better meet human perception needs and also improve video analytics performance. Existing work in this area mainly focuses on the quality enhancement for high-resolution videos or still images, but few algorithms are developed for enhancing surveillance videos, which normally have low resolution, high noises and compression artifacts. In addition, for snow or rain conditions, the image quality of near-field view is degraded by the obscuration of apparent snowflakes or raindrops, while the quality of far-field view is degraded by the obscuration of fog-like snowflakes or raindrops. Very few video quality enhancement algorithms have been developed to handle both problems. In this paper, we propose a novel video quality enhancement algorithm for see-through snow, fog or heavy rain. Our algorithm not only improves human visual perception experiences for video surveillance, but also reveal more video contents for better video content analyses. The proposed algorithm handles both near-field and far-field snow/rain effects by proposed a two-step approach: (1) the near-field enhancement algorithm identifies obscuration pixels by snow or rain in the near-field view and removes these pixels as snowflakes or raindrops; different from state-of-the-art methods, our proposed algorithm in this step can detect snowflakes on foreground objects or background, and apply different methods to fill in the removed regions. (2) The far-field enhancement algorithm restores the image’s contrast information not only to reveal more details in the far-field view, but also to enhance the overall image’s quality; in this step, the proposed algorithm adaptively enhances the global and local contrast, which is inspired on the human visual system, and accounts for the perceptual sensitivity to noises, compression artifacts, and the texture of image content. From our extensive testing, the proposed approach significantly improves the visual quality of surveillance videos by removing snow/fog/rain effects.     

面向视频的细粒度多模态实体链接

随着互联网和大数据的飞速发展,数据规模越来越大,种类也越来越多.视频作为其中重要的一种信息方式,随着近期短视频的发展,占比越来越大.如何对这些大规模视频进行理解分析,成为学界关注的热点.实体链接作为一种背景知识补全方式,可以提供丰富的外部知识.视频上的实体链接可以有效地帮助理解视频内容,从而实现对视频内容的分类、检索、推荐等.但是现有的视频链接数据集和方法的粒度过粗,因此提出面向视频的细粒度实体链接,并立足于直播场景,构建了细粒度视频实体链接数据集.此外,依据细粒度视频链接任务的难点,提出利用大模型抽取视频中的实体及其属性,并利用对比学习得到视频和对应实体的更好表示.实验结果表明,该方法能够有效地处理视频上的细粒度实体链接任务.     

Search-based composition, streaming and playback of video archive content

Locating content in existing video archives is both a time and bandwidth consuming process since users might have to download and manually watch large portions of superfluous videos. In this paper, we present two novel prototypes using an Internet based video composition and streaming system with a keyword-based search interface that collects, converts, analyses, indexes, and ranks video content. At user requests, the system can automatically sequence out portions of single videos or aggregate content from multiple videos to produce a single, personalized video stream on-the-fly.     

Techniques used and open challenges to the analysis,indexing and retrieval of digital video

Video in digital format is now commonplace and widespread in both professional use, and in domestic consumer products from camcorders to mobile phones. Video content is growing in volume and while we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. In this paper, we give a brief review of the state of the art of video analysis, indexing and retrieval and we point to research directions which we think are promising and could make searching and browsing of video archives based on video content, as easy as searching and browsing (text) web pages. We conclude the paper with a list of grand challenges for researchers working in the area.