Measuring Concept Similarities in Multimedia Ontologies: Analysis and Evaluations

The recent development of large-scale multimedia concept ontologies has provided a new momentum for research in the semantic analysis of multimedia repositories. Different methods for generic concept detection have been extensively studied, but the question of how to exploit the structure of a multimedia ontology and existing inter-concept relations has not received similar attention. In this paper, we present a clustering-based method for modeling semantic concepts on low-level feature spaces and study the evaluation of the quality of such models with entropy-based methods. We cover a variety of methods for assessing the similarity of different concepts in a multimedia ontology. We study three ontologies and apply the proposed techniques in experiments involving the visual and semantic similarities, manual annotation of video, and concept detection. The results show that modeling inter-concept relations can provide a promising resource for many different application areas in semantic multimedia processing.     

一种基于基本语义单元合成Petri网的足球视频查询描述模型

足球是最具世界性的体育运动之一，球迷遍布五大洲，因此在体育视频节目中足球备受广大观众青睐．在分析了足球视频特点的基础上，提出了一种基于基本语义单元合成Petri网的足球视频查询描述模型．该模型首先定义了一种类似文本字词集合的足球视频基本语义单元集合，在此基础上采用基本语义单元合成Petri网模型建立了一种足球查询语义的描述模型，并分别构建了进球、进攻、角球、犯规、换人等足球语义．初步的实验结果验证了该模型的有效性，并能推广至球类视频和其他体育视频．     

Information retrieval and hypertext: competing technologies or complementary access methods

Abstract. Information retrieval typically involves accessing textual information from a database in response to a user's vague information need. Hypertext or hypermedia, on the other hand, involves a user browsing through a database of textual or multimedia information in response to a variety of types of information need. Thus information retrieval can be said to have a searching metaphor while hypertext has a browsing analogy. Initially, these two technologies for information access appear to be very different, almost competitive in nature. In this paper information retrieval systems are briefly reviewed and hypertext systems are also examined. These two techniques for accessing information have been integrated into a prototype system which is described. The system dynamically generates guided tours in response to a user's query and the tour guides the user through the hypertext. Some experiments reporting on the effectiveness of this as an information access strategy are given.     

Everyday concept detection in visual lifelogs: validation,relationships and trends

The Microsoft SenseCam is a small lightweight wearable camera used to passively capture photos and other sensor readings from a user’s day-to-day activities. It captures on average 3,000 images in a typical day, equating to almost 1 million images per year. It can be used to aid memory by creating a personal multimedia lifelog, or visual recording of the wearer’s life. However the sheer volume of image data captured within a visual lifelog creates a number of challenges, particularly for locating relevant content. Within this work, we explore the applicability of semantic concept detection, a method often used within video retrieval, on the domain of visual lifelogs. Our concept detector models the correspondence between low-level visual features and high-level semantic concepts (such as indoors, outdoors, people, buildings, etc.) using supervised machine learning. By doing so it determines the probability of a concept’s presence. We apply detection of 27 everyday semantic concepts on a lifelog collection composed of 257,518 SenseCam images from 5 users. The results were evaluated on a subset of 95,907 images, to determine the accuracy for detection of each semantic concept. We conducted further analysis on the temporal consistency, co-occurance and relationships within the detected concepts to more extensively investigate the robustness of the detectors within this domain.     

基于感知概念和有限状态机的体育视频语义内容分析模型

视频内容自动分析领域中,关键的挑战在干如何识别重要对象和如何建模对象之间的时空关系.本文基于感知概念(Perception Concepts,简称PCs)和有限状态机(Finite State Machines.简称FSMs)提出一种语义内容分析模型自动描述和探测体育视频中有意义的语义内容.根据体育视频中可识别的特征元素,定义PCs来表示视频中重要的语义模式;设计PC-FSM模型来描述PCs间的时空关系;采用一个图匹配方法自动探测视频中的高层语义.本文提出的方法使用户能够根据其自身的兴趣和知识设计体育视频的查询描述,并将语义内容探测问题转换为图匹配问题.实验结果验证了本文提出的方法的有效性.     

Detecting students-at-risk in computer programming classes with learning analytics from students’ digital footprints

User Modeling and User-Adapted Interaction - Different sources of data about students, ranging from static demographics to dynamic behavior logs, can be harnessed from a variety sources at Higher...     

Passively recognising human activities through lifelogging

Lifelogging is the process of automatically recording aspects of one’s life in digital form. This includes visual lifelogging using wearable cameras such as the SenseCam and in recent years many interesting applications for this have emerged and are being actively researched. One of the most interesting of these, and possibly the most far-reaching, is using visual lifelogs as a memory prosthesis but there are also applications in job-specific activity recording, general lifestyle analysis and market analysis.In this work we describe a technique which allowed us to develop automatic classifiers for visual lifelogs to infer different lifestyle traits or characteristics. Their accuracy was validated on a set of 95 k manually annotated images and through one-on-one interviews with those who gathered the images. These automatic classifiers were then applied to a collection of over 3 million lifelog images collected by 33 individuals sporadically over a period of 3.5 years. From this collection we present a number of anecdotal observations to demonstrate the future potential of lifelogging to capture human behaviour. These anecdotes include: the eating habits of office workers; to the amount of time researchers spend outdoors through the year; to the observation that retired people in our study appear to spend quite a bit of time indoors eating with friends. We believe this work demonstrates the potential of lifelogging techniques to assist behavioural scientists in future.