Semantic annotation and search of cultural-heritage collections: The MultimediaN E-Culture demonstrator

In this article we describe a Semantic Web application for semantic annotation and search in large virtual collections of cultural-heritage objects, indexed with multiple vocabularies. During the annotation phase we harvest, enrich and align collection metadata and vocabularies. The semantic-search facilities support keyword-based queries of the graph (currently 20 M triples), resulting in semantically grouped result clusters, all representing potential semantic matches of the original query. We show two sample search scenario’s. The annotation and search software is open source and is already being used by third parties. All software is based on established Web standards, in particular HTML/XML, CSS, RDF/OWL, SPARQL and JavaScript.     

利用本体和主题词表的集成构造RDF模式

为了减少语义异构性带来的信息发现、集成和存取的困难,论述了语义元数据构造,提出了通过集成现存的本体和主题词表构造元数据模式的一种新方法,即元数据模式构造的两步方法:在主题词T和本体O之间的连接关系规范;概念主题词表的自动构造。这个集成基于主题词术语和本体概念之间的蕴含关系规范,并产生具体应用的元数据模式,同时也给出利用结果元数据模式构造RDF模式的过程。     

传统民族服饰数字化采集元数据构建

针对当前传统民族服饰元数据数字化采集没有统一的标准和规范以及采集后的元 数据没有进行文化内涵深入挖掘的问题,以国际通用的 DC 元数据为基础,提炼出传统民族服 饰数字化采集基础元素,深入挖掘传统民族服饰文化内涵进行基础元素扩展,同时与服装、刺 绣以及纺织图片元数据作对比映射,构建出一套基于文化本体、内涵挖掘、数字化色彩描述的 多层可扩展的民族服饰的元数据,以蝙蝠纹蜡染背扇为例,说明传统民族服饰数字化采集元数 据的标注过程,并通过 RDF/XML 的元数据标注和 SparQL 的元数据查询仿真,实现了一套更 具可操作、可扩展的传统民族服饰数字化采集规范。     

Integrating ontologies and thesauri for RDF schema creation and metadata querying

In this paper we present a new approach for building metadata schemas by integrating existing ontologies and structured vocabularies (thesauri). This integration is based on the specification of inclusion relationships between thesaurus terms and ontology concepts and results in application-specific metadata schemas incorporating the structural views of ontologies and the deep classification schemes provided by thesauri. We will also show how the result of this integration can be used for RDF schema creation and metadata querying. In our context, (metadata) queries exploit the inclusion semantics of term relationships, which introduces some recursion. We will present a fairly simple database-oriented solution for querying such metadata which avoids a (recursive) tree traversal and is based on a linear encoding of thesaurus hierarchies. Published online: 22 September 2000     

基于结果模式的Deep Web数据标注方法

全面准确地标注Deep Web查询结果是Deep Web数据集成的关键问题,但现有的Web数据库标注方法还不能较好地解决该问题,为此提出一种基于结果模式的Deep Web数据标注方法。首先通过结果页面解析和抽取结构化数据来完成数据预处理的工作,并在集成结果模式和待标注数据之间建立正确的语义映射,进而确定Deep Web数据的标注信息。通过对4个领域Web数据库进行实验测试,结果表明所提方法能有效地标注Deep Web查询结果数据。     

基于形式概念分析和语义关联规则的目标图像标注

基于目标的图像标注一直是图像处理和计算机视觉领域中一个重要的研究问题.图像目标的多尺度性、多形变性使得图像标注十分困难.目标分割和目标识别是目标图像标注任务中两大关键问题.本文提出一种基于形式概念分析(Formal concept analysis, FCA)和语义关联规则的目标图像标注方法, 针对目标建议算法生成图像块中存在的高度重叠问题, 借鉴形式概念分析中概念格的思想, 按照图像块的共性将其归成几个图像簇挖掘图像类别模式, 利用类别概率分布判决和平坦度判决分别去除目标噪声块和背景噪声块, 最终得到目标语义簇; 针对语义目标判别问题, 首先对有效图像簇进行特征融合形成共性特征描述, 通过分类器进行类别判决, 生成初始目标图像标注, 然后利用图像语义标注词挖掘语义关联规则, 进行图像标注的语义补充, 以避免挖掘类别模式时丢失较小的语义目标.实验表明, 本文提出的图像标注算法既能保证语义标注的准确性, 又能保证语义标注的完整性, 具有较好的图像标注性能.     

中文网页语义标注:由句子到RDF表示

语义网远景的实现需要自动化的语义标注方法,提出了一种在领域本体指导下,针对中文网页的语义标注方法,运用统计学方法与自然语言处理技术,以文档中句子为处理对象,采取识别和组合两个阶段来完成句子向RDF表示的映射,它具有以下特点:以统计方法获得领域相关词汇,构造领域词汇标注列表作为外部领域知识,降低对通用语言本体的依赖;显式的属性类型标注方法识别出句子中表达关系的词汇,标注为属性类型,利于后续关系抽取;构造句子的句法依存关系树(森林),按照依存关系对词汇进行组合,形成RDF陈述.实验结果显示此方法较基于主谓宾语法关系的语义标注方法更为有效.     

Utilising semantic technologies for intelligent indexing and retrieval of digital images

The proliferation of digital media has led to a huge interest in classifying and indexing media objects for generic search and usage. In particular, we are witnessing a colossal growth in digital image repositories that are difficult to navigate using free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. Our search engine relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. We also show how our well-analysed and designed domain ontology contributes to the implicit expansion of user queries as well as the exploitation of lexical databases for explicit semantic-based query expansion.     

Semantic hierarchies for image annotation: A survey

In this survey, we argue that using structured vocabularies is capital to the success of image annotation. We analyze literature on image annotation uses and user needs, and we stress the need for automatic annotation. We briefly expose the difficulties posed to machines for this task and how it relates to controlled vocabularies. We survey contributions in the field showing how structures are introduced. First we present studies that use unstructured vocabulary, focusing on those introducing links between categories or between features. Then we review work using structured vocabularies as an input and analyze how the structure is exploited.     

Effective and Efficient Multi-Facet Web Image Annotation

The vast amount of images available on the Web request for an effective and efficient search service to help users find relevant images.The prevalent way is to provide a keyword interface for users to submit queries.However,the amount of images without any tags or annotations are beyond the reach of manual efforts.To overcome this,automatic image annotation techniques emerge,which are generally a process of selecting a suitable set of tags for a given image without user intervention.However,there are three main challenges with respect to Web-scale image annotation:scalability,noiseresistance and diversity.Scalability has a twofold meaning:first an automatic image annotation system should be scalable with respect to billions of images on the Web;second it should be able to automatically identify several relevant tags among a huge tag set for a given image within seconds or even faster.Noise-resistance means that the system should be robust enough against typos and ambiguous terms used in tags.Diversity represents that image content may include both scenes and objects,which are further described by multiple different image features constituting different facets in annotation.In this paper,we propose a unified framework to tackle the above three challenges for automatic Web image annotation.It mainly involves two components:tag candidate retrieval and multi-facet annotation.In the former content-based indexing and concept-based codebook are leveraged to solve scalability and noise-resistance issues.In the latter the joint feature map has been designed to describe different facets of tags in annotations and the relations between these facets.Tag graph is adopted to represent tags in the entire annotation and the structured learning technique is employed to construct a learning model on top of the tag graph based on the generated joint feature map.Millions of images from Flickr are used in our evaluation.Experimental results show that we have achieved 33% performance improvements compared with those single facet approaches in terms of three metrics:precision,recall and F1 score.     

扩展Java元模型以实现诠释的可视化建模

诠释（annotation）是JavaSE5引入的一种新的编程语言成分，目前得到广泛关注。但从可视化建模来看，已有的Java元模型和UML2元模型并不支持诠释规范，这导致诠释的语义特征在高层抽象中难以体现，也不能实现诠释的可视化、规范化建模。本文扩展已有的Java元模型，以支持诠释的可视化建模。文章提出了诠释的4个重要特征，添加了3个元类以扩展Java元模型，使其能反映诠释特征，并给出2个图符以支持诠释的可视化建模。新的Java元模型符合MOF规范，设计简单，方便实现以支持MDA及相关工具的开发。     

Multimedia tool suite for the visualization of drama heritage metadata

This paper presents a multimedia tool suite for, on the one hand, the annotation of metadata that encode the dramatic qualities of cultural heritage items, and, on the other, the visualization of such metadata for drama analysis and didactics. The tool suite relies upon an ontology of drama to devise an annotation schema for the metadata concerning the dramatic qualities. The two major modules of the tool suite are a web-based platform, that allows for the insertion of the annotation metadata, and a visualization program for the interactive exploration of such metadata, respectively. The tool suite was tested on the cross-media studies of drama analysis and teaching of drama structure through the application to classical examples.     

IMAF: in situ indoor modeling and annotation framework on mobile phones

The widespread use of smart phones with GPS and orientation sensors opens up new possibilities for location-based annotations in outdoor environments. However, a completely different approach is required for indoors. In this study, we introduce IMAF, a novel indoor modeling and annotation framework on a mobile phone. The framework produces a 3D room model in situ with five selections from user without prior knowledge on actual geometry distance or additional apparatus. Using the framework, non-experts can easily capture room dimensions and annotate locations and objects within the room for linking virtual information to the real space represented by an approximated box. For registering 3D room model to the real space, an hybrid method of visual tracking and device sensors obtains accurate orientation tracking result and still achieves interactive frame-rates for real-time applications on a mobile phone. Once the created room model is registered to the real space, user-generated annotations can be attached and viewed in AR and VR modes. Finally, the framework supports object-based space to space registration for viewing and creating annotations from different views other than the view that generated the annotations. The performance of the proposed framework is demonstrated with achieved model accuracy, modeling time, stability of visual tracking and satisfaction of annotation. In the last section, we present two exemplar applications built on IMAF.     

Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition

Managing a large number of digital photos is a challenging task for casual users. Personal photos often don’t have rich metadata, or additional information associated with them. However, available metadata can play a crucial role in managing photos. Labeling the semantic content of photos (i.e., annotating them), can increase the amount of metadata and facilitate efficient management. However, manual annotation is tedious and labor intensive while automatic metadata extraction techniques often generate inaccurate and irrelevant results. This paper describes a semi-automatic annotation strategy that takes advantage of human and computer strengths. The semi-automatic approach enables users to efficiently update automatically obtained metadata interactively and incrementally. Even though automatically identified metadata are compromised with inaccurate recognition errors, the process of correcting inaccurate information can be faster and easier than manually adding new metadata from scratch. In this paper, we introduce two photo clustering algorithms for generating meaningful photo groups: (1) Hierarchical event clustering; and (2) Clothing based person recognition, which assumes that people who wear similar clothing and appear in photos taken in one day are very likely to be the same person. To explore our semi-automatic strategies, we designed and implemented a prototype called SAPHARI (Semi-Automatic PHoto Annotation and Recognition Interface). The prototype provides an annotation framework which focuses on making bulk annotations on automatically identified photo groups. The prototype automatically creates photo clusters based on events, people, and file metadata so that users can easily bulk annotation photos. We performed a series of user studies to investigate the effectiveness and usability of the semi-automatic annotation techniques when applied to personal photo collections. The results show that users were able to make annotations significantly faster with event clustering using SAPHARI. We also found that users clearly preferred the semi-automatic approaches.     

利用Ontology改进的自动化图像标注方法

图像检索系统大多是利用图像的底层特征如颜色、纹理和图像来分析图像,没有考虑图像内容及其对象的内容语义,导致对图像的理解不佳.为使系统能更准确的理解图像中的对象及其深层语义,分析了目前图像标注的优缺点,提出了一种以底层特征为基础,利用本体论建构的知识辅助计算机分析图像中实体对象,判断对象与对象间在现实世界中存在的合理相关性,进而对图像进行标注.实验结果显示加入本体论辅助标注图像大大提高了图像识别的准确性.     

中文文本的事件时空信息标注

基于文本数据源的地理空间信息解析研究侧重于地名实体、空间关系等空间语义角色的标注和抽取,忽略了丰富的时间信息、主题事件信息及其时空一体化信息。该文通过分析中文文本中事件信息描述的语言特点和事件的时空语义特征,基于地名实体和空间关系标注研究成果,制定了中文文本的事件时空信息标注体系和标注模式,并以GATE(General Architecture for Text Engineering)为标注平台,以网页文本为数据源,构建了事件时空信息标注语料库。研究成果为中文文本中地理信息的语义解析提供标准化的训练和测试数据。
     

An Annotation Engine for Supporting Video Database Population

Segmentation, video data modeling, and annotation are indispensable operations necessary for creating and populating a video database. To support such video databases, annotation data can be collected as metadata for the database and subsequently used for indexing and query evaluation. In this paper we describe the design and development of a video annotation engine, called Vane, intended to solve this problem as a domain-independent video annotation application.Using the Vane tool, the annotation of raw video data is achieved through metadata collection. This process, which is performed semi-automatically, produces tailored SGML documents whose purpose is to describe information about the video content. These documents constitute the metadatabase component of the video database. The video data model which has been developed for the metadata, is as open as possible for multiple domain-specific applications. The tool is currently in use to annotate a video archive comprised of educational and news video content.     

The LEMO annotation framework: weaving multimedia annotations with the web

Cultural institutions and museums have realized that annotations contribute valuable metadata for search and retrieval, which in turn can increase the visibility of the digital items they expose via their digital library systems. By exploiting annotations created by others, visitors can discover content they would not have found otherwise, which implies that annotations must be accessible and processable for humans and machines. Currently, however, there exists no widely adopted annotation standard that goes beyond specific media types. Most institutions build their own in-house annotation solution and employ proprietary annotation models, which are not interoperable with those of other systems. As a result, annotation data are usually stored in closed data silos and visible and processable only within the scope of a certain annotation system. As the main contribution of this paper, we present the LEMO Annotation Framework. It (1) provides a uniform annotation model for multimedia contents and various types of annotations, (2) can address fragments of various content-types in a uniform, interoperable manner and (3) pulls annotations out of closed data silos and makes them available as interoperable, dereferencable Web resources. With the LEMO Annotation Framework annotations become part of the Web and can be processed, linked, and referenced by other services. This in turn leads to even higher visibility and increases the potential value of annotations.