基于MPEG-7的视频语义检索系统

设计和实现一个支持语义的分布式视频检索系统:"语寻"。该系统利用一个改进的视频语义处理工具(该工具基于IBM VideoAnnEx标注工具,并增加镜头语义图标注和自然语言处理的功能)对视频进行语义分析和标注,生成包含语义信息的MPEG-7描述文件,然后对视频的MPEG-7描述文件建立分布式索引,并同时分布式存储视频文件;系统提供丰富的Web查询接口,包括关键字语义扩展查询,语义图查询以及自然语句查询,当用户提交语义查询意图后,便能够迅速地检索到感兴趣的视频和片段,并且可以浏览点播;整个系统采用分布式架构,具备良好的可扩展性,并能够支持海量视频信息的索引和检索。     

Mapping among knowledge bases and data repositories: Precise definition of its syntax and semantics

Nowadays there is an explosion of data repositories available in the Global Information System. In this context, users meet with the problems related to the heterogeneity and distribution of the data repositories. Access to the data may be facilitated by providing semantic views over the repositories that hide all the technical and organizative details associated with them. In this paper we present the definition of the mapping relation that permits one to relate data elements of the semantic views and those of the underlying data repositories. We define the syntax and the semantics of the mapping. From the syntactic point of view, mapping expressions are tuples whose main components are relational algebra expressions. From the semantic point of view, the notions of well formed mapping relation and the extension defined by that mapping are introduced. Moreover, we demonstrate the completeness property of the mapping relation in the sense that it allows for answers to all queries formulated over the semantic view using the data stored in the data repositories.     

A spatio-temporal semantic model for multimedia database systemsand multimedia information systems

As more information sources become available in multimedia systems, the development of abstract semantic models for video, audio, text, and image data is becoming very important. An abstract semantic model has two requirements: it should be rich enough to provide a friendly interface of multimedia presentation synchronization schedules to the users and it should be a good programming data structure for implementation in order to control multimedia playback. An abstract semantic model based on an augmented transition network (ATN) is presented. The inputs for ATNs are modeled by multimedia input strings. Multimedia input strings provide an efficient means for iconic indexing of the temporal/spatial relations of media streams and semantic objects. An ATN and its subnetworks are used to represent the appearing sequence of media streams and semantic objects. The arc label is a substring of a multimedia input string. In this design, a presentation is driven by a multimedia input string. Each subnetwork has its own multimedia input string. Database queries relative to text, image, and video can be answered via substring matching at subnetworks. Multimedia browsing allows users the flexibility to select any part of the presentation they prefer to see. This means that the ATN and its subnetworks can be included in multimedia database systems which are controlled by a database management system (DBMS). User interactions and loops are also provided in an ATN. Therefore, ATNs provide three major capabilities: multimedia presentations, temporal/spatial multimedia database searching, and multimedia browsing     

Real-time data analytics and event detection for IoT-enabled communication systems

Enterprise Communication Systems are designed in such a way to maximise the efficiency of communication and collaboration within the enterprise. With users becoming mobile, the Internet of Things (IoT) can play a crucial role in this process, but is far from being seamlessly integrated into modern online communications. In this paper, we present a semantic infrastructure for gathering, integrating and reasoning upon heterogeneous, distributed and continuously changing data streams by means of semantic technologies and rule-based inference. Our solution exploits semantics to go beyond today’s ad-hoc integration and processing of heterogeneous data sources for static and streaming data. It provides flexible and efficient processing techniques that can transform low-level data into high-level abstractions and actionable knowledge, bridging the gap between IoT and online Enterprise Communication Systems. We document the technologies used for acquisition and semantic enrichment of sensor data, continuous semantic query processing for integration and filtering, as well as stream reasoning for decision support. Our main contributions are the following, (i) we define and deploy a semantic processing pipeline for IoT-enabled Communication Systems, which builds upon existing systems for semantic data acquisition, continuous query processing and stream reasoning, detailing the implementation of each component of our framework; (ii) we present a rich semantic information model for representing and linking IoT data, social data and personal data in the Enterprise Communication scenario, by reusing and extending existing standard semantic models; (iii) we define and develop an expressive stream reasoning component as part of our framework, based on continuous query processing and non-monotonic reasoning for semantic streams, (iv) we conduct experiments to comparatively evaluate the performance of our data acquisition and semantic annotation layer based on OpenIoT, and the performance of our expressive reasoning layer in the scenario of Enterprise Communication.     

MQSS: multimodal query suggestion and searching for video search

In this paper, we propose a multimodal query suggestion method for video search which can leverage multimodal processing to improve the quality of search results. When users type general or ambiguous textual queries, our system MQSS provides keyword suggestions and representative image examples in an easy-to-use dropdown manner which can help users specify their search intent more precisely and effortlessly. It is a powerful complement to initial queries. After the queries are formulated as multimodal query (i.e., text, image), the new queries are input to individual search models, such as text-based, concept-based and visual example-based search model. Then we apply multimodal fusion method to aggregate the above-mentioned several search results. The effectiveness of MQSS is demonstrated by evaluations over a web video data set.     

基于本体和XML的数据交换研究

介绍了基于本体和XML的数据交换系统的构造方法，通过用本体描述特定领域中的概念及其之间的关系，并用本体信息标记数据模型，使数据模型不仅具有结构信息，还具有了语义信息，为实现语义完整的数据转换提供了支持并方便了数据转换中的查询重组处理。     

Selection and dissemination of digital video via the Virtual Video Browser

In this paper we describe the Virtual Video Browser (VVB) software application designed to allow the interactive browsing and content-based query of a video database and to facilitate the subsequent playout of selected titles. The VVB is a manifestation of our mechanisms for the location, identification, and delivery of digital audio and video in a distributed system which can be extended to several application domains including multimedia-based home entertainment, catalog shopping, and distance learning.The VVB employs a two phase retrieval process to serve its users. In the query phase, user queries are sent to a metadata server for processing. In the subsequent playout phase, a connection is established between the client workstation and a video server for the delivery of video data. The VVB incorporates a simple query interface that lets users specify their preferences to the system and retrieve the appropriate video. The application is designed to work in a distributed environment where video sequences are stored in different databases interconnected via a network. It has been shown to be a viable target application useful for investigating research problems related to building interactive multimedia systems.Portions of this work were presented at the Intl. Workshop on Services in Distributed and Networked Environments (SDNE), Prague, Czech Republic, June 1994, and at the 1st ACM Intl. Conf. on Multimedia, Anaheim CA, August 1993.     

Spatio-temporal querying in video databases

A video data model that supports spatio-temporal querying in videos is presented. The data model is focused on the semantic content of video streams. Objects, events, activities, and spatial properties of objects are main interests of the model. The data model enables the user to query fuzzy spatio-temporal relationships between video objects and also trajectories of moving objects. A prototype of the proposed model has been implemented.     

Design,realization and user evaluation of the SmartVortex Visual Query System for accessing data streams in industrial engineering applications

Objective: In this paper we present a Visual Query System, which allows users to graphically build queries over data streams and traditional relational data. The Visual Query System runs on top of a DSMS (Data Stream Management System).Methods: The system has been designed and implemented following the user-centered design approach. Two different releases of the system have been incrementally and iteratively designed and evaluated. The first release has been evaluated using heuristic evaluation. The second release, whose design was a refinement based on the results of the foregoing heuristic evaluation, was evaluated by several users. Moreover, a comparative evaluation involving users has been conducted on the second release.Results: The paper presents the visual query system, and the results ofthe various evaluations.Conclusion: Visual Query Systems for data streams are an effective tool for industrial engineers in querying and analyzing data streams from equipments.Practice and implications: The proposed visual query system has been employed in real industrial scenarios in order to visually query and analyze data streams produced by industrial machines.     

Merging Results for Distributed Content Based Image Retrieval

Searching information through the Internet often requires users to separately contact several digital libraries, use each library interface to author the query, analyze retrieval results and merge them with results returned by other libraries. Such a solution could be simplified by using a centralized server that acts as a gateway between the user and several distributed repositories: The centralized server receives the user query, forwards the user query to federated repositories—possibly translating the query in the specific format required by each repository—and fuses retrieved documents for presentation to the user. To accomplish these tasks efficiently, the centralized server should perform some major operations such as: resource selection, query transformation and data fusion. In this paper we report on some aspects of MIND, a system for managing distributed, heterogeneous multimedia libraries (MIND, 2001, http://www.mind-project.org). In particular, this paper focusses on the issue of fusing results returned by different image repositories. The proposed approach is based on normalization of matching scores assigned to retrieved images by individual libraries. Experimental results on a prototype system show the potential of the proposed approach with respect to traditional solutions.     

Rule-based spatiotemporal query processing for video databases

In our earlier work, we proposed an architecture for a Web-based video database management system (VDBMS) providing an integrated support for spatiotemporal and semantic queries. In this paper, we focus on the task of spatiotemporal query processing and also propose an SQL-like video query language that has the capability to handle a broad range of spatiotemporal queries. The language is rule-based in that it allows users to express spatial conditions in terms of Prolog-type predicates. Spatiotemporal query processing is carried out in three main stages: query recognition, query decomposition, and query execution.Received: 11 October 2001, Accepted: 3 October 2003, Published online: 12 December 2003Edited by: A. Buchmann Correspondence to: Özgür UlusoyThis work is supported by the Scientific and Research Council of Turkey (TÜBITAK) under Project Code 199E025. This work was done while the first author was at Bilkent University.     

教学视频的文本语义镜头分割和标注

为了对教学视频这一专门类别视频进行自动标注,本文首先提取视频中的字幕信息,通过文本预处理后,使用视频中的字幕文本信息内容结合潜在狄利克雷分布（Latent Dirichlet allocation,LDA）主题模型方法获得视频镜头在主题上的概率分布,通过计算主题概率分布差异,进行语义层面镜头分割。然后以镜头为样本,使用安全的半监督支持向量机（Safe semi-supervised support vector machine,S4VM）方法,通过少量的标注镜头样本,完成对未标注镜头的自动标注。实验结果表明,本文方法利用字幕文本信息和LDA模型,有效完成了视频的语义镜头分割,不仅可以对镜头完成标注,而且可以对整个视频进行关键词标注。     

Discovering Document Semantics QBYS: A System for Querying the WWW by Semantics

This paper describes our research into a query-by-semantics approach to searching the World Wide Web. This research extends existing work, which had focused on a query-by-structure approach for the Web. We present a system that allows users to request documents containing not only specific content information, but also to specify that documents be of a certain type. The system captures and utilizes structure information as well as content during a distributed query of the Web. The system also allows the user the option of creating their own document types by providing the system with example documents. In addition, although the system still gives users the option of dynamically querying the web, the incorporation of a document database has improved the response time involved in the search process. Based on extensive testing and validation presented herein, it is clear that a system that incorporates structure and document semantic information into the query process can significantly improve search results over the standard keyword search.     

Optimizing top-k selection queries over multimedia repositories

Repositories of multimedia objects having multiple types of attributes (e.g., image, text) are becoming increasingly common. A query on these attributes will typically, request not just a set of objects, as in the traditional relational query model (filtering), but also a grade of match associated with each object, which indicates how well the object matches the selection condition (ranking). Furthermore, unlike in the relational model, users may just want the k top-ranked objects for their selection queries for a relatively small k. In addition to the differences in the query model, another peculiarity of multimedia repositories is that they may allow access to the attributes of each object only through indexes. We investigate how to optimize the processing of top-k selection queries over multimedia repositories. The access characteristics of the repositories and the above query model lead to novel issues in query optimization. In particular, the choice of the indexes used to search the repository strongly influences the cost of processing the filtering condition. We define an execution space that is search-minimal, i.e., the set of indexes searched is minimal. Although the general problem of picking an optimal plan in the search-minimal execution space is NP-hard, we present an efficient algorithm that solves the problem optimally with respect to our cost model and execution space when the predicates in the query are independent. We also show that the problem of optimizing top-k selection queries can be viewed, in many cases, as that of evaluating more traditional selection conditions. Thus, both problems can be viewed together as an extended filtering problem to which techniques of query processing and optimization may be adapted.     

KDA: a knowledge-based database assistant with a query guidingfacility

A knowledge-based database assistant (KDA) which integrates a natural language query system with a skeleton-based query guiding facility is provided. When a user works with the KDA natural language query system, the query guiding facility can supply several kinds of skeletons to guide users in performing database retrieval tasks. A semantic network model, S-Net, is introduced to represent the knowledge for natural language query processing and skeleton generation. Methods for implementing the system are discussed