Range query processing on single and multi GPU environments

Metric-space similarity search has proven suitable in a number of application domains such as multimedia retrieval and computational biology to name a few. These applications usually work on very large databases that are often indexed to speed-up on-line searches. To achieve efficient throughput, it is essential to exploit the intrinsic parallelism in the respective search query processing algorithms. Many strategies have been proposed in the literature to parallelize these algorithms either on shared or distributed memory multiprocessor systems. Lately, GPUs have been used to implement brute-force parallel search strategies instead of using index data structures. Indexing poses difficulties when it comes to achieve efficient exploitation of GPU resources. In this paper we propose single and multi GPU metric space techniques that efficiently exploit GPU tailored index data structures for parallel similarity search in large databases. The experimental results show that our proposal outperforms previous index-based sequential and OpenMP search strategies.     

Retrieval of high-dimensional visual data: current state,trends and challenges ahead

Information retrieval algorithms have changed the way we manage and use various data sources, such as images, music or multimedia collections. First, free text information of documents from varying sources became accessible in addition to structured data in databases, initially for exact search and then for more probabilistic models. Novel approaches enable content-based visual search of images using computerized image analysis making visual image content searchable without requiring high quality manual annotations. Other multimedia data followed such as video and music retrieval, sometimes based on techniques such as extracting objects and classifying genre. 3D (surface) objects and solid textures have also been produced in quickly increasing quantities, for example in medical tomographic imaging. For these two types of 3D information sources, systems have become available to characterize the objects or textures and search for similar visual content in large databases. With 3D moving sequences (i.e., 4D), in particular medical imaging, even higher-dimensional data have become available for analysis and retrieval and currently present many multimedia retrieval challenges. This article systematically reviews current techniques in various fields of 3D and 4D visual information retrieval and analyses the currently dominating application areas. The employed techniques are analysed and regrouped to highlight similarities and complementarities among them in order to guide the choice of optimal approaches for new 3D and 4D retrieval problems. Opportunities for future applications conclude the article. 3D or higher-dimensional visual information retrieval is expected to grow quickly in the coming years and in this respect this article can serve as a basis for designing new applications.     

图象内容检索中的相似性度量和分层索引

基于内容的图象检索是图象理解应用于多媒体领域的产物,是下一代智能多媒体数据库的关键技术。本文针对基于内容的静态图象检索,提出了一种度量图象间相似程度的方法,同时还给出了一个通过分层聚类构造二叉树式分层索引数据结构的算法。     

Design issues of multimedia information indexing and retrieval systems

Multimedia information indexing and retrieval systems are required to manage and use ever increasing multimedia information effectively and efficiently. This paper first provides an over-view of general capabilities and architecture of multimedia information indexing and retrieval systems, and then discusses a number of main design issues including data modelling, user interface, feature extraction, indexing, feature similarity measures and quality of service guarantees.     

Multimedia file forensics system exploiting file similarity search

With the fast increase of multimedia contents, efficient forensics investigation methods for multimedia files have been required. In multimedia files, the similarity means that the identical media (audio and video) data are existing among multimedia files. This paper proposes an efficient multimedia file forensics system based on file similarity search of video contents. The proposed system needs two key techniques. First is a media-aware information detection technique. The first critical step for the similarity search is to find the meaningful keyframes or key sequences in the shots through a multimedia file, in order to recognize altered files from the same source file. Second is a video fingerprint-based technique (VFB) for file similarity search. The byte for byte comparison is an inefficient similarity searching method for large files such as multimedia. The VFB technique is an efficient method to extract video features from the large multimedia files. It also provides an independent media-aware identification method for detecting alterations to the source video file (e.g., frame rates, resolutions, and formats, etc.). In this paper, we focus on two key challenges: to generate robust video fingerprints by finding meaningful boundaries of a multimedia file, and to measure video similarity by using fingerprint-based matching. Our evaluation shows that the proposed system is possible to apply to realistic multimedia file forensics tools.

     

Similarity-Based Retrieval of Temporal Specifications and its Application to the Retrieval of Multimedia Documents

In this paper, we describe a similarity-based retrieval framework for temporal information, such as multimedia presentations. We develop techniques that allow users to query and retrieve multimedia documents, based on their temporal content. For this purpose, we describe different temporal data models and a set of similarity metrics applicable for different retrieval tasks. We develop algorithms that efficiently compute these metrics and report on experiment results. We also develop algorithms that efficiently index temporal structures based on these measures and show that the proposed variant of multi dimensional scaling is efficient and provides high quality retrieval of temporal specifications.     

基于内容的信息检索技术研究

介绍了多媒体图象数据查询和检索的系统结构;阐述了图象数据查询和检索的基本过程和相关技术;提出了图象分割、特征提取、相似匹配、快速存取的具体算法,并对它们进行了详细分析。     

Multimedia news exploration and retrieval by integrating keywords,relations and visual features

Multimedia news may be organized by the keywords and categories for exploration and retrieval applications, but it is very difficult to integrate the relation and visual information into the traditional category browsing and keyword-based search framework. This paper propose a new semantic model that can integrate keyword, relation and visual information in a uniform framework. Based on this semantic representation framework, the news exploration and retrieval applications can be organized by not only keywords and categories but also relations and visual properties. We also proposed a set of algorithms to automatically extract the proposed semantic model automatically from large collection of multimedia news reports.     

基于多哈希算法的大规模图像快速检索方法

哈希技术被视为最有潜力的相似性搜索方法,其可以用于大规模多媒体数据搜索场合。为了解决在大规模图像情况下,数据检索效率低下的问题,提出了一种基于分段哈希码的倒排索引树结构,该索引结构将哈希码进行分段处理,对每段哈希码维护一个倒排索引树结构,并结合高效的布隆过滤器构建哈希索引结构。为了进一步提高检索准确性,设计了一种准确的排序融合算法,对多个哈希算法的排序结果分别构建加权无向图,采用PageRank的思想对基于多个哈希算法的排序列表的融合技术进行了详细的说明。实验结果表明,基于分段哈希码的倒排索引树结构能极大地提升数据的检索速度。此外,相比于传统的单个哈希算法排序技术,基于多个哈希算法的排序列表融合技术的检索准确率优势显著。     

Content-based retrieval in digital libraries

With the recent developments in multimedia and telecommunication technologies, content-based information is becoming increasingly important for various areas such as digital libraries, interactive video and multimedia publishing. Multimedia data refers to simple structured data (such as numbers and short strings), large unstructured data (such as text documents, images, audio and video data) and complex structured data (such as maps, graphs, charts and tables). In this article, we briefly address content-based retrieval and the issues of representation, storage and retrieval of multimedia objects in digital libraries. We then very briefly identify some open areas of research     

CSVD: clustering and singular value decomposition for approximate similarity search in high-dimensional spaces

Nearest-neighbor search of high-dimensionality spaces is critical for many applications, such as content-based retrieval from multimedia databases, similarity search of patterns in data mining, and nearest-neighbor classification. Unfortunately, even with the aid of the commonly used indexing schemes, the performance of nearest-neighbor (NN) queries deteriorates rapidly with the number of dimensions. We propose a method, called Clustering with Singular Value Decomposition (CSVD), which supports efficient approximate processing of NN queries, while maintaining good precision-recall characteristics. CSVD groups homogeneous points into clusters and separately reduces the dimensionality of each cluster using SVD. Cluster selection for NN queries relies on a branch-and-bound algorithm and within-cluster searches can be performed with traditional or in-memory indexing methods. Experiments with texture vectors extracted from satellite images show that CSVD achieves significantly higher dimensionality reduction than plain SVD for the same normalized mean squared error (NMSE), which translates into a higher efficiency in processing approximate NN queries.     

A spatio-temporal semantic model for multimedia database systemsand multimedia information systems

As more information sources become available in multimedia systems, the development of abstract semantic models for video, audio, text, and image data is becoming very important. An abstract semantic model has two requirements: it should be rich enough to provide a friendly interface of multimedia presentation synchronization schedules to the users and it should be a good programming data structure for implementation in order to control multimedia playback. An abstract semantic model based on an augmented transition network (ATN) is presented. The inputs for ATNs are modeled by multimedia input strings. Multimedia input strings provide an efficient means for iconic indexing of the temporal/spatial relations of media streams and semantic objects. An ATN and its subnetworks are used to represent the appearing sequence of media streams and semantic objects. The arc label is a substring of a multimedia input string. In this design, a presentation is driven by a multimedia input string. Each subnetwork has its own multimedia input string. Database queries relative to text, image, and video can be answered via substring matching at subnetworks. Multimedia browsing allows users the flexibility to select any part of the presentation they prefer to see. This means that the ATN and its subnetworks can be included in multimedia database systems which are controlled by a database management system (DBMS). User interactions and loops are also provided in an ATN. Therefore, ATNs provide three major capabilities: multimedia presentations, temporal/spatial multimedia database searching, and multimedia browsing     

Synchronization-Oriented Placement and Retrieval Strategies for Delay-Sensitive Media Streams

Multimedia data include text,image,audio and video,etc.Recent developments and advances in the areas of mass storage technology and high speed networks make it feasible for multimedia computing systems to offer services such as multimedia e-mial,teleconferencing and various vivid games.The storage and retrieval of multimedia data are two of the most important technologies in the futre multimedia systems.This paper analyzes the synchronization requirements of the delay-sensitive media streams,classifies the synchronization hierarchically and proposes several effective strategies for the intra-media and inter-media data placement problems.     

Succinct nearest neighbor search

An efficient and universal similarity search solution is a holy grail for multimedia information retrieval. Most similarity indexes work by mapping the original multimedia objects into simpler representations, which are then searched by proximity using a suitable distance function.     

CORE: a content-based retrieval engine for multimedia information systems

Rapid advances in multimedia technology necessitate the development of a generic multimedia information system with a powerful retrieval engine for prototyping multimedia applications. We develop a content-based retrieval engine (CORE) that makes use of novel indexing techniques for multimedia object retrieval. We formalize the concepts related to multimedia information systems such as multimedia objects and content-based retrieval. We bring out the requirements and challenges of a multimedia information system. The architecture of CORE is described in detail along with the associated retrieval mechanisms and indexing techniques. Various modules developed for efficient retrieval are presented with some APIs. The efficacy of CORE is demonstrated in the development of two multimedia systems, a computer-aided facial image inference and retrieval (CAFIIR) system and a system for trademark archival and retrieval (STAR), which have been developed at the Institute of Systems Science (ISS). We expect that CORE will be useful for effective prototyping of other such multimedia applications.Mainly supported by National Science & Technology Board of SingaporePartly working in Real World Computing Partnership, Novel Function Institute of Systems Science Laboratory since April 1994.     

Learning similarity matching in multimedia content-based retrieval

Many multimedia content-based retrieval systems allow query formulation with the user setting the relative importance of features (e.g., color, texture, shape, etc.) to mimic the user's perception of similarity. However, the systems do not modify their similarity matching functions, which are defined during the system development. We present a neural network-based learning algorithm for adapting the similarity matching function toward the user's query preference based on his/her relevance feedback. The relevance feedback is given as ranking errors (misranks) between the retrieved and desired lists of multimedia objects. The algorithm is demonstrated for facial image retrieval using the NIST Mugshot Identification Database with encouraging results     

Supporting ranked Boolean similarity queries in MARS

To address the emerging needs of applications that require access to and retrieval of multimedia objects, we are developing the Multimedia Analysis and Retrieval System (MARS). In this paper, we concentrate on the retrieval subsystem of MARS and its support for content-based queries over image databases. Content-based retrieval techniques have been extensively studied for textual documents in the area of automatic information retrieval. This paper describes how these techniques can be adapted for ranked retrieval over image databases. Specifically, we discuss the ranking and retrieval algorithms developed in MARS based on the Boolean retrieval model and describe the results of our experiments that demonstrate the effectiveness of the developed model for image retrieval     

Multimedia data mining: state of the art and challenges

Advances in multimedia data acquisition and storage technology have led to the growth of very large multimedia databases. Analyzing this huge amount of multimedia data to discover useful knowledge is a challenging problem. This challenge has opened the opportunity for research in Multimedia Data Mining (MDM). Multimedia data mining can be defined as the process of finding interesting patterns from media data such as audio, video, image and text that are not ordinarily accessible by basic queries and associated results. The motivation for doing MDM is to use the discovered patterns to improve decision making. MDM has therefore attracted significant research efforts in developing methods and tools to organize, manage, search and perform domain specific tasks for data from domains such as surveillance, meetings, broadcast news, sports, archives, movies, medical data, as well as personal and online media collections. This paper presents a survey on the problems and solutions in Multimedia Data Mining, approached from the following angles: feature extraction, transformation and representation techniques, data mining techniques, and current multimedia data mining systems in various application domains. We discuss main aspects of feature extraction, transformation and representation techniques. These aspects are: level of feature extraction, feature fusion, features synchronization, feature correlation discovery and accurate representation of multimedia data. Comparison of MDM techniques with state of the art video processing, audio processing and image processing techniques is also provided. Similarly, we compare MDM techniques with the state of the art data mining techniques involving clustering, classification, sequence pattern mining, association rule mining and visualization. We review current multimedia data mining systems in detail, grouping them according to problem formulations and approaches. The review includes supervised and unsupervised discovery of events and actions from one or more continuous sequences. We also do a detailed analysis to understand what has been achieved and what are the remaining gaps where future research efforts could be focussed. We then conclude this survey with a look at open research directions.     

基于整体区域相似的互动式图象检索模型

高效的基于内容的图象检索在许多领域得到了广泛的应用 ,基于内容的图象检索研究领域已经建立了一些系统 ,但在实际使用中 ,这些系统均有如下欠缺 :(1)这些系统均期望以相同的方法来处理各种不同类型的图象检索 ;(2 )这些系统在设计时 ,均缺乏从使用者的需求出发 .实际上 ,由于不同的检索方式是针对不同类型的图象 ,为此 ,提出了一个基于整体区域相似匹配的图象互动式检索系统 ,该系统是一个基于小波变换的特征提取和图象整体区域相似的、语义分类和互动方法的图象检索系统 .与其他检索方法相比较 ,此方法允许自适应查找和互动 ,因此可缩小查找范围 ,以提高检索效率 .实验结果表明 ,该系统比其他一些系统精确和高效