Multimodal Video Indexing: A Review of the State-of-the-art

Efficient and effective handling of video documents depends on the availability of indexes. Manual indexing is unfeasible for large video collections. In this paper we survey several methods aiming at automating this time and resource consuming process. Good reviews on single modality based video indexing have appeared in literature. Effective indexing, however, requires a multimodal approach in which either the most appropriate modality is selected or the different modalities are used in collaborative fashion. Therefore, instead of separately treating the different information sources involved, and their specific algorithms, we focus on the similarities and differences between the modalities. To that end we put forward a unifying and multimodal framework, which views a video document from the perspective of its author. This framework forms the guiding principle for identifying index types, for which automatic methods are found in literature. It furthermore forms the basis for categorizing these different methods.     

Query by image and video content: a colored-based stochastic model approach

For efficient image retrieval, the image database should be processed to extract a representing feature vector for each member image in the database. A reliable and robust statistical image indexing technique based on a stochastic model of an image color content has been developed. Based on the developed stochastic model, a compact 12-dimensional feature vector was defined to tag images in the database system. The entries of the defined feature vector are the mean, variance, and skewness of the image color histogram distributions as well as correlation factors between color components of the RGB color space. It was shown using statistical analysis that the feature vector provides sufficient knowledge about the histogram distribution. The reliability and robustness of the proposed technique against common intensity artifacts and noise was validated through several experiments conducted for that purpose. The proposed technique outperforms traditional and other histogram based techniques in terms of feature vector size and properties, as well as performance.     

Wavelet correlogram: A new approach for image indexing and retrieval

In this paper, a new algorithm for content-based image indexing and retrieval is presented. The proposed method is based on a combination of multiresolution image decomposition and color correlation histogram. According to the new algorithm, wavelet coefficients of the image are computed first using a directional wavelet transform such as Gabor wavelets. A quantization step is then applied before computing one-directional autocorrelograms of the wavelet coefficients. Finally, index vectors are constructed using these one-directional wavelet correlograms. The retrieval results obtained by application of our new method on a 1000 image database demonstrated a significant improvement in effectiveness and efficiency compared to the indexing and retrieval methods based on image color correlogram or wavelet transform.     

基于迭代分形的图象压缩和检索方法

图象所具有的海量性和无序性的特点，决定了多媒体应用的构建必须解决图象数据的高效压缩和有效检索两个关键问题，而由于传统的压缩和检索技术的研究是相互分离的，因而限制了多媒体应用系统整体性能的提高，针对此问题，从两者相互结合的观点，对图象压缩和检索方法进行了研究，首先在小波变换域内，基于迭代分形对图象数据进行压缩，然后在图象分形码的基础上，利用迭代函数系统分布特性构建的特征量来支持图象检索，实验结果验证了该方法的可行性和有效性，同时也表明了基于迭代分形的图象检索方法所具有的巨大应用潜力。     

Latent Semantic Kernels

Kernel methods like support vector machines have successfully been used for text categorization. A standard choice of kernel function has been the inner product between the vector-space representation of two documents, in analogy with classical information retrieval (IR) approaches.Latent semantic indexing (LSI) has been successfully used for IR purposes as a technique for capturing semantic relations between terms and inserting them into the similarity measure between two documents. One of its main drawbacks, in IR, is its computational cost.In this paper we describe how the LSI approach can be implemented in a kernel-defined feature space.We provide experimental results demonstrating that the approach can significantly improve performance, and that it does not impair it.     

XSD: A Hierarchical Access Method for Indexing XML Schemata

Search operations and browsing facilities over an XML document database require special support at the physical level. Typical search operations involve path queries. This paper proposes a hierarchical access method to support such operations and to facilitate browsing. It advocates the idea of searching large XML collections by administering efficiently XML schemata. The proposed approach may be used for indexing XML documents according to their structural proximity. This is obtained by organizing the schemata of a large XML document collection in a hierarchical way by merging structurally close schemata. The proposed structure, which is called XML Schema Directory (XSD), is a balanced tree and it may serve two purposes: (1) to accelerate XML query processing and (2) to facilitate browsing. Received 15 March 2001 / Revised 12 April 2001 / Accepted in revised form 11 May 2001     

Fast Extraction of Semantic Features from a Latent Semantic Indexed Text Corpus

This paper proposes a projection-based symmetrical factorisation method for extracting semantic features from collections of text documents stored in a Latent Semantic space. Preliminary experimental results demonstrate this yields a comparable representation to that provided by a novel probabilistic approach which reconsiders the entire indexing problem of text documents and works directly in the original high dimensional vector-space representation of text. The employed projection index is derived here from the a priori constraints on the problem. The principal advantage of this approach is computational efficiency and is obtained by the exploitation of the Latent Semantic Indexing as a preprocessing stage. Simulation results on subsets of the 20-Newsgroups text corpus in various settings are provided. This revised version was published online in August 2006 with corrections to the Cover Date.     

Reflectance based object recognition

Neighboring points on a smoothly curved surface have similar surface normals and illumination conditions. Therefore, their brightness values can be used to compute the ratio of their reflectance coefficients. Based on this observation, we develop an algorithm that estimates a reflectance ratio for each region in an image with respect to its background. The algorithm is efficient as it computes ratios for all image regions in just two raster scans. The region reflectance ratio represents a physical property that is invariant to illumination and imaging parameters. Several experiments are conducted to demonstrate the accuracy and robustness of ratio invariant.The ratio invariant is used to recognize objects from a single brightness image of a scene. Object models are automatically acquired and represented using a hash table. Recognition and pose estimation algorithms are presented that use ratio estimates of scene regions as well as their geometric properties to index the hash table. The result is a hypothesis for the existence of an object in the image. This hypothesis is verified using the ratios and locations of other regions in the scene. This approach to recognition is effective for objects with printed characters and pictures. Recognition experiments are conducted on images with illumination variations, occlusions, and shadows. The paper is concluded with a discussion on the simultaneous use of reflectance and geometry for visual perception.