Enhanced Gabor wavelet correlogram feature for image indexing and retrieval

In this paper, a new feature scheme called enhanced Gabor wavelet correlogram (EGWC) is proposed for image indexing and retrieval. EGWC uses Gabor wavelets to decompose the image into different scales and orientations. The Gabor wavelet coefficients are then quantized using optimized quantization thresholds. In the next step, the autocorrelogram of the quantized wavelet coefficients is computed in each wavelet scale and orientation. Finally, the EGWC index vector simply consists of the autocorrelogram coefficients. Due to non-orthogonality of Gabor decomposition, the resulting wavelet coefficients suffer from redundancy, which increases the computational cost and reduces the effectiveness of EGWC. Here, we present a solution to handle the redundancy problem using non-maximum suppression and adjustment of autocorrelogram distance parameters as a function of the wavelet scale. The retrieval results obtained by applying EGWC to index two image databases with 5,000 natural images and 1,792 texture images demonstrated its better performance in terms of retrieval rates with respect to the state-of-the-art content-based and multidirectional texture indexing algorithms.     

明显区域块和空间分布特征的图像检索

基于小波变换理论提出了一种明显区域块检测方法,改进了环型分割算法,使对视觉有意义的区域特征提取更加快捷、方便。该算法不仅考虑到区域内的图像特征,而且还考虑到明显区域块的空间分布信息,把环型区域的颜色矩和在明显区域块附近的Gabor特点,作为索引图像的特征向量。使用Corel图像库测试了提出的方法。实验表明,该方法切实可行。     

Pattern classification models for classifying and indexing audio signals

In the age of digital information, audio data has become an important part in many modern computer applications. Audio classification and indexing has been becoming a focus in the research of audio processing and pattern recognition. In this paper, we propose effective algorithms to automatically classify audio clips into one of six classes: music, news, sports, advertisement, cartoon and movie. For these categories a number of acoustic features that include linear predictive coefficients, linear predictive cepstral coefficients and mel-frequency cepstral coefficients are extracted to characterize the audio content. The autoassociative neural network model (AANN) is used to capture the distribution of the acoustic feature vectors. Then the proposed method uses a Gaussian mixture model (GMM)-based classifier where the feature vectors from each class were used to train the GMM models for those classes. During testing, the likelihood of a test sample belonging to each model is computed and the sample is assigned to the class whose model produces the highest likelihood. Audio clip extraction, feature extraction, creation of index, and retrieval of the query clip are the major issues in automatic audio indexing and retrieval. A method for indexing the classified audio using LPCC features and k-means clustering algorithm is proposed.     

Bayesian belief network based broadcast sports video indexing

This paper presents a probabilistic Bayesian belief network (BBN) method for automatic indexing of excitement clips of sports video sequences. The excitement clips from sports video sequences are extracted using audio features. The excitement clips are comprised of multiple subclips corresponding to the events such as replay, field-view, close-ups of players, close-ups of referees/umpires, spectators, players’ gathering. The events are detected and classified using a hierarchical classification scheme. The BBN based on observed events is used to assign semantic concept-labels to the excitement clips, such as goals, saves, and card in soccer video, wicket and hit in cricket video sequences. The BBN based indexing results are compared with our previously proposed event-association based approach and found BBN is better than the event-association based approach. The proposed scheme provides a generalizable method for linking low-level video features with high-level semantic concepts. The generic nature of the proposed approach in the sports domain is validated by demonstrating successful indexing of soccer and cricket video excitement clips. The proposed scheme offers a general approach to the automatic tagging of large scale multimedia content with rich semantics. The collection of labeled excitement clips provide a video summary for highlight browsing, video skimming, indexing and retrieval.     

Image retrieval based on shape similarity by edge orientation autocorrelogram

This paper introduces a new feature vector for shape-based image indexing and retrieval. This feature classifies image edges based on two factors: their orientations and correlation between neighboring edges. Hence it includes information of continuous edges and lines of images and describes major shape properties of images. This scheme is effective and robustly tolerates translation, scaling, color, illumination, and viewing position variations. Experimental results show superiority of proposed scheme over several other indexing methods. Averages of precision and recall rates of this new indexing scheme for retrieval as compared with traditional color histogram are 1.99 and 1.59 times, respectively. These ratios are 1.26 and 1.04 compared to edge direction histogram.     

A content-based goods image recommendation system

The information of e-commerce images varies and different users may focus on different contents of the same image for different purpose. So the research on recommendation by computers is becoming more and more important. But retrieval based only on keywords obviously falls short for massive numbers of resource images. In this paper, we focus on a recommendation system of goods images based on image content. Goods images have a relatively homogenous background and have a wide range of applications. The recommendation consists of three stages. First, the image is pre-processed by removing the background. Second, a weighted representation model is proposed to represent the image. The separated features are extracted and normalized, and then the weights of each feature are computed based on the samples browsed by the users. Third, a feature indexing scheme is put forward based on the proposed representation. A binary-tree is used for the indexing, and a binary-tree updating algorithm is also given. Finally, the recommended images are given by a features combination searching scheme. Experimental results on a real goods image database show that our algorithm can achieve high accuracy in recommending similar goods images with high speed.     

HIRBIR: A hierarchical approach to region-based image retrieval

This paper proposes a hierarchical approach to region-based image retrieval (HIRBIR) based on wavelet transform whose decomposition property is similar to human visual processing. First, automated image segmentation is performed fast in the low-low (LL) frequency subband of the wavelet domain that shows the desirable low image resolution. In the proposed system, boundaries between segmented regions are deleted to improve the robustness of region-based image retrieval against segmentation-related uncertainty. Second, a region feature vector is hierarchically represented by information in all wavelet subbands, and each feature component of a feature vector is a unified color–texture feature. Such a feature vector captures well the distinctive features (e.g., semantic texture) inside one region. Finally, employing a hierarchical feature vector, the weighted distance function for region matching is tuned meaningfully and easily, and a progressive stepwise indexing mechanism with relevance feedback is performed naturally and effectively in our system. Through experimental results and comparison with other methods, the proposed HIRBIR shows a good tradeoff between retrieval effectiveness and efficiency as well as easy implementation for region-based image retrieval.     

基于小波变换和支持向量机的音频分类

音频特征提取是音频分类的基础,而音频分类又是内容的音频检索的关键。综合分析了语音和音乐的区别性特征,提出一种基于小波变换和支持向量机的音频特征提取和分类的方法,用于纯语音、音乐、带背景音乐的语音以及环境音的分类,并且评估了新特征集合在SVM分类器上的分类效果。实验结果表明,提出的音频特征有效、合理,分类性能较好。     

Embedding neural networks for semantic association in content based image retrieval

Content based image retrieval (CBIR) systems provide potential solution of retrieving semantically similar images from large image repositories against any query image. The research community are competing for more effective ways of content based image retrieval, so they can be used in serving time critical applications in scientific and industrial domains. In this paper a Neural Network based architecture for content based image retrieval is presented. To enhance the capabilities of proposed work, an efficient feature extraction method is presented which is based on the concept of in-depth texture analysis. For this wavelet packets and Eigen values of Gabor filters are used for image representation purposes. To ensure semantically correct image retrieval, a partial supervised learning scheme is introduced which is based on K-nearest neighbors of a query image, and ensures the retrieval of images in a robust way. To elaborate the effectiveness of the presented work, the proposed method is compared with several existing CBIR systems, and it is proved that the proposed method has performed better then all of the comparative systems.

     

局部化数字水印算法

数字水印是一种嵌入到图象,视频或音频数据中的不可见标志,可以用于多媒体数字的版权保护,认证和标注等,为了提高在频率域嵌入水印抵抗裁剪攻击的能力,提出一种局部化的图象数字水印算法,该算法利用图象中相对稳定的特征点标示水印嵌入的位置,并在与每个特征点对应的局部区域中独立地嵌入水印,这样,当只有部分图象时,仍能通过这些特下点来定位并提取水印,此算法中,水印的嵌入是在局部图象的小波域中进行的,并采用对小波系数进行特殊量化的方法来隐藏水印比特,而水印的提取不需要原始图象参与,实验结果证明,算法对裁剪有很强的抵抗能力,同时对压缩,滤波,噪声,StirMark攻击等也有较好的鲁棒性。     

Wavelet correlogram: A new approach for image indexing and retrieval

In this paper, a new algorithm for content-based image indexing and retrieval is presented. The proposed method is based on a combination of multiresolution image decomposition and color correlation histogram. According to the new algorithm, wavelet coefficients of the image are computed first using a directional wavelet transform such as Gabor wavelets. A quantization step is then applied before computing one-directional autocorrelograms of the wavelet coefficients. Finally, index vectors are constructed using these one-directional wavelet correlograms. The retrieval results obtained by application of our new method on a 1000 image database demonstrated a significant improvement in effectiveness and efficiency compared to the indexing and retrieval methods based on image color correlogram or wavelet transform.     

Compressed domain content based retrieval using H.264 DC-pictures

A fast and simple method for content based retrieval using the DC-pictures of H.264 coded video without full decompression is presented. Compressed domain retrieval is very desirable for content analysis and retrieval of compressed image and video. Even though, DC-pictures are among the most widely used compressed domain indexing and retrieval methods in pre H.264 coded videos, they are not generally used in the H.264 coded video. This is due to two main facts, first, the I-frame in the H.264 standard are spatially predicatively coded and second, the H.264 standard employs Integer Discrete Cosine Transform. In this paper we have applied color histogram indexing method on the DC-pictures derived from H.264 coded I-frames. Since the method is based on independent I-frame coded pictures, it can be used either for video analysis of H.264 coded videos, or image retrieval of the I-frame based coded images such as advanced image coding. The retrieval performance of the proposed algorithm is compared with that the fully decoded images. Simulation results indicate that the performance of the proposed method is very close to the fully decompressed image systems. Moreover the proposed method has much lower computational load.     

Spatial Color Indexing Using Rotation,Translation, and Scale Invariant Anglograms

As color plays an essential role in image composition, many color indexing techniques have been studied for content-based image retrieval. This paper examines the use of a computational geometry-based spatial color indexing methodology for effective and efficient image retrieval. In this scheme, an image is evenly divided into a number of M * N non-overlapping blocks, and each individual block is abstracted as a unique feature point labeled with its spatial location and dominant colors. For each set of feature points labeled with the identical color, we construct a Delaunay triangulation and then compute the feature point histogram by discretizing and counting the angles produced by this triangulation. The concatenation of all these feature point histograms serves as the image index, the so-called color anglogram. An important contribution of this work is to encode the spatial color information using geometric triangulation, which is rotation, translation, and scale invariant. We have compared the proposed approach with two of the best performing of recent spatial color indexing schemes, Color-WISE and the color correlogram approaches, respectively, at image block and pixel levels of different granularity. Various experimental results demonstrate the efficacy of our techniques.     

视频局部特征描述子的紧凑表示方法

目的 随着手持移动设备的迅猛发展和大数据时代的到来,以多媒体数据为核心的视觉搜索等研究和应用得到了广泛关注。其中局部特征描述子的压缩、存储和传输起到了举足轻重的作用。为此在传统图像/视频压缩框架中,提出一种高效的视觉局部特征的紧凑表示方法,使得传统内容编码可以适应广泛的检索分析等需求。方法 为了得到紧凑、有区分度、同时高效的局部特征表示,首先引入了多参考的预测机制,在消除了时空冗余的同时,通过充分利用视频纹理编码的信息,消除了来自纹理-特征之间的冗余。此外,还提出了一种新的率失真优化方法——码率-准确率最优化方法,使得基于匹配/检索应用的性能达到最优。结果 在不同数据集上进行验证实验,和最新的视频局部描述子压缩框架进行比较,本文方法能够在保证匹配和检索性能的基础上,显著地减少特征带来的比特消耗,达到大约150:1的压缩比。结论 本文方法适用于传统图像/视频编码框架,通过在码流中嵌入少量表示特征的信息,即可实现高效的检索性能,是一种面向检索等智能设备应用的新型多媒体内容编码框架。     

基于多模态融合与多层注意力的视频内容文本表述研究

针对现有视频内容文本表述模型存在生成的文本表述单一、准确率不高等问题,提出一种融合帧级图像及音频信息的视频内容文本表述模型。基于自注意力机制设计单模态嵌入层网络结构,并将其嵌入单模态特征中学习单模态特征参数。采用联合表示、协作表示两种方法对单模态嵌入层输出的高维特征向量进行双模态特征融合,使模型能关注视频中不同目标间的交互关系,从而生成更加丰富、准确的视频文本表述。使用大规模数据集对模型进行预训练,并提取视频帧、视频所携带的音频等表征信息,将其送入编解码器实现视频内容的文本表述。在MSR-VTT和LSMDC数据集上的实验结果表明,所提模型的BLEU4、METEOR、ROUGEL和CIDEr指标分别为0.386、0.250、0.609和0.463,相较于MSR-VTT挑战赛中IIT DeIhi发布的模型,分别提升了0.082、0.037、0.115和0.257,能有效提升视频内容文本表述的准确率。     

Scalable color image indexing and retrieval using vector wavelets

This paper presents a scalable content-based image indexing and retrieval system based on vector wavelet coefficients of color images. Highly decorrelated wavelet coefficient planes are used to acquire a search efficient feature space. The feature space is subsequently indexed using properties of all the images in the database. Therefore, the feature key of an image not only corresponds to the content of the image itself but also to how much the image is different from the other images being stored in the database. The search time linearly depends on the number of images similar to the query image and is independent of the database size. We show that, in a database of 5,000 images, query search takes less than 30 msec on a 266 MHz Pentium II processor, compared to several seconds of retrieval time in the earlier systems proposed in the literature     

An efficient compressed domain video indexing method

Video indexing is employed to represent the features of video sequences. Motion vectors derived from compressed video are preferred for video indexing because they can be accessed by partial decoding; thus, they are used extensively in various video analysis and indexing applications. In this study, we introduce an efficient compressed domain video indexing method and implement it on the H.264/AVC coded videos. The video retrieval experimental evaluations indicate that the video retrieval based on the proposed indexing method outperforms motion vector based video retrieval in 74 % of queries with little increase in computation time. Furthermore, we compared our method with a pixel level video indexing method which employs both temporal and spatial features. Experimental evaluation results indicate that our method outperforms the pixel level method both in performance and speed. Hence considering the speed and precision characteristics of indexing methods, the proposed method is an efficient indexing method which can be used in various video indexing and retrieval applications.