Capture and fusion of 3d surface texture

Image fusion is a process that multiple images of a scene are combined to form a single image. The aim of image fusion is to preserve the full content and retain important features of each original image. In this paper, we propose a novel approach based on wavelet transform to capture and fusion of real-world rough surface textures, which are commonly used in multimedia applications and referred to as3D surface texture. These textures are different from 2D textures as their appearances can vary dramatically with different illumination conditions due to complex surface geometry and reflectance properties. In our approach, we first extract gradient/height and albedo maps from sample 3D surface texture images as their representation. Then we measure saliency of wavelet coefficients of these 3D surface texture representations. The saliency values reflect the meaningful content of the wavelet coefficients and are consistent with human visual perception. Finally we fuse the gradient/height and albedo maps based on the measured saliency values. This novel scheme aims to preserve the original texture patterns together with geometry and reflectance characteristics from input images. Experimental results show that the proposed approach can not only capture and fuse 3D surface texture under arbitrary illumination directions, but also has the ability to retain the surface geometry properties and preserve perceptual features in the original images.     

Periodicity, directionality, and randomness: Wold features forimage modeling and retrieval

One of the fundamental challenges in pattern recognition is choosing a set of features appropriate to a class of problems. In applications such as database retrieval, it is important that image features used in pattern comparison provide good measures of image perceptual similarities. We present an image model with a new set of features that address the challenge of perceptual similarity. The model is based on the 2D Wold decomposition of homogeneous random fields. The three resulting mutually orthogonal subfields have perceptual properties which can be described as “periodicity,” “directionality,” and “randomness,” approximating what are indicated to be the three most important dimensions of human texture perception. The method presented improves upon earlier Wold-based models in its tolerance to a variety of local inhomogeneities which arise in natural textures and its invariance under image transformation such as rotation. An image retrieval algorithm based on the new texture model is presented. Different types of image features are aggregated for similarity comparison by using a Bayesian probabilistic approach. The, effectiveness of the Wold model at retrieving perceptually similar natural textures is demonstrated in comparison to that of two other well-known pattern recognition methods. The Wold model appears to offer a perceptually more satisfying measure of pattern similarity while exceeding the performance of these other methods by traditional pattern recognition criteria. Examples of natural scene Wold texture modeling are also presented     

Spherical Correlation of Visual Representations for 3D Model Retrieval

In recent years we have seen a tremendous growth in the amount of freely available 3D content, in part due to breakthroughs for 3D model design and acquisition. For example, advances in range sensor technology and design software have dramatically reduced the manual labor required to construct 3D models. As collections of 3D content continue to grow rapidly, the ability to perform fast and accurate retrieval from a database of models has become a necessity. At the core of this retrieval task is the fundamental challenge of defining and evaluating similarity between 3D shapes. Some effective methods dealing with this challenge consider similarity measures based on the visual appearance of models. While collections of rendered images are discriminative for retrieval tasks, such representations come with a few inherent limitations such as restrictions in the image viewpoint sampling and high computational costs. In this paper we present a novel algorithm for model similarity that addresses these issues. Our proposed method exploits techniques from spherical signal processing to efficiently evaluate a visual similarity measure between models. Extensive evaluations on multiple datasets are provided.     

A multi-representation re-ranking model for Personalized Product Search

In recent years, a multitude of e-commerce websites arose. Product Search is a fundamental part of these websites, which is often managed as a traditional retrieval task. However, Product Search has the ultimate goal of satisfying specific and personal user needs, leading users to find and purchase what they are looking for, based on their preferences. To maximize users’ satisfaction, Product Search should be treated as a personalized task. In this paper, we propose and evaluate a simple yet effective personalized results re-ranking approach based on the fusion of the relevance score computed by a well-known ranking model, namely BM25, with the scores deriving from multiple user/item representations. Our main contributions are: (1) we propose a score fusion-based approach for personalized re-ranking that leverages multiple user/item representations, (2) our approach accounts for both content-based features and collaborative information (i.e. features extracted from the user–item interactions graph), (3) the proposed approach is fast and scalable, can be easily added on top of any search engine and it can be extended to include additional features. The performed comparative evaluations show that our model can significantly increase the retrieval effectiveness of the underlying retrieval model and, in the great majority of cases, outperforms modern Neural Network-based personalized retrieval models for Product Search.     

改进颜色和小波变换纹理的图像检索方法研究

针对传统图像检索无法体现对检索示例图像中多个不同对象的检索要求程度的问题,提出一种改进颜色特征和小波变换纹理特征的图像检索方法。首先提取出图像的多个感兴趣区域,由感兴趣的不同程度分别赋予不同大小的权值;然后提取颜色特征和纹理特征,分别用对应位置相似度计算、感兴趣区域与检索数据库中图像整体的相似度计算和整体检索示例图像与检索图像数据库中图像相似度计算三种不同方法计算出两幅图像的相似度,取最大的相似度作为两幅图像的最终相似度;对检索示例图像与检索数据库中每个图像的相似度按大小进行排序,选择最相似的图像作为检索结果。实验结果表明,该方法提高了对图像检索的性能,体现了个性化检索,对图像检索具有很好的效果。     

基于改进双树复小波和灰度-梯度共生矩阵的纹理图像检索算法

针对双树复小波变换缺少不同尺度纹理的空间分布特征的缺陷,提出了一种改进双树复小波和灰度-梯度共生矩阵相融合的纹理图像检索新算法。首先,该算法将图像进行非均匀分块,并对分块的图像进行双树复小波变换,以此增加不同尺度下的空间信息;其次,利用灰度-梯度共生矩阵提取4个统计量特征;然后, 融合 两种方法提取的纹理特征以得到图像检索的纹理特征;最后,用Canberra距离进行相似性度量并输出图像检索的结果。实验结果表明,该方法对纹理图像有较好的检索效果。     

一种主动感知的纹理图像检索方法(英文)

基于Contourlet变换和支持向量机提出了一种新的纹理图像检索方法。在这种方法中,能量和广义高斯分布参数被用做Contourlet子带图像的特征。通过这种表示,提出了由一类和二类支持向量机组成的两阶段检索算法来完成感知相似性测度。通过具有640个纹理图像的VisTex库和具有1760个纹理图像的Brodatz库证明了所提方法的有效性。实验结果表明,对于这两个纹理库,新的纹理图像检索方法的平均检索率分别达99.38%和98.07%。     

一种基于区域综合特征的彩色图像检索方法

提出了一种基于区域综合特征的彩色图像检索算法.该算法首先结合MPEG-7视觉内容描述对真彩色图像进行量化处理,并将量化后的图像划分成若干个子区域.然后选取子区域的主要颜色及其所占百分率作为颜色特征,选取子区域的熵、能量和对比度作为纹理特征.再综合利用上述颜色、纹理两个特征计算图像间内容的相似度,并进行彩色图像检索.仿真实验表明,该算法能够准确和高效地查找出用户所需内容的彩色图像,并且具有较好的查准率、查全率和较快的检索速度.     

3D head model classification by evolutionary optimization of the Extended Gaussian Image representation

Classification of 3D head models based on their shape attributes for subsequent indexing and retrieval are important in many applications, as in hierarchical content-based retrieval of these head models for virtual scene composition, and the automatic annotation of these characters in such scenes. While simple feature representations are preferred for more efficient classification operations, these features may not be adequate for distinguishing between the subtly different head model classes. In view of these, we propose an optimization approach based on genetic algorithm (GA) where the original model representation is transformed in such a way that the classification rate is significantly enhanced while retaining the efficiency and simplicity of the original representation. Specifically, based on the Extended Gaussian Image (EGI) representation for 3D models which summarizes the surface normal orientation statistics, we consider these orientations as random variables, and proceed to search for an optimal transformation for these variables based on genetic optimization. The resulting transformed distributions for these random variables are then used as the modified classifier inputs. Experiments have shown that the optimized transformation results in a significant improvement in classification results for a large variety of class structures. More importantly, the transformation can be indirectly realized by bin removal and bin count merging in the original histogram, thus retaining the advantage of the original EGI representation.     

图像纹理特征相似性度量的研究

纹理是图像的重要属性,基于纹理特征检索图像是当前的研究热点,对图像的纹理进行相似性比较是进行图像检索的关键.根据纹理的特点,本文将通用的向量空间模型进行拓展,构建了一个针对簇集进行相似性匹配的模型-聚类空间模型,对图像纹理相似性进行度量,并据此实现了无需分割的多纹理图像检索.我们分别针对单纹理图像和自然图像库进进行了实验,获得的实验结果与人类视觉认知的结果一致.     

Mobile multi-view object image search

High user interaction capability of mobile devices can help improve the accuracy of mobile visual search systems. At query time, it is possible to capture multiple views of an object from different viewing angles and at different scales with the mobile device camera to obtain richer information about the object compared to a single view and hence return more accurate results. Motivated by this, we propose a new multi-view visual query model on multi-view object image databases for mobile visual search. Multi-view images of objects acquired by the mobile clients are processed and local features are sent to a server, which combines the query image representations with early/late fusion methods and returns the query results. We performed a comprehensive analysis of early and late fusion approaches using various similarity functions, on an existing single view and a new multi-view object image database. The experimental results show that multi-view search provides significantly better retrieval accuracy compared to traditional single view search.     

Content-based image retrieval approach for biometric security using colour,texture and shape features controlled by fuzzy heuristics

In this paper, we discuss a new content-based image retrieval approach for biometric security, which is based on colour, texture and shape features and controlled by fuzzy heuristics. The proposed approach is based on the three well-known algorithms: colour histogram, texture and moment invariants. The use of these three algorithms ensures that the proposed image retrieval approach produces results which are highly relevant to the content of an image query, by taking into account the three distinct features of the image and similarity metrics based on Euclidean measure. Colour histogram is used to extract the colour features of an image. Gabor filter is used to extract the texture features and the moment invariant is used to extract the shape features of an image. The evaluation of the proposed approach is carried out using the standard precision and recall measures, and the results are compared with the well-known existing approaches. We present results which show that our proposed approach performs better than these approaches.     

基于渲染图像角度结构特征的三维模型检索方法

为了充分利用三维模型的颜色、形状、纹理等特征,提出以三维模型渲染图像为数据集,利用渲染图像角度结构特征实现三维模型检索。首先,该方法以三维模型渲染图像为测试集,利用已有类别标记的自然图像作为训练集,通过骨架形状上下文特征对渲染图像进行分类,提取角度结构特征,建立特征库;然后,对输入的自然图像提取角度结构特征,与特征库中的角度结构特征进行相似度匹配计算,实现三维模型检索。实验结果表明, 充分利用 渲染图像的颜色、形状和空间信息是实现三维模型检索的有效方法。     

一种基于多特征融合的彩色图像检索方法

提出了一种基于特征融合的图像检索方法。利用图像的HSV直方图特征建立图像颜色直方图,并采用直方图二次式距离公式取得图像相似性度量值;利用图像的纹理特征建立256维的LBP特征向量,并利用欧式距离取得相似性度量值;通过两特征融合的方法取得图像检索中关键图和检索图之间的相似度值,使得检索取得更好的效果。实验表明,在查准率和...     

New Spiking Cortical Model for Invariant Texture Retrieval and Image Processing

Based on the studies of existing local-connected neural network models, in this brief, we present a new spiking cortical neural networks model and find that time matrix of the model can be recognized as a human subjective sense of stimulus intensity. The series of output pulse images of a proposed model represents the segment, edge, and texture features of the original image, and can be calculated based on several efficient measures and forms a sequence as the feature of the original image. We characterize texture images by the sequence for an invariant texture retrieval. The experimental results show that the retrieval scheme is effective in extracting the rotation and scale invariant features. The new model can also obtain good results when it is used in other image processing applications.      

A new matching strategy for content based image retrieval system

Adopting effective model to access the desired images is essential nowadays with the presence of a huge amount of digital images. The present paper introduces an accurate and rapid model for content based image retrieval process depending on a new matching strategy. The proposed model is composed of four major phases namely: features extraction, dimensionality reduction, ANN classifier and matching strategy. As for the feature extraction phase, it extracts a color and texture features, respectively, called color co-occurrence matrix (CCM) and difference between pixels of scan pattern (DBPSP). However, integrating multiple features can overcome the problems of single feature, but the system works slowly mainly because of the high dimensionality of the feature space. Therefore, the dimensionality reduction technique selects the effective features that jointly have the largest dependency on the target class and minimal redundancy among themselves. Consequently, these features reduce the calculation work and the computation time in the retrieval process. The artificial neural network (ANN) in our proposed model serves as a classifier so that the selected features of query image are the input and its output is one of the multi classes that have the largest similarity to the query image. In addition, the proposed model presents an effective feature matching strategy that depends on the idea of the minimum area between two vectors to compute the similarity value between a query image and the images in the determined class. Finally, the results presented in this paper demonstrate that the proposed model provides accurate retrieval results and achieve improvement in performance with significantly less computation time compared with other models.     

利用脉冲耦合神经网络的纹理图像检索方法

为提高基于内容的图像检索系统(CBIR)中纹理特征提取的有效性,进一步提升CBIR系统的整体性能。提出了一种基于脉冲耦合神经网络的纹理图像检索方法。脉冲耦合神经网络(PCNN)是新一代的人工神经网络,在数据处理上具有很多优势。特征提取时具有平移、旋转、尺度、扭曲等不变性,以及很好的抗噪性,而这一点非常适合于图像检索系统。利用PCNN及简化模型ICM得到对应于不同灰度值的二值图像序列,计算序列中每幅图像的熵序列,其一维的特征矢量作为纹理特征。采用Eu-clidean距离进行相似度计算,建立了一套基于示例查询图像的纹理图像检索系统。实验结果表明,与小波包等特征提取方法相比,该方法不仅对噪声具有较强的鲁棒性,同时能降低特征向量维数,具有尺度、平移和旋转不变性,而且能取得更高的检索率。     

On image search result aggregation

In text retrieval, search result aggregation has been demonstrated how it has outperformed the retrieval results by single retrieval models. In general, search result aggregation for a specific query is based on combining different search results, which are produced by using different feature representations and/or different retrieval models. Particularly, several well-known combination methods, such as Borda count and CombSUM, have been proposed in the literature. However, in image retrieval the semantic gap problem limits the performances of current image retrieval systems. Since very few studies focus on search result aggregation in image retrieval, the aim of this paper is to assess the retrieval performances of different search result aggregation strategies and combination methods. Specifically, five different feature representations, five different distance functions as the retrieval models, and five different combination methods are used. Our experimental results based on Caltech 101, Caltech 256, and NUS-WIDE-LITE show that search result aggregation can definitely outperform single search results. In addition, among three aggregation strategies the one by combining five search results based on each best feature representation by their best distance function can provide the highest rate of precision rate.     

底层内容特征的融合在图像检索中的研究进展

在基于内容的图像检索中,提取颜色、纹理、形状或空间信息等底层特征是目前最常用且简便的表征图像的方法。但使用单一底层特征容易忽视特征间的相互联系,无法对图像以各种形式提供的信息加以充分利用,限制了众多特征联合诠释图像的可能性。底层内容特征的融合可以全面同时互补地表示图像中包含的各类信息,有效地利用特征间的联系,提高了图像内容表示的效率和精度。该文对现有的底层内容的融合特征提取算法进行总结,提出了一种以融合的层次及融合内容为依据的分类体系,指出了基于融合特征的图像检索现今存在的问题以及一些可能的研究方向。