一种多特征动态集成的3D模型检索方法

基于内容的3D模型检索是多媒体信息检索的热点研究问题之一,其主要的问题是提取3D模型的形状特征,但是采用单一特征很难保证检索系统对于任意输入查询模型都有很好的检索准确率.为了提高准确率,引入多种形状特征,通过加权求和在输出层融合,得到总体上模型间的相似度.每种特征采用动态权值,针对输入查询模型的不同,根据用户反馈对知识库中的权值自动更新.采用了形状分布特征和球面调和特征进行系统的建模,实验表明,文中方法比采用单特征检索的准确率大大提高.     

基于特征的二值商标分层检索方法

针对以往商标图像搜索方法单一,并且缺少用户反馈的缺点,文中提出一种基于形状特征和纹理特征的二值商标图像分层搜索方法.该方法首先利用HU不变矩提取图像的形状特征;然后利用欧氏距离来测量图像的相似度,进行第一次筛选得到一个数目不定的候选集;接着利用灰度直方图的方法提取候选集中的图像的纹理信息,进行第二次筛选;最后通过用户的反馈进行不断的优化.从而使检索出来的图像更加迅速和准确,更加符合人的视觉感受.     

基于NSCT边缘直方图的草图检索

基于草图的图像检索,存在着大量的应用空间。针对查询草图和数据库图像的差异,提出了一种利用非降采样Cotourlet（NSCT）变换和边缘直方图的草图检索算法。利用NSCT变换分解图像,对所有子带进行边缘检测,提取子带的边缘直方图用来作为描述图像和草图的特征。将该方法与边缘直方图比较实验表明,其检索效果更准确。     

采用形态学边界特征的医学图像检索

特征提取是基于内容的图像检索(CBIR)中的关键步骤,如何有效提取反映高层语义的图像特征对于医学图像的检索至关重要.提出一种基于边界形状特征的医学图像检索方法.该方法首先通过多尺度形态学方法检测图像边界点,然后对边界图像进行形状特征提取,构建边界的形状密度直方图,最后通过相似性匹配实现医学图像检索.实验结果证明了所提取的边界形状特征在医学图像检索中的有效性,通过对比实验给出了结果分析和进一步的研究思路.     

改进颜色和小波变换纹理的图像检索方法研究

针对传统图像检索无法体现对检索示例图像中多个不同对象的检索要求程度的问题,提出一种改进颜色特征和小波变换纹理特征的图像检索方法。首先提取出图像的多个感兴趣区域,由感兴趣的不同程度分别赋予不同大小的权值;然后提取颜色特征和纹理特征,分别用对应位置相似度计算、感兴趣区域与检索数据库中图像整体的相似度计算和整体检索示例图像与检索图像数据库中图像相似度计算三种不同方法计算出两幅图像的相似度,取最大的相似度作为两幅图像的最终相似度;对检索示例图像与检索数据库中每个图像的相似度按大小进行排序,选择最相似的图像作为检索结果。实验结果表明,该方法提高了对图像检索的性能,体现了个性化检索,对图像检索具有很好的效果。     

多特征动态融合的三维模型检索方法

提出一种基于二维正交投影图像的多特征动态融合的三维模型检索方法.首先计算三维模型的二维正交投影图像,然后提取二维正交投影图像的投影直方图和Zernike矩特征,通过加权求和在输出层融合,得到总体上模型间的相似度.每种特征采用动态权值,针对输入查询模型的不同,根据用户反馈自动更新知识库中的权值.实验表明,该方法在提高检索准确率的同时,也能保证检索效率.     

基于空间注意力下边缘图融合的草图图像检索

基于草图的图像检索任务根据用户提供的手绘草图,从图像数据库中检索得到与该草图对应的自然图像.与传统基于内容的图像检索不同,草图和自然图像间存在明显的域差异,这使得二者的特征难以直接进行比较.针对自然图像边缘图和草图的相似性,提出了空间注意力下的边缘图融合模型,将自然图像和对应的边缘图分别编码到各自的特征空间,再通过空间注意力掩膜进行加权融合,进而用于草图图像检索.所提模型可以更有效地编码物体边缘轮廓的特征,分别在Sketchy和Flickr15K数据集的草图图像检索任务上取得了比前人方法更高的Recall@1和MeanAP指标.     

基于颜色和形状特征的图像检索方法

提出了一种基于颜色和形状特征的图像检索方法。在对HSV颜色模型量化的基础上,提取颜色直方图作为图像的颜色特征。在提取形状特征时,结合颜色量化结果,利用图像分割提取图像的形状特征,利用两特征的加权距离计算图像之间的相似度,而后进行图像检索。实验结果表明,该方法取得了较好的检索效果。     

颜色和形状特征的商品图像检索技术的应用

针对基于内容的商品图像检索,提出了以颜色和区域形状为特征的检索技术方案.以非等间隔量化方法提取颜色直方图,同时设置显著水平阈值提取视觉主色,从而获取颜色特征.利用二维函数矩构造出的Hu矩特征值,并对矩特征值进行权重设置,提取了比较符合人类视觉的区域形状特征.两种特征组合检索,实现了商品图像的较高相似性度量的匹配.     

基于目标形状的图像检索

针对已有的基于形状的图像检索中目标形状描述方法的不足对其进行改进。首先对目标图像进行一系列预处理,得到图像的外部轮廓,利用改进的霍夫变换提取目标轮廓的线性特征;然后引入成对几何特征即有向相对角和有向相对位置来描述图像的形状;最后利用直方图相交算法衡量图像特征间的相似度。实验证明,利用本文改进的方法所描述的形状属性来检索数据库中的图像具有较高的效率。     

Sketch-based 3D model retrieval by incorporating 2D-3D alignment

Sketch-based 3D model retrieval is very important for applications such as 3D modeling and recognition. In this paper, a sketch-based retrieval algorithm is proposed based on a 3D model feature named View Context and 2D relative shape context matching. To enhance the accuracy of 2D sketch-3D model correspondence as well as the retrieval performance, we propose to align a 3D model with a query 2D sketch before measuring their distance. First, we efficiently select some candidate views from a set of densely sampled views of the 3D model to align the sketch and the model based on their View Context similarities. Then, we compute the more accurate relative shape context distance between the sketch and every candidate view, and regard the minimum one as the sketch-model distance. To speed up retrieval, we precompute the View Context and relative shape context features of the sample views of all the 3D models in the database. Comparative and evaluative experiments based on hand-drawn and standard line drawing sketches demonstrate the effectiveness and robustness of our approach and it significantly outperforms several latest sketch-based retrieval algorithms.     

基于渲染图像角度结构特征的三维模型检索方法

为了充分利用三维模型的颜色、形状、纹理等特征,提出以三维模型渲染图像为数据集,利用渲染图像角度结构特征实现三维模型检索。首先,该方法以三维模型渲染图像为测试集,利用已有类别标记的自然图像作为训练集,通过骨架形状上下文特征对渲染图像进行分类,提取角度结构特征,建立特征库;然后,对输入的自然图像提取角度结构特征,与特征库中的角度结构特征进行相似度匹配计算,实现三维模型检索。实验结果表明, 充分利用 渲染图像的颜色、形状和空间信息是实现三维模型检索的有效方法。     

基于球体投影的三维模型检索

针对草图检索三维模型时存在的域不匹配和如何选取视图等问题,提出一种基于球体投影的三维模型检索方法。针对域不匹配问题,提出基于球体投影的二维视图获取方法,并使用高斯差分和贝塞尔曲线完成线图的提取;利用草图和投影图像之间的关系构建分类器,以获取模型的最优视图;通过两个Siamese网络获取草图和二维视图的特征,并用联合贝叶斯（Joint Bayesian）方法来融合二者的输出,从而获得最终结果。实验证明了该方法的可行性,与其他方法相比具有更好的检索效果。     

3D model retrieval using principal plane analysis and dynamic programming

Three dimensional models play an important role in many applications; the problem is how to select the appropriate models from a 3D database rapidly and accurately. In recent years, a variety of shape representations, statistical methods, and geometric algorithms have been proposed for matching 3D shapes or models. In this paper, we propose a 3D shape representation scheme based on a combination of principal plane analysis and dynamic programming. The proposed 3D shape representation scheme consists of three steps. First, a 3D model is transformed into a 2D image by projecting the vertices of the model onto its principal plane. Second, the convex hall of the 2D shape of the model is further segmented into multiple disjoint triangles using dynamic programming. Finally, for each triangle, a projection score histogram and moments are extracted as the feature vectors for similarity searching. Experimental results showed the robustness of the proposed scheme, which resists translation, rotation, scaling, noise, and destructive attacks. The proposed 3D model retrieval method performs fairly well in retrieving models having similar characteristics from a database of 3D models.     

Extracting 3D model feature lines based on conditional random fields

We propose a 3D model feature line extraction method using templates for guidance. The 3D model is first projected into a depth map, and a set of candidate feature points are extracted. Then, a conditional random fields (CRF) model is established to match the sketch points and the candidate feature points. Using sketch strokes, the candidate feature points can then be connected to obtain the feature lines, and using a CRF-matching model, the 2D image shape similarity features and 3D model geometric features can be effectively integrated. Finally, a relational metric based on shape and topological similarity is proposed to evaluate the matching results, and an iterative matching process is applied to obtain the globally optimized model feature lines. Experimental results showed that the proposed method can extract sound 3D model feature lines which correspond to the initial sketch template.     

基于深度卷积神经网络的三维模型检索

为了更有效地利用三维模型数据集进行特征的自主学习,提出一种使用自然图像作为输入源,以三维模型的较优视图集为基础,通过深度卷积神经网络的训练提取深度特征用于检索的三维模型检索方法。首先,从多个视点对三维模型进行视图提取,并根据灰度熵的排序选取较优视图;然后,通过深度卷积神经网络对视图集进行训练,从而提取较优视图的深度特征并进行降维,同时,对输入的自然图像提取边缘轮廓图,经过相似度匹配获得一组三维模型;最后,基于检索结果中同类模型总数占检索列表长度的比例对列表进行重排序,从而获得最终的检索结果。实验结果表明,该算法能够有效利用深度卷积神经网络对三维模型的视图进行深度特征提取,同时降低了输入源的获取难度,有效提高了检索效果。     

3D face reconstruction from a single image using a single reference face shape

Human faces are remarkably similar in global properties, including size, aspect ratio, and location of main features, but can vary considerably in details across individuals, gender, race, or due to facial expression. We propose a novel method for 3D shape recovery of faces that exploits the similarity of faces. Our method obtains as input a single image and uses a mere single 3D reference model of a different person's face. Classical reconstruction methods from single images, i.e., shape-from-shading, require knowledge of the reflectance properties and lighting as well as depth values for boundary conditions. Recent methods circumvent these requirements by representing input faces as combinations (of hundreds) of stored 3D models. We propose instead to use the input image as a guide to "mold" a single reference model to reach a reconstruction of the sought 3D shape. Our method assumes Lambertian reflectance and uses harmonic representations of lighting. It has been tested on images taken under controlled viewing conditions as well as on uncontrolled images downloaded from the Internet, demonstrating its accuracy and robustness under a variety of imaging conditions and overcoming significant differences in shape between the input and reference individuals including differences in facial expressions, gender, and race.     

3D model retrieval based on color + geometry signatures

Color plays a significant role in the recognition of 3D objects and scenes from the perspective of cognitive psychology. In this paper, we propose a new 3D model retrieval method, focusing on not only the geometric features but also the color features of 3D mesh models. Firstly, we propose a new sampling method that samples the models in the regions of either geometry-high-variation or color-high-variation. After collecting geometry + color sensitive sampling points, we cluster them into several classes by using a modified ISODATA algorithm. Then we calculate the feature histogram of each model in the database using these clustered sampling points. For model retrieval, we compare the histogram of an input model to the stored histograms in the database to find out the most similar models. To evaluate the retrieval method based on the new color + geometry signatures, we use the precision/recall performance metric to compare our method with several classical methods. Experiment results show that color information does help improve the accuracy of 3D model retrieval, which is consistent with the postulate in psychophysics that color should strongly influence the recognition of objects.     

DeepSketch 3

Freehand sketches are a simple and powerful tool for communication. They are easily recognized across cultures and suitable for various applications. In this paper, we use deep convolutional neural networks (ConvNets), state-of-the-art in the field of sketch recognition, to address several applications of automatic sketch processing: complete and partial sketch recognition, sketch retrieval using query-by-example (QbE), and sketch-based image retrieval (SBIR) i.e the retrieval of images using a QbE paradigm but where the query is a sketch. We first focus on improving sketch recognition. For this purpose we compare different ConvNet architectures, training paradigms and data fusion schemes. This enabled us to outperform previous state-of-the-art in two large scale benchmarks for sketch classification. We achieved a mean average accuracy of 79.18% for the TU-Berlin sketch benchmark and 93.02% for the sketchy database. For partial sketch recognition, we were able to produce a system that achieves a mean average accuracy of 52.58% with only 40% of the strokes. We then conduct a comprehensive study of ConvNets features to enhance sketch retrieval and image retrieval, using a kNN similarity search paradigm in the ConvNet feature space. For the sketch retrieval tasks, we compare the performance obtained with features extracted from various depths (ConvNet layers) using one of the best performing model from the previous work. For the sketch-based image retrieval (SBIR), a sketch query is used to retrieve images of objects that belong to the same category, or even with a shape and pose close to the sketch query. The main challenge in the field of SBIR is to obtain efficient cross-domain features for sketch-image similarity measure. For this, besides comparing features extracted from different depth, we additionally compare different training approaches (some novel) for the ConvNets applied to sketches and images. Eventually, our best SBIR system achieves state-of-the-art results on the sketchy database (close to 40% recall at k = 1).