深度学习和迭代量化在图像检索中的应用研究

基于内容的图像检索的关键在于对图像进行特征提取和对特征进行多比特量化编码 。近年来,基于内容的图像检索使用低级可视化特征对图像进行描述,存在“语义鸿沟”问题;其次,传统量化编码使用随机生成的投影矩阵,该矩阵与特征数据无关,因此不能保证量化的精确度。针对目前存在的这些问题,本文结合深度学习思想与迭代量化思想,提出基于卷积神经网络VGG16和迭代量化(Iterative Quantization, ITQ)的图像检索方法。使用在公开数据集上预训练VGG16网络模型,提取基于深度学习的图像特征;使用ITQ方法对哈希哈函数进行训练,不断逼近特征与设定比特数的哈希码之间的量化误差最小值,实现量化误差的最小化;最后使用获得的哈希码进行图像检索。本文使用查全率、查准率和平均精度均值作为检索效果的评价指标,在Caltech256图像库上进行测试。实验结果表明,本文提出的算法在检索优于其他主流图像检索算法。      

基于Word Embedding的遥感影像检测分割

遥感影像检测分割技术通常需提取影像特征并通过深度学习算法挖掘影像的深层特征来实现.然而传统特征（如颜色特征、纹理特征、空间关系特征等）不能充分描述影像语义信息,而单一结构或串联算法无法充分挖掘影像的深层特征和上下文语义信息.针对上述问题,本文通过词嵌入将空间关系特征映射成实数密集向量,与颜色、纹理特征的结合.其次,本文构建基于注意力机制下图卷积网络和独立循环神经网络的遥感影像检测分割并联算法（Attention Graph Convolution Networks and Independently Recurrent Neural Network,ATGIR）.该算法首先通过注意力机制对结合后的特征进行概率权重分配;然后利用图卷积网络（GCNs）算法对高权重的特征进一步挖掘并生成方向标签,同时使用独立循环神经网络（IndRNN）算法挖掘影像特征中的上下文信息,最后用Sigmoid分类器完成影像检测分割任务.以胡杨林遥感影像检测分割任务为例,我们验证了提出的特征提取方法和ATGIR算法能有效提升胡杨林检测分割任务的性能.     

基于兴趣目标的图像检索

当前图像检索算法通常针对整体图像提取特征以完成检索任务.然而,在很多情况下用户只会关注图像的一部分,即他们的兴趣目标.此时,从整体图像提取的特征一部分是有效的,另一部分则是无效的且会对检索过程带来消极影响.为此,本文提出基于兴趣目标的图像检索方案,并借助于现有的显著性检测、图像分割、特征提取等技术实现一款有效的图像检索算法.首先采用HS （Hierarchical Saliency,分层显著性）检测算法分析用户的兴趣目标并应用SC （Saliency-based Image Cut,基于显著性的图像分割）算法将其分割,然后针对兴趣目标提取HSV （Hue、Saturation、Value,色调、饱和度、明度）颜色特征、SIFT （Scale Invariant Feature Transform,尺度不变特征变换）局部特征和CNN （Convolutional Neural Network,卷积神经网络）语义特征,最后计算其与数据库图像的相似度并根据相似度排序返回检索结果.仿真实验结果表明,本文算法在解决"这是什么东西"这类图像检索任务时明显优于现有的图像检索算法.     

现勘图像检索综述

现勘图像检索是进行证据图像比对以获取物证信息的重要手段.本文基于目前应用广泛的现勘图像数据库,根据图像内容将图像分为鞋印、指纹、纹身等种类.并通过对现勘图像的两项关键技术即低层数字特征提取和高层语义分析的总结,从颜色特征、纹理特征、边缘提取等方面综述了现勘图像低层数字特征提取技术,从利用语义模板和数据库本体结构、机器学习算法、引入人工反馈三大类高层语义提取技术综述了现勘图像高层语义分析的研究成果.最后,结合公安行业利用现勘图像获取物证线索的实际应用需求,指出了通过引入公安行业先验知识来提高检索效率等研究方向.     

Similarity-based online feature selection in content-based image retrieval.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.     

AMSFF-Net: Attention-Based Multi-Stream Feature Fusion Network for Single Image Dehazing

In this paper, an end-to-end convolutional neural network is proposed to recover haze-free image named as Attention-Based Multi-Stream Feature Fusion Network (AMSFF-Net). The encoder-decoder network structure is used to construct the network. An encoder generates features at three resolution levels. The multi-stream features are extracted using residual dense blocks and fused by feature fusion blocks. AMSFF-Net has ability to pay more attention to informative features at different resolution levels using pixel attention mechanism. A sharp image can be recovered by the good kernel estimation. Further, AMSFF-Net has ability to capture semantic and sharp textural details from the extracted features and retain high-quality image from coarse-to-fine using mixed-convolution attention mechanism at decoder. The skip connections decrease the loss of image details from the larger receptive fields. Moreover, deep semantic loss function emphasizes more semantic information in deep features. Experimental findings prove that the proposed method outperforms in synthetic and real-world images.     

基于特征复用的膨胀-残差网络的SAR图像超分辨重建

对于合成孔径雷达(SAR)图像,传统的超分辨重建方法对视觉特征的人为构造十分依赖,基于普通卷积神经网络(CNN)的超分辨重建方法对微小目标的重建能力较弱,对边缘轮廓的保真度较差。针对以上问题,该文提出一种基于特征复用的膨胀-残差卷积超分辨网络模型,同时引入感知损失,实现了精确的SAR图像4倍语义级超分辨。该方法为增加网络感受野,采用膨胀-残差卷积(DR-CNN)结构用于限制模型中特征图分辨率的严重损失,提高网络对微小细节的敏感度;为实现不同层级的特征最大化利用,将不同层级的特征图进行级联,形成一种特征复用结构(FRDR-CNN),以此大幅度提升特征提取模块的效率,进一步提升超分辨精度;针对SAR图像特殊的相干斑噪声干扰,引入感知损失,使得该方法在恢复图像边缘和精细的纹理信息方面具有优越表现。文中实验表明,与传统算法以及目前较为流行的几种全卷积神经网络超分辨重建算法相比,该文采用的FRDR-CNN模型在视觉上对小物体的超分辨重建能力更强,对边界等轮廓信息的重建更准确,客观指标中的峰值信噪比(PSNR)和结构相似性指数(SSIM)分别为33.5023 dB和0.5127,边缘保持系数(EPD-ROA)在水平和垂直方向上分别为0.4243和0.4373。      

A memory learning framework for effective image retrieval.

Most current content-based image retrieval systems are still incapable of providing users with their desired results. The major difficulty lies in the gap between low-level image features and high-level image semantics. To address the problem, this study reports a framework for effective image retrieval by employing a novel idea of memory learning. It forms a knowledge memory model to store the semantic information by simply accumulating user-provided interactions. A learning strategy is then applied to predict the semantic relationships among images according to the memorized knowledge. Image queries are finally performed based on a seamless combination of low-level features and learned semantics. One important advantage of our framework is its ability to efficiently annotate images and also propagate the keyword annotation from the labeled images to unlabeled images. The presented algorithm has been integrated into a practical image retrieval system. Experiments on a collection of 10,000 general-purpose images demonstrate the effectiveness of the proposed framework.     

基于深度卷积神经网络和二进制哈希学习的图像检索方法

随着图像数据的迅猛增长,当前主流的图像检索方法采用的视觉特征编码步骤固定,缺少学习能力,导致其图像表达能力不强,而且视觉特征维数较高,严重制约了其图像检索性能。针对这些问题,该文提出一种基于深度卷积神径网络学习二进制哈希编码的方法,用于大规模的图像检索。该文的基本思想是在深度学习框架中增加一个哈希层,同时学习图像特征和哈希函数,且哈希函数满足独立性和量化误差最小的约束。首先,利用卷积神经网络强大的学习能力挖掘训练图像的内在隐含关系,提取图像深层特征,增强图像特征的区分性和表达能力。然后,将图像特征输入到哈希层,学习哈希函数使得哈希层输出的二进制哈希码分类误差和量化误差最小,且满足独立性约束。最后,给定输入图像通过该框架的哈希层得到相应的哈希码,从而可以在低维汉明空间中完成对大规模图像数据的有效检索。在3个常用数据集上的实验结果表明,利用所提方法得到哈希码,其图像检索性能优于当前主流方法。     

CCFNet: Cross-Complementary fusion network for RGB-D scene parsing of clothing images

Schemes to complement context relationships by cross-scale feature fusion have appeared in many RGB-D scene parsing algorithms; however, most of these works conduct multi-scale information interaction after multi-modal feature fusion, which ignores the information loss of the two modes in the original coding. Therefore, a cross-complementary fusion network (CCFNet) is designed in this paper to calibrate the multi-modal information before feature fusion, so as to improve the feature quality of each mode and the information complementarity ability of RGB and the depth map. First, we divided the features into low, middle, and high levels, among which the low-level features contain the global details of the image and the main learning features include texture, edge, and other features. The middle layer features contain not only some global detail features but also some local semantic features. Additionally, the high-level features contain rich local semantic features. Then, the feature information lost in the coding process of low and middle level features is supplemented and extracted through the designed cross feature enhancement module, and the high-level features are extracted through the feature enhancement module. In addition, the cross-modal fusion module is designed to integrate multi-modal features of different levels. The experimental results verify that the proposed CCFNet achieves excellent performance on the RGB-D scene parsing dataset containing clothing images, and the generalization ability of the model is verified by the dataset NYU Depth V2.     

基于深度哈希算法的极光图像分类与检索方法

面对形态万千、变化复杂的海量极光数据,对其进行分类与检索为进一步研究地球磁场物理机制和空间信息具有重要意义。该文基于卷积神经网络(CNN)对图像特征提取方面的良好表现,以及哈希编码可以满足大规模图像检索对检索时间的要求,提出一种端到端的深度哈希算法用于极光图像分类与检索。首先在CNN中嵌入空间金字塔池化(SPP)和幂均值变换(PMT)来提取图像中多种尺度的区域信息;其次在全连接层之间加入哈希层,将全连接层最能表现图像的高维语义信息映射为紧凑的二值哈希码,并在低维空间使用汉明距离对图像对之间的相似性进行度量;最后引入多任务学习机制,充分利用图像标签信息和图像对之间的相似度信息来设计损失函数,联合分类层和哈希层的损失作为优化目标,使哈希码之间可以保持更好的语义相似性,有效提升了检索性能。在极光数据集和 CIFAR-10 数据集上的实验结果表明,所提出方法检索性能优于其他现有检索方法,同时能够有效用于极光图像分类。     

基于高层特征图组合及池化的高分辨率遥感图像检索

高分辨率遥感图像内容复杂,提取特征来准确地表达图像内容是提高检索性能的关键。卷积神经网络(CNN)迁移学习能力强,其高层特征能够有效迁移到高分辨率遥感图像中。为了充分利用高层特征的优点,该文提出一种基于高层特征图组合及池化的方法来融合不同CNN中的高层特征。首先将高层特征作为特殊的卷积层特征,进而在不同输入尺寸下保留高层输出的特征图;然后将不同高层输出的特征图组合成一个更大的特征图,以综合不同CNN学习到的特征;接着采用最大池化的方法对组合特征图进行压缩,提取特征图中的显著特征;最后,采用主成分分析(PCA)来降低显著特征的冗余度。实验结果表明,与现有检索方法相比,该方法提取的特征在检索效率和准确率上都有优势。     

A novel long-term learning algorithm for relevance feedback in content-based image retrieval

Relevance feedback has proven to be a powerful tool to bridge the semantic gap between low-level features and high-level human concepts in content-based image retrieval (CBIR). However, traditional short-term relevance feedback technologies are confined to using the current feedback record only. Log-based long-term learning captures the semantic relationships among images in a database by analyzing the historical relevance information to boost the retrieval performance effectively. In this paper, we propose an expanded-judging model to analyze the historical log data’s semantic information and to expand the feedback sample set from both positive and negative relevant information. The index table is used to facilitate the log analysis. The expanded-judging model is applied in image retrieval by combining with short-term relevance feedback algorithms. Experiments were carried out to evaluate the proposed algorithm based on the Corel image database. The promising experimental results validate the effectiveness of our proposed expanded-judging model.     

基于教与学优化算法的相关反馈图像检索

为提高基于内容的图像检索的检索性能和检索速度,克服低层视觉特征与高层语义概念间的“语义鸿沟”,提出一种基于教与学优化的图像检索相关反馈算法（TLBO-RF）.结合图像检索问题的特殊性和粒子群优化算法的优点,对TLBO算法中个体的更新机制进行了改进,通过将相关图像集的中心作为教师以及引入学员最好学习状态Pbest,使之朝用户感兴趣的相关图像区域快速收敛.将该算法与目前效果最好的两种基于进化算法的相关反馈技术在两套标准图像测试集上进行对比,结果表明本文算法相较于另外两种算法具有明显的优势,不仅提高了图像检索性能,同时也加快了图像检索速度,更好地满足了用户的检索要求.     

基于卷积神经网络结合NSCT的红外与可见光图像融合

传统红外与可见光融合图像在复杂环境下存在细节缺失,特征单一导致目标模糊等问题,本文提出一种基于卷积神经网络结合非下采样轮廓波变换（non-subsampled contourlet transform,NSCT）的红外与可见光图像进行融合的方法。首先,通过卷积神经网络提取红外与可见光目标特征信息,同时利用NSCT对源图像进行多尺度分解,得到源图像的高频系数与低频系数;其次,结合目标特征图利用自适应模糊逻辑与局部方差对比度分别对源图像高频子带与低频子带进行融合;最后,通过逆NSCT变换得到融合图像并与其他5种传统算法进行对比;实验结果表明,本文方法在多个客观评价指标上均有所提高。     

Research on the parallelization of image quality analysis algorithm based on deep learning

Image quality assessment is an indispensable in computer vision applications, such as image classification, image parsing. With the development of Internet, image data acquisition becomes more conveniently. However, image distortion is inevitable due to imperfect image acquisition system, image transmission medium and image recording equipment. Traditional image quality assessment algorithms only focus on low-level visual features such as color or texture, which could not encode high-level features effectively. CNN-based methods have shown satisfactory results in image quality assessment. However, existing methods have problems such as incomplete feature extraction, partial image block distortion, and inability to determine scores. So in this paper, we propose a novel framework for image quality assessment based on deep learning. We incorporate both low-level visual features and high-level semantic features to better describe images. And image quality is analyzed in a parallel processing mode. Experiments are conducted on LIVE and TID2008 datasets demonstrate the proposed model can predict the quality of the distorted image well, and both SROCC and PLCC can reach 0.92 or higher.     

A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback.

A content-based image retrieval (CBIR) framework for diverse collection of medical images of different imaging modalities, anatomic regions with different orientations and biological systems is proposed. Organization of images in such a database (DB) is well defined with predefined semantic categories; hence, it can be useful for category-specific searching. The proposed framework consists of machine learning methods for image prefiltering, similarity matching using statistical distance measures, and a relevance feedback (RF) scheme. To narrow down the semantic gap and increase the retrieval efficiency, we investigate both supervised and unsupervised learning techniques to associate low-level global image features (e.g., color, texture, and edge) in the projected PCA-based eigenspace with their high-level semantic and visual categories. Specially, we explore the use of a probabilistic multiclass support vector machine (SVM) and fuzzy c-mean (FCM) clustering for categorization and prefiltering of images to reduce the search space. A category-specific statistical similarity matching is proposed in a finer level on the prefiltered images. To incorporate a better perception subjectivity, an RF mechanism is also added to update the query parameters dynamically and adjust the proposed matching functions. Experiments are based on a ground-truth DB consisting of 5000 diverse medical images of 20 predefined categories. Analysis of results based on cross-validation (CV) accuracy and precision-recall for image categorization and retrieval is reported. It demonstrates the improvement, effectiveness, and efficiency achieved by the proposed framework.     

A unified image retrieval framework on local visual and semantic concept-based feature spaces

This paper presents a learning-based unified image retrieval framework to represent images in local visual and semantic concept-based feature spaces. In this framework, a visual concept vocabulary (codebook) is automatically constructed by utilizing self-organizing map (SOM) and statistical models are built for local semantic concepts using probabilistic multi-class support vector machine (SVM). Based on these constructions, the images are represented in correlation and spatial relationship-enhanced concept feature spaces by exploiting the topology preserving local neighborhood structure of the codebook, local concept correlation statistics, and spatial relationships in individual encoded images. Finally, the features are unified by a dynamically weighted linear combination of similarity matching scheme based on the relevance feedback information. The feature weights are calculated by considering both the precision and the rank order information of the top retrieved relevant images of each representation, which adapts itself to individual searches to produce effective results. The experimental results on a photographic database of natural scenes and a bio-medical database of different imaging modalities and body parts demonstrate the effectiveness of the proposed framework.     

A Multi-Pronged Approach to Improving Semantic Extraction of News Video

In this paper we describe a multi-strategy approach to improving semantic extraction from news video. Experiments show the value of careful parameter tuning, exploiting multiple feature sets and multilingual linguistic resources, applying text retrieval approaches for image features, and establishing synergy between multiple concepts through undirected graphical models. We present a discriminative learning framework called Multi-concept Discriminative Random Field (MDRF) for building probabilistic models of video semantic concept detectors by incorporating related concepts as well as the low-level observations. The model exploits the power of discriminative graphical models to simultaneously capture the associations of concept with observed data and the interactions between related concepts. Compared with previous methods, this model not only captures the co-occurrence between concepts but also incorporates the raw data observations into a unified framework. We also describe an approximate parameter estimation algorithm and present results obtained from the TRECVID 2006 data. No single approach, however, provides a consistently better result for all concept detection tasks, which suggests that extracting video semantics should exploit multiple resources and techniques rather than naively relying on a single approach