基于融合特征的现勘图像检索结果优化算法

刑侦现勘图像数据库是具有保密性高、图像内容罕见等极具行业特色的图像数据库.针对现勘图像内容复杂、目标物体不明确的特点,提出了DCT-DCT波纹理特征,并与HSV颜色直方图特征、GIST特征相融合构成融合特征.与常用的图像特征相比,DCT-DCT波纹理特征能够得到较高的检索效率,而融合特征的平均检索查准率高于构成其本身的三种特征的平均检索查准率.最后,将语义分析技术引入到检索过程中,提出基于检索结果优化的现勘图像检索算法,利用支持向量机（Support Vector Machine,SVM）分类器对查询图像进行语义提取,并对初次检索的结果进行语义分析,根据初检结果中语义类别的占比选择二次检索方案,该算法能在按例查询的基础上进一步提高平均检索查准率.     

现勘图像检索综述

现勘图像检索是进行证据图像比对以获取物证信息的重要手段.本文基于目前应用广泛的现勘图像数据库,根据图像内容将图像分为鞋印、指纹、纹身等种类.并通过对现勘图像的两项关键技术即低层数字特征提取和高层语义分析的总结,从颜色特征、纹理特征、边缘提取等方面综述了现勘图像低层数字特征提取技术,从利用语义模板和数据库本体结构、机器学习算法、引入人工反馈三大类高层语义提取技术综述了现勘图像高层语义分析的研究成果.最后,结合公安行业利用现勘图像获取物证线索的实际应用需求,指出了通过引入公安行业先验知识来提高检索效率等研究方向.     

Structural image retrieval using automatic image annotation and region based inverted file

Image retrieval has lagged far behind text retrieval despite more than two decades of intensive research effort. Most of the research on image retrieval in the last two decades are on content based image retrieval or image retrieval based on low level features. Recent research in this area focuses on semantic image retrieval using automatic image annotation. Most semantic image retrieval techniques in literature, however, treat an image as a bag of features/words while ignore the structural or spatial information in the image. In this paper, we propose a structural image retrieval method based on automatic image annotation and region based inverted file. In the proposed system, regions in an image are treated the same way as keywords in a structural text document, semantic concepts are learnt from image data to label image regions as keywords and weight is assigned to each keyword according to spatial position and relationship. As the result, images are indexed and retrieved in the same way as structural document retrieval. Specifically, images are broken down to regions which are represented using colour, texture and shape features. Region features are then quantized to create visual dictionaries which are similar to monolingual dictionaries like English or Chinese dictionaries. In the next step, a semantic dictionary similar to a bilingual dictionary like the English–Chinese dictionary is learnt to mapping image regions to semantic concepts. Finally, images are then indexed and retrieved using a novel region based inverted file data structure. Results show the proposed method has significant advantage over the widely used Bayesian annotation models.     

图像的情感语义研究进展

人对图像的感知和理解是主观性的。图像的情感语义研究的目的是希望计算机能够描述和表达人观察图像所引起的情感反应，使用带有主观感情色彩的语义表述图像，其结果可用于图像的情感评估、图像检索、美术和广告设计等方面，是一个崭新的且有很大挑战性前沿课题。本文从图像情感语义提取的结构框架出发，对图像的情感语义研究进展进行评述，包括了情感语义提取过程中的关键技术：图像特征提取、情感空间的选取与建立、建立图像特征到情感空间的映射等方面，并对国内外的研究现状进行跟踪研究。     

Audio Feature Extraction and Analysis for Scene Segmentation and Classification

Understanding of the scene content of a video sequence is very important for content-based indexing and retrieval of multimedia databases. Research in this area in the past several years has focused on the use of speech recognition and image analysis techniques. As a complimentary effort to the prior work, we have focused on using the associated audio information (mainly the nonspeech portion) for video scene analysis. As an example, we consider the problem of discriminating five types of TV programs, namely commercials, basketball games, football games, news reports, and weather forecasts. A set of low-level audio features are proposed for characterizing semantic contents of short audio clips. The linear separability of different classes under the proposed feature space is examined using a clustering analysis. The effective features are identified by evaluating the intracluster and intercluster scattering matrices of the feature space. Using these features, a neural net classifier was successful in separating the above five types of TV programs. By evaluating the changes between the feature vectors of adjacent clips, we also can identify scene breaks in an audio sequence quite accurately. These results demonstrate the capability of the proposed audio features for characterizing the semantic content of an audio sequence.     

图像语义自描述性的实现方法研究

根据语义特征进行检索是多媒体检索技术的发展趋势.图像是最重要的媒体之一,本文提出了一种实现图像语义自描述性的方法.通过在JPEG文件中嵌入描述图像语义的XML文件,使计算机可以对图像进行语义级的检索.实验表明,该方法创建的语义描述可以从图像文件中读取,具有标准化、结构性和可读性的特点,尤其适用于在Internet上检索图像.该方法对于提高图像信息的利用率具有重要的意义.     

Histological image retrieval based on semantic content analysis

The demand for automatic recognition and retrieval of medical images for screening, reference, and management is increasing. We present an intelligent content-based image retrieval system called I-Browse, which integrates both iconic and semantic content for histological image analysis. The I-Browse system combines low-level image processing technology with high-level semantic analysis of medical image content through different processing modules in the proposed system architecture. Similarity measures are proposed and their performance is evaluated. Furthermore, as a byproduct of semantic analysis, I-Browse allows textual annotations to be generated for unknown images. As an image browser, apart from retrieving images by image example, it also supports query by natural language.     

Depth estimation of supervised monocular images based on semantic segmentation

In recent years, the research method of depth estimation of target images using Convolutional Neural Networks (CNN) has been widely recognized in the fields of artificial intelligence, scene understanding and three-dimensional (3D) reconstruction. The fusion of semantic segmentation information and depth estimation will further improve the quality of acquired depth images. However, how to deeply combine image semantic information with image depth information and use image edge information more accurately to improve the accuracy of depth image is still an urgent problem to be solved. For this purpose, we propose a novel depth estimation model based on semantic segmentation to estimate the depth of monocular images in this paper. Firstly, a shared parameter model of semantic segmentation information and depth estimation information is built, and the semantic segmentation information is used to guide depth acquisition in an auxiliary way. Then, through the multi-scale feature fusion module, the feature information contained in the neural network on different layers is fused, and the local feature information and global feature information are effectively used to generate high-resolution feature maps, so as to achieve the goal of improving the quality of depth image by optimizing the semantic segmentation model. The experimental results show that the model can fully extract and combine the image feature information, which improves the quality of monocular depth vision estimation. Compared with other advanced models, our model has certain advantages.     

基于中文自然语言的纹理分类新方法

提出了一种新的基于中文自然语言纹理描述词的纹理分类方法,建立了自然纹理分类体系,并用最小二乘支持向量机对纹理进行分类,实现了纹理的视觉特征到语义描述的转换.实验结果证明,该方法在图像理解和基于内容的图像检索中有助于缩小纹理特征的数学描述和人类理解之间的"语义鸿沟".     

基于语义分割的双目场景流估计

针对现有场景流计算方法在复杂场景、大位移和运动遮挡等情况下易产生运动边缘模糊的问题,提出一种基于语义分割的双目场景流估计方法.首先,根据图像中的语义信息类别,通过深度学习的卷积神经网络模型将图像划分为带有语义标签的区域;针对不同语义类别的图像区域分别进行运动建模,利用语义知识计算光流信息并通过双目立体匹配的半全局匹配方法计算图像视差信息.然后,对输入图像进行超像素分割,通过最小二乘法耦合光流和视差信息,分别求解每个超像素块的运动参数.最后,在优化能量函数中添加语义分割边界的约束信息,通过更新像素到超像素块的映射关系和超像素块到移动平面的映射关系得到最终的场景流估计结果.采用KITTI 2015标准测试图像序列对本文方法和代表性的场景流计算方法进行对比分析.实验结果表明,本文方法具有较高的精度和鲁棒性,尤其对于复杂场景、运动遮挡和运动边缘模糊的图像具有较好的边缘保护作用.     

基于视觉感知的图像检索的研究

基于内容图像检索的一个突出问题是图像低层特征与高层语义之间存在的巨大鸿沟.针对相关反馈和感兴趣区检测在弥补语义鸿沟时存在主观性强、耗时的缺点,提出了视觉信息是一种客观反映图像高层语义的新特征,基于视觉信息进行图像检索可以有效减小语义鸿沟;并在总结视觉感知的研究进展和实现方法的基础上,给出了基于视觉感知的图像检索在感兴趣区检测、图像分割、相关反馈和个性化检索四个方面的研究思路.     

基于语义的视频检索关键技术综述

随着大量视频的出现,视频内容检索是当今多媒体应用的一个重要研究方向。现有的视频检索技术多是基于低层特征,这些低层特征与高层语义概念相差较多,严重影响了视频内容检索系统的实用性。由于低层特征和高层语义概念间的语义鸿沟,如何从视频内容中提取人类思维中的语义概念,正成为目前视频内容检索中最具有挑战性的研究内容。文中介绍了语义视频检索出现的背景和国内外最新研究动态,分析了现有方法的优缺点,对现有的关键技术进行综述。     

Perceptually based techniques for image segmentation and semantic classification

We present a new approach for semantic image analysis that combines knowledge of human perception with an understanding of signal characteristics to segment natural scenes into perceptually uniform regions, and then uses the region statistics to extract semantic information. Applications include content-based image retrieval and region of interest extraction for efficient compression/transmission over heterogeneous networks     

Clustering of hierarchical image database to reduce inter-and intra-semantic gaps in visual space for finding specific image semantics

Empowering content based systems to assign image semantics is an interesting concept. This work explores semantically categorized image database and forms a hierarchical visual search space. Overlapping of visual features of images from different categories and subcategories are possible reasons behind inter-semantic and intra-semantic gaps. Usually each category/node in the image database has a single representation, but variability and broadness of semantic limit the usage of such representation. This work explores the application of agglomerative hierarchical clustering to automatically identify groups within a semantic in the visual space. Visual signatures of dominant clusters corresponding to a node represent its semantic. Adaptive selection of branches on this clustered data facilitates efficient semantic assignment to query image in reduced search cost. Based on the concept, content based semantic retrieval system is developed and tested on hierarchical and non-hierarchical databases. Results showcase capability of the proposed system to reduce inter- and intra-semantic gaps.     

An Indexing and Retrieval Mechanism for Complex Similarity Queries in Image Databases

A content-based image retrieval mechanism to support complex similarity queries is presented. The image content is defined by three kinds of features: quantifiable features describing the visual information, nonquantifiable features describing the semantic information, and keywords describing more abstract semantic information. In correspondence with these feature sets, we construct three types of indexes: visual indexes, semantic indexes, and keyword indexes. Index structures are elaborated to provide effective and efficient retrieval of images based on their contents. The underlying index structure used for all indexes is the HG-tree. In addition to the HG-tree, the signature file and hashing technique are also employed to index keywords and semantic features. The proposed indexing scheme combines and extends the HG-tree, the signature file, and the hashing scheme to support complex similarity queries. We also propose a new evaluation strategy to process the complex similarity queries. Experiments have been carried out on large image collections to demonstrate the effectiveness of the proposed retrieval mechanism.     

A CBIR system based on saliency driven local image features and multi orientation texture features

In Content-based Image Retrieval (CBIR), the user provides the query image in which only a selective portion of the image carries the foremost vital information known as the object region of the image. However, the human visual system also focuses on a particular salient region of an image to instinctively understand its semantic meaning. Therefore, the human visual attention technique can be well imposed in the CBIR scheme. Inspired by these facts, we initially utilized the signature saliency map-based approach to decompose the image into its respective main object region (ObR) and non-object region (NObR). ObR possesses most of the vital image information, so block-level normalized singular value decomposition (SVD) has been used to extract salient features of the ObR. In most natural images, NObR plays a significant role in understanding the actual semantic meaning of the image. Accordingly, multi-directional texture features have been extracted from NObR using Gabor filter on different wavelengths. Since the importance of ObR and NObR features are not equal, a new homogeneity-based similarity matching approach has been devised to enhance retrieval accuracy. Finally, we have demonstrated retrieval performances using both the combined and distinct ObR and NObR features on seven standard coral, texture, object, and heterogeneous datasets. The experimental outcomes show that the proposed CBIR system has a promising retrieval efficiency and outperforms various existing systems substantially.     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.