基于深度学习的图像检索系统

基于内容的图像检索系统关键的技术是有效图像特征的获取和相似度匹配策略.在过去,基于内容的图像检索系统主要使用低级的可视化特征,无法得到满意的检索结果,所以尽管在基于内容的图像检索上花费了很大的努力,但是基于内容的图像检索依旧是计算机视觉领域中的一个挑战.在基于内容的图像检索系统中,存在的最大的问题是“语义鸿沟”,即机器从低级的可视化特征得到的相似性和人从高级的语义特征得到的相似性之间的不同.传统的基于内容的图像检索系统,只是在低级的可视化特征上学习图像的特征,无法有效的解决“语义鸿沟”.近些年,深度学习技术的快速发展给我们提供了希望.深度学习源于人工神经网络的研究,深度学习通过组合低级的特征形成更加抽象的高层表示属性类别或者特征,以发现数据的分布规律,这是其他算法无法实现的.受深度学习在计算机视觉、语音识别、自然语言处理、图像与视频分析、多媒体等诸多领域取得巨大成功的启发,本文将深度学习技术用于基于内容的图像检索,以解决基于内容的图像检索系统中的“语义鸿沟”问题.     

Learning Pseudo Metric for Intelligent Multimedia Data Classification and Retrieval

While people compare images using semantic concepts, computers compare images using low-level visual features that sometimes have little to do with these semantics. To reduce the gap between the high-level semantics of visual objects and the low-level features extracted from them, in this paper we develop a framework of learning pseudo metrics (LPM) using neural networks for semantic image classification and retrieval. Performance analysis and comparative studies, by experimenting on an image database, show that the LPM has potential application to multimedia information processing.     

基于特征互补率矩阵的图像分类方法

在基于内容的图像检索和分类系统中,图像的底层特征和高层语义之间存在着语义鸿沟,有效减小语义鸿沟是一个需要广泛研究的问题。为此,提出一种基于特征互补率矩阵的图像分类方法,该方法通过计算视觉特征互补率矩阵进而指导融合特征集的选择,利用测度学习算法得到一个合适的距离测度以反映图像高层语义的相似度。实验结果表明,该方法能有效提高图像分类精度。     

Region-based image retrieval with high-level semantics using decision tree learning

Semantic-based image retrieval has attracted great interest in recent years. This paper proposes a region-based image retrieval system with high-level semantic learning. The key features of the system are: (1) it supports both query by keyword and query by region of interest. The system segments an image into different regions and extracts low-level features of each region. From these features, high-level concepts are obtained using a proposed decision tree-based learning algorithm named DT-ST. During retrieval, a set of images whose semantic concept matches the query is returned. Experiments on a standard real-world image database confirm that the proposed system significantly improves the retrieval performance, compared with a conventional content-based image retrieval system. (2) The proposed decision tree induction method DT-ST for image semantic learning is different from other decision tree induction algorithms in that it makes use of the semantic templates to discretize continuous-valued region features and avoids the difficult image feature discretization problem. Furthermore, it introduces a hybrid tree simplification method to handle the noise and tree fragmentation problems, thereby improving the classification performance of the tree. Experimental results indicate that DT-ST outperforms two well-established decision tree induction algorithms ID3 and C4.5 in image semantic learning.     

轮胎花纹图像检索技术综述

轮胎花纹图像检索在交通事故处理及刑事案件侦破中是获取破案信息的重要手段,虽然基于内容的图像检索技术已发展数十年,但由于轮胎花纹图像数据的来源及应用场景特殊等因素,目前这方面的研究文献并不多。在研究近年来轮胎花纹图像检索领域相关文献的基础上,对该领域的技术现状进行总结分析。首先,围绕轮胎花纹纹理特征提取和高层语义特征提取两项关键技术描述了该领域的主要研究成果,并总结了轮胎花纹数据库以及检索性能评价指标。然后,分别针对轮胎花纹低层特征和高层特征提取进行实验对比并分析结果。最后,结合现有技术及实际应用需求,分析了该领域的技术发展趋势并指出了未来的研究方向。     

基于目标区域的语义图像检索

提出了一种基于高层语义的图像检索方法,该方法首先将图像分割成区域,提取每个区域的颜色、形状、位置特征,然后使用这些特征对图像对象进行聚类,得到每幅图像的语义特征向量;采用模糊C均值算法对图像进行聚类,在图像检索时,查询图像和聚类中心比较,然后在距离最小的类中进行检索。实验表明,提出的方法可以明显提高检索效率,缩小低层特征和高层语义之间的“语义鸿沟”。     

Visual Ontology Construction for Digitized Art Image Retrieval

Current investigations on visual information retrieval are generally content-based methods. The significant difference between similarity in low-level features and similarity in high-level semantic meanings is still a major challenge in the area of image retrieval. In this work, a scheme for constructing visual ontology to retrieve art images is proposed. The proposed ontology describes images in various aspects, including type ＆ style, objects and global perceptual effects. Concepts in the ontology could be automatically derived. Various art image classification methods are employed based on low-level image features. Non-objective semantics are introduced, and how to express these semantics is given. The proposed ontology scheme could make users more naturally find visual information and thus narrows the “semantic gap”. Experimental implementation demonstrates its good potential for retrieving art images in a human-centered manner.     

基于SVR分类的多特征彩色图像检索算法

传统的基于内容图像检索技术对图像领域没有限制,而宽泛的图像领域不仅严重影响了系统检索精度,而且增加了系统查询时间.本文提出一种基于SVR(Support Vector Regression)分类的多特征彩色图像检索新算法,该算法首先提取出图像的颜色、空间和纹理信息并作为图像的特征向量,然后以回归型支持向量机(SVR)为学习机器,对图像库进行分类处理以缩小图像领域范围,最后在较小的领域范围内进行图像检索.实验结果表明,本文算法能够准确和高效地查找出用户所需内容的彩色图像,并且具有较好的查准率和查全率.     

基于SVM的图像低层特征与高层语义的关联

在基于内容的图像检索中，针对图像的低层可视特征与高层语义特征之间的鸿沟，提出了一种基于支持向量机（SVM）的语义关联方法。通过对图像低层特征的分析，提取了颜色和形状特征向量（221维），将它们作为支持向量机的输入向量，对图像类进行学习，建立图像低层特征与高层语义的关联，并应用于鸟类、花卉、海洋以及建筑物等几个典型的语义类别检索。实验结果表明，该方法可适应于不同用户的图像检索，并提高了检索性能。     

Wildlife video key-frame extraction based on novelty detection in semantic context

There is a growing evidence that visual saliency can be better modeled using top-down mechanisms that incorporate object semantics. This suggests a new direction for image and video analysis, where semantics extraction can be effectively utilized to improve video summarization, indexing and retrieval. This paper presents a framework that models semantic contexts for key-frame extraction. Semantic context of video frames is extracted and its sequential changes are monitored so that significant novelties are located using a one-class classifier. Working with wildlife video frames, the framework undergoes image segmentation, feature extraction and matching of image blocks, and then a co-occurrence matrix of semantic labels is constructed to represent the semantic context within the scene. Experiments show that our approach using high-level semantic modeling achieves better key-frame extraction as compared with its counterparts using low-level features.     

基于纹理语义特征的图像检索研究

针对Tamura纹理模型，提出了基于语言变量的图像纹理语义特征描述方法；并通过遗传程序设计构造从低层视觉特征到高层语义特征的映射；最后根据这些模糊语义值进行图像检索．实验结果表明系统不仅能得到出众的检索效率，而且与人类的视知觉具有比较好的一致性，提出的方法对于缩小低层视觉特征和高层语义特征之间的“语义鸿沟”具有很大的意义．     

图像检索中基于粗集理论的特征加权方法

为了提高图像检索的效率，近年来相关反馈机制被引入到基于内容的图像检索领域，而在基于内容的图像检索系统中，多特征融合检索中的特征加权又是一个重要的问题。该文提出了一种新的基于特征加权的相关反馈方法，在粗集理论的基础上，结合用户标记的反馈图像建立决策表，通过决策规则的精度来对多个特征加权，使图像检索和人的感知更加接近。实验表明该方法是有效的，并较Rui的相关反馈方法在性能上有很大提高。     

Extracting semantic concepts from images: a decisive feature pattern mining approach

One major challenge in the content-based image retrieval (CBIR) and computer vision research is to bridge the so-called “semantic gap” between low-level visual features and high-level semantic concepts, that is, extracting semantic concepts from a large database of images effectively. In this paper, we tackle the problem by mining the decisive feature patterns (DFPs). Intuitively, a decisive feature pattern is a combination of low-level feature values that are unique and significant for describing a semantic concept. Interesting algorithms are developed to mine the decisive feature patterns and construct a rule base to automatically recognize semantic concepts in images. A systematic performance study on large image databases containing many semantic concepts shows that our method is more effective than some previously proposed methods. Importantly, our method can be generally applied to any domain of semantic concepts and low-level features. Wei Wang received his Ph.D. degree in Computing Science and Engineering from the State University of New York (SUNY) at Buffalo in 2004, under Dr. Aidong Zhang's supervision. He received the B.Eng. in Electrical Engineering from Xi'an Jiaotong University, China in 1995 and the M.Eng. in Computer Engineering from National University of Singapore in 2000, respectively. He joined Motorola Inc. in 2004, where he is currently a senior research engineer in Multimedia Research Lab, Motorola Applications Research Center. His research interests can be summarized as developing novel techniques for multimedia data analysis applications. He is particularly interested in multimedia information retrieval, multimedia mining and association, multimedia database systems, multimedia processing and pattern recognition. He has published 15 research papers in refereed journals, conferences, and workshops, has served in the organization committees and the program committees of IADIS International Conference e-Society 2005 and 2006, and has been a reviewer for some leading academic journals and conferences. In 2005, his research prototype of “seamless content consumption” was awarded the “most innovative research concept of the year” from the Motorola Applications Research Center. Dr. Aidong Zhang received her Ph.D. degree in computer science from Purdue University, West Lafayette, Indiana, in 1994. She was an assistant professor from 1994 to 1999, an associate professor from 1999 to 2002, and has been a professor since 2002 in the Department of Computer Science and Engineering at the State University of New York at Buffalo. Her research interests include bioinformatics, data mining, multimedia systems, content-based image retrieval, and database systems. She has authored over 150 research publications in these areas. Dr. Zhang's research has been funded by NSF, NIH, NIMA, and Xerox. Dr. Zhang serves on the editorial boards of International Journal of Bioinformatics Research and Applications (IJBRA), ACMMultimedia Systems, the International Journal of Multimedia Tools and Applications, and International Journal of Distributed and Parallel Databases. She was the editor for ACM SIGMOD DiSC (Digital Symposium Collection) from 2001 to 2003. She was co-chair of the technical program committee for ACM Multimedia 2001. She has also served on various conference program committees. Dr. Zhang is a recipient of the National Science Foundation CAREER Award and SUNY Chancellor's Research Recognition Award.     

Linking Visual Cues and Semantic Terms Under Specific Digital Video Domains

Semantic retrieval from video databases is becoming a very important research topic in the area of multimedia. This kind of tasks require the development of video data representation models which include the relationships between low-level visual cues and the semantic concepts inferred from them. This paper presents a work based on semiotic studies that includes the extraction of simple visual features from commercials and a statistical analysis of them and their relationships with high-level semantic terms. Well-known algorithms have been implemented and enhanced for feature extraction, as well as a novel probabilistic approach to color naming. The statistical analysis consists of finding correlations between variables, as well as the dimensions in feature space that best explain the variance of the data set. Some interesting conclusions are reached at the end of the work about how commercials are grouped in feature space with respect to different levels of semantics.     

基于支持向量机的图像语义分类

图像的低层可视特征与高层语义特征之间存在着一道鸿沟,人们不能直接理解由计算机自动生成的低层特征.另外,基于内容的图像分类和检索的性能极大地依赖于可视特征的提取和描述.出于这些考虑,提出了新的图像纹理、边缘描述子提取方法,并将它们表示为直方图.在此基础上,集成纹理、边缘和颜色直方图作为图像的特征向量,用支持向量机(SVM)实现图像的语义分类.实验结果表明,集成的图像特征表示在图像分类实验中取得了很好的效果,具有比其他特征表示(如Gabor纹理、颜色直方图)更好的性能.     

基于多语义特征的彩色图像检索技术研究

基于语义内容的图像检索已成为解决图像低层特征与人类高级语义之间"语义鸿沟"的关键.以性能优越的回归型支持向量机(SVR)理论为基础,结合重要的图像边缘信息及人眼视觉特性,提出了一种基于多语义特征的彩色图像检索新算法.该算法首先利用Canny检测算子提取原始图像的边缘信息,并得到低层纹理特征,同时利用SVR将低层特征映射到高级语义,以获得图像的高级纹理语义.然后结合人眼视觉系统感知特性,给出基于重要区域主要颜色的高级颜色语义.最后根据上述高级语义特征(纹理语义和颜色语义)进行图像检索.实验结果表明,该算法能够有效地对图像高级语义进行刻画,不仅图像匹配检索效果良好,而且具有稳定的检索性能,其对于缩小低层视觉特征与高级语义概念之间的"语义鸿沟"具有重要意义.     

图像语义自动标注及其粒度分析方法

缩小图像低层视觉特征与高层语义之间的鸿沟, 以提高图像语义自动标注的精度, 进而快速满足用户检索图像的需求,一直是图像语义自动标注研究的关键. 粒度分析方法是一种层次的、重要的数据分析方法, 为复杂问题的求解提供了新的思路. 图像理解与分析的粒度不同, 图像语义标注的精度则不同, 检索的效率及准确度也就不同. 本文对目前图像语义自动标注模型的方法进行综述和分析, 阐述了粒度分析方法的思想、模型及其在图像语义标注过程中的应用, 探索了以粒度分析为基础的图像语义自动标注方法并给出进一步的研究方向.