基于视觉语义主题与反馈日志的图像自动标注

为了提高图像标注性能,提出了一种基于视觉语义主题与反馈日志的图像自动标注方法。首 先,提取图像 前景与背景区域,分别进行处理;其次,基于WordNet构建标注词之间的语义关系模型,并 结合概率潜在语义分析(PLSA) 与高斯混合模型(GMM)建立图像底层特征、视觉语义主题与标注     

基于Matlab的图像自动标注

图像自动标注在检索大量数字图像时起到关键作用,它能将图像的视觉特征转化为图像的标注字信息,为用户的使用及检索带来极大的方便。研究了图像自动语义标注方法,设计并实现了基于Matlab图像自动标注系统,能够提取图像颜色特征和纹理特征,与已标注图像进行相似性度量并标注出图像语义关键词     

基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注

图像属性标注是一种更细化的图像标注，它能缩小认知与特征间"语义鸿沟".现有研究多基于单特征且未挖掘属性蕴含的深层语义，故无法准确刻画图像内容.改进有效区域基因选择算法融合图像特征，并设计迁移学习策略，实现材质属性标注；基于判别相关分析挖掘特征间跨模态语义，以改进相对属性模型，标注材质属性蕴含的深层语义-实用属性.实验表明：材质属性标注精准度达63.11%，较最强基线提升1.97%；实用属性标注精准度达59.15%，较最强基线提升2.85%；层次化的标注结果能全面刻画图像内容.     

基于图正则化与非负组稀疏的自动图像标注

设计一个稳健的自动图像标注系统的重要环节是提取能够有效描述图像语义的视觉特征。由于颜色、纹理和形状等异构视觉特征在表示特定图像语义时所起作用的重要程度不同且同一类特征之间具有一定的相关性,该文提出了一种图正则化约束下的非负组稀疏(Graph Regularized Non-negative Group Sparsity, GRNGS)模型来实现图像标注,并通过一种非负矩阵分解方法来计算其模型参数。该模型结合了图正则化与l2,1-范数约束,使得标注过程中所选的组群特征能体现一定的视觉相似性和语义相关性。在Corel5K和ESP Game等图像数据集上的实验结果表明:相较于一些最新的图像标注模型,GRNGS模型的鲁棒性更强,标注结果更精确。     

基于潜在主题融合的跨媒体图像语义标注

图像语义标注是图像语义分析研究中的一个重要问题.在主题模型的基础上,本文提出一种新颖的跨媒体图像标注方法来进行图像间语义的传播.首先,对训练图像使用主题模型,抽取视觉模态和文本模态信息的潜在语义主题.然后,通过使用一个权重参数来融合两种模态信息的主题分布,从而学习到一种融合主题分布.最后,在融合主题分布的基础上训练一个标注模型来给目标图像赋予合适的语义信息.在标准的MSRC和Corel5K数据集上将提出的方法与最近著名的标注方法进行比较实验.标注性能的详细评价结果表明提出方法的有效性.     

基于二次表示的空间目标图像分类

针对空间目标图像的特点,该文提出一种基于局部不变特征的空间目标图像分类方法.该方法首先提取每幅图像的局部不变特征,利用混合高斯模型(GMM)建立全局的视觉模式,然后依据最大后验概率匹配局部特征和视觉模式来构造整个训练集图像的共现矩阵,采用概率潜在语义分析(PLSA)模型得到图像的潜在类别表示来实现图像的二次表示,最后利用SVM算法实现分类.实验结果验证了该方案的有效性.     

基于最大熵模型的语义角色标注

提出了基于最大熵模型的语义角色标注方法,该方法以浅层句法分析为基础,把短语或命名实体作为标注的基本单元,将最大熵模型用于句子中谓词的语义角色标注.该方法的关键在于模型参数估计和特征选择.具体应用中采用IIS算法学习模型参数,并选择基于句法成分的、基于谓词的、句法成分一谓词关系、语义四类特征作为模型特征集.将该方法用于信息抽取中事件表述语句的语义角色标注,对"职务变动"和"会见"两类事件的表述语句进行事件要素的语义角色标注,在各自的测试集上分别获得了76.3%和72.2%的综合指标F值.     

面向自然场景分类的贝叶斯网络局部语义建模方法

本文提出了一种基于贝叶斯网络的局部语义建模方法.网络结构涵盖了区域邻域的方向特性和区域语义之间的邻接关系.基于这种局部语义模型,建立了场景图像的语义表述,实现自然场景分类.通过对已标注集的图像样本集的学习训练,获得贝叶斯刚络的参数.对于待分类的图像,利用该模型融合区域的特征及其邻接区域的信息,推理得到区域的语义概率;并通过网络迭代收敛得到整幅图像的区域语义标记和语义概率;最后在此基础上形成图像的全局描述,实现场景分类.该方法利用了场景内部对象之间的上下文关系,弥补了仅利用底层特征进行局部语义建模的不足.通过在六类自然场景图像数据集上的实验表明,本文所提的局部语义建模和图像描述方法是有效的.     

基于GAN的ISAR图像语义分割方法

逆合成孔径雷达（ISAR）成像技术能够对空间目标进行远距离成像,刻画目标的外形、结构和尺寸等信息。ISAR图像语义分割能够获取目标的感兴趣区域,是ISAR图像解译的重要技术支撑,具有非常重要的研究价值。由于ISAR图像表征性较差,图像中散射点的不连续和强散射点存在的旁瓣效应使得人工精准标注十分困难,基于交叉熵损失的传统深度学习语义分割方法在语义标注不精准情况下无法保证分割性能的稳健。针对这一问题,提出了一种基于生成对抗网络（GAN）的ISAR图像语义分割方法,采用对抗学习思想学习ISAR图像分布到其语义分割图像分布的映射关系,同时通过构建分割图像的局部信息和全局信息来保证语义分割的精度。基于仿真卫星目标ISAR图像数据集的实验结果证明,本文方法能够取得较好的语义分割结果,且在语义标注不够精准的情况下模型更稳健。     

基于偏序结构的图像标注

图像标注旨在为图像分配一系列的语义标签描述图像的内容。针对高级语义与低级特征之间的语义鸿沟问题,本文提出了基于偏序结构的图像标注方法。首先,计算训练图像与测试图像的相似性得分,得到测试图像的初始邻近集及邻近标签;然后通过构建的属性偏序结构,获得邻近标签的相关语义,提高标签的丰富度,以及利用构建的对象偏序结构,得到最终的候选集。为了提高标注的准确率,设置一个频率阈值筛选出频率较高的标签作为最终的关键词。通过实验证明,实验结果有效地提高了标注的准确率和召回率。      

A unified image retrieval framework on local visual and semantic concept-based feature spaces

This paper presents a learning-based unified image retrieval framework to represent images in local visual and semantic concept-based feature spaces. In this framework, a visual concept vocabulary (codebook) is automatically constructed by utilizing self-organizing map (SOM) and statistical models are built for local semantic concepts using probabilistic multi-class support vector machine (SVM). Based on these constructions, the images are represented in correlation and spatial relationship-enhanced concept feature spaces by exploiting the topology preserving local neighborhood structure of the codebook, local concept correlation statistics, and spatial relationships in individual encoded images. Finally, the features are unified by a dynamically weighted linear combination of similarity matching scheme based on the relevance feedback information. The feature weights are calculated by considering both the precision and the rank order information of the top retrieved relevant images of each representation, which adapts itself to individual searches to produce effective results. The experimental results on a photographic database of natural scenes and a bio-medical database of different imaging modalities and body parts demonstrate the effectiveness of the proposed framework.     

一种基于视觉词袋模型的人脸识别方法

针对人脸图像因受表情、光照、角度等因素影响,导致人脸识别率较低的状况,提出了一种基于视觉词袋模型的人脸识别方法。该方法首先对图像进行分块并提取局部特征,其次利用训练样本的所有局部特征训练全局的混合高斯模型,然后以此为初始化训练单张图像的混合高斯模型,生成该图像全局特征向量,最后用PLDA进行人脸识别。通过在LFW数据库上进行实验,结果显示本方法的识别率高于传统的特征提取方法,证明了本方法具有更强的识别性能。     

Image annotation based on multi-view robust spectral clustering

Nowadays, image annotation has been a hot topic in the semantic retrieval field due to the abundant growth of digital images. The purpose of these methods is to realize the content of images and assign appropriate keywords to them. Extensive efforts have been conducted in this field, which effectiveness is limited between low-level image features and high-level semantic concepts. In this paper, we propose a Multi-View Robust Spectral Clustering (MVRSC) method, which tries to model the relationship between semantic and multi-features of training images based on the Maximum Correntropy Criterion. A Half-Quadratic optimization framework is used to solve the objective function. According to the constructed model, a few tags are suggested based on a novel decision-level fusion distance. The stability condition and bound calculation of MVRSC are analyzed, as well. Experimental results on real-world Flickr and 500PX datasets, and Corel5K confirm the superiority of the proposed method over other competing models.     

Modeling,classifying and annotating weakly annotated images using Bayesian network

In this paper, we propose a probabilistic graphical model to represent weakly annotated images. We consider an image as weakly annotated if the number of keywords defined for it is less than the maximum number defined in the ground truth. This model is used to classify images and automatically extend existing annotations to new images by taking into account semantic relations between keywords. The proposed method has been evaluated in visual-textual classification and automatic annotation of images. The visual-textual classification is performed by using both visual and textual information. The experimental results, obtained from a database of more than 30,000 images, show an improvement by 50.5% in terms of recognition rate against only visual information classification. Taking into account semantic relations between keywords improves the recognition rate by 10.5%. Moreover, the proposed model can be used to extend existing annotations to weakly annotated images, by computing distributions of missing keywords. Semantic relations improve the mean rate of good annotations by 6.9%. Finally, the proposed method is competitive with a state-of-art model.     

An Indexing and Retrieval Mechanism for Complex Similarity Queries in Image Databases

A content-based image retrieval mechanism to support complex similarity queries is presented. The image content is defined by three kinds of features: quantifiable features describing the visual information, nonquantifiable features describing the semantic information, and keywords describing more abstract semantic information. In correspondence with these feature sets, we construct three types of indexes: visual indexes, semantic indexes, and keyword indexes. Index structures are elaborated to provide effective and efficient retrieval of images based on their contents. The underlying index structure used for all indexes is the HG-tree. In addition to the HG-tree, the signature file and hashing technique are also employed to index keywords and semantic features. The proposed indexing scheme combines and extends the HG-tree, the signature file, and the hashing scheme to support complex similarity queries. We also propose a new evaluation strategy to process the complex similarity queries. Experiments have been carried out on large image collections to demonstrate the effectiveness of the proposed retrieval mechanism.     

Automatic image annotation based on Gaussian mixture model considering cross-modal correlations

Automatic image annotation has been an active topic of research in the field of computer vision and pattern recognition for decades. In this paper, we present a new method for automatic image annotation based on Gaussian mixture model (GMM) considering cross-modal correlations. To be specific, we first employ GMM fitted by the rival penalized expectation-maximization (RPEM) algorithm to estimate the posterior probabilities of each annotation keyword. Next, a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity by seamlessly integrating the information from both image low level visual features and high level semantic concepts together, which can effectively avoid the phenomenon that different images with the same candidate annotations would obtain the same refinement results. Followed by the rank-two relaxation heuristics over the built label similarity graph is applied to further mine the correlation of the candidate annotations so as to capture the refining annotation results, which plays a crucial role in the semantic based image retrieval. The main contributions of this work can be summarized as follows: (1) Exploiting GMM that is trained by the RPEM algorithm to capture the initial semantic annotations of images. (2) The label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels. (3) Refining the candidate set of annotations generated by the GMM through solving the max-bisection based on the rank-two relaxation algorithm over the weighted label graph. Compared to the current competitive model SGMM-RW, we can achieve significant improvements of 4% and 5% in precision, 6% and 9% in recall on the Corel5k and Mirflickr25k, respectively.     

Selecting salient frames for spatiotemporal video modeling and segmentation.

We propose a new statistical generative model for spatiotemporal video segmentation. The objective is to partition a video sequence into homogeneous segments that can be used as "building blocks" for semantic video segmentation. The baseline framework is a Gaussian mixture model (GMM)-based video modeling approach that involves a six-dimensional spatiotemporal feature space. Specifically, we introduce the concept of frame saliency to quantify the relevancy of a video frame to the GMM-based spatiotemporal video modeling. This helps us use a small set of salient frames to facilitate the model training by reducing data redundancy and irrelevance. A modified expectation maximization algorithm is developed for simultaneous GMM training and frame saliency estimation, and the frames with the highest saliency values are extracted to refine the GMM estimation for video segmentation. Moreover, it is interesting to find that frame saliency can imply some object behaviors. This makes the proposed method also applicable to other frame-related video analysis tasks, such as key-frame extraction, video skimming, etc. Experiments on real videos demonstrate the effectiveness and efficiency of the proposed method.     

A unified framework for image retrieval using keyword and visual features.

In this paper, a unified image retrieval framework based on both keyword annotations and visual features is proposed. In this framework, a set of statistical models are built based on visual features of a small set of manually labeled images to represent semantic concepts and used to propagate keywords to other unlabeled images. These models are updated periodically when more images implicitly labeled by users become available through relevance feedback. In this sense, the keyword models serve the function of accumulation and memorization of knowledge learned from user-provided relevance feedback. Furthermore, two sets of effective and efficient similarity measures and relevance feedback schemes are proposed for query by keyword scenario and query by image example scenario, respectively. Keyword models are combined with visual features in these schemes. In particular, a new, entropy-based active learning strategy is introduced to improve the efficiency of relevance feedback for query by keyword. Furthermore, a new algorithm is proposed to estimate the keyword features of the search concept for query by image example. It is shown to be more appropriate than two existing relevance feedback algorithms. Experimental results demonstrate the effectiveness of the proposed framework.     

科技项目申请书关键词提取方法

关键词提取在文本相似度计算得到应用。传统的关键词提取方法忽略文本中的未登录词以及缺乏对词语语义的理解。针对科技项目申请书,研究提出一种基于未登录词识别与语义的关键词提取方法。应用Lucene和统计相融合的方法进行分词,并识别未登录词作为申请书关键词的一部分;依据社会网络理论构建词语语义相似度网络,并计算词语关联度提取申请书其他关键词。实验结果表明,与传统的关键词提取方法相比,新方法能提取更准确的关键词,有更好的科技项目相似性检查效果。     

A Gaussian Mixture Model-based regularization method in adaptive image restoration

A GMM （Gaussian Mixture Model） based adaptive image restoration is proposed in this paper. The feature vectors of pixels are selected and extracted. Pixels are clustered into smooth, edge or detail texture region according to variance-sum criteria function of the feature vectors. Then parameters of GMM are calculated by using the statistical information of these feature vectors. GMM predicts the regularization parameter for each pixel adaptively. Hopfield Neural Network （Hopfield-NN） is used to optimize the objective function of image restoration, and network weight value matrix is updated by the output of GMM. Since GMM is used, the regularization parameters share properties of different kind of regions. In addition, the regularization parameters are different from pixel to pixel. GMM-based regularization method is consistent with human visual system, and it has strong generalization capability. Comparing with non-adaptive and some adaptive image restoration algorithms, experimental results show that the proposed algorithm obtains more preferable restored images.