Scene classification based on a hierarchical convolutional sparse auto-encoder for high spatial resolution imagery

Efficiently representing and recognizing the semantic classes of the subregions of large-scale high spatial resolution (HSR) remote-sensing images are challenging and critical problems. Most of the existing scene classification methods concentrate on the feature coding approach with handcrafted low-level features or the low-level unsupervised feature learning approaches, which essentially prevent them from better recognizing the semantic categories of the scene due to their limited mid-level feature representation ability. In this article, to overcome the inadequate mid-level representation, a patch-based spatial-spectral hierarchical convolutional sparse auto-encoder (HCSAE) algorithm, based on deep learning, is proposed for HSR remote-sensing imagery scene classification. The HCSAE framework uses an unsupervised hierarchical network based on a sparse auto-encoder (SAE) model. In contrast to the single-level SAE, the HCSAE framework utilizes the significant features from the single-level algorithm in a feedforward and full connection approach to the maximum extent, which adequately represents the scene semantics in the high level of the HCSAE. To ensure robust feature learning and extraction during the SAE feature extraction procedure, a ‘dropout’ strategy is also introduced. The experimental results using the UC Merced data set with 21 classes and a Google Earth data set with 12 classes demonstrate that the proposed HCSAE framework can provide better accuracy than the traditional scene classification methods and the single-level convolutional sparse auto-encoder (CSAE) algorithm.     

一种新的基于语义聚类和图算法的自动图像标注方法

针对图像检索中的语义鸿沟问题,提出了一种新颖的自动图像标注方法。该方法首先采用了一种基于软约束的半监督图像聚类算法(SHMRF-Kmeans)对已标注图像的区域进行语义聚类,这种聚类方法可以同时考虑图像的视觉信息和语义信息。并利用图算法——Manifold排序学习算法充分发掘语义概念与区域聚类中心的关系,得到两者的联合概率关系表。然后利用此概率关系表标注未知标注的图像。该方法与以前的方法相比可以更加充分地结合图像的视觉特征和高层语义。通过在通用图像集上的实验结果表明,本文提出的自动图像标注方法是有效的。     

一种适合弱标签数据集的图像语义标注方法

真实环境下数据集中广泛存在着标签噪声问题,数据集的弱标签性已严重阻碍了图像语义标注的实用化进程.针对弱标签数据集中的标签不准确、不完整和语义分布失衡现象,提出了一种适用于弱标签数据集的图像语义标注方法.首先,在视觉内容与标签语义的一致性约束、标签相关性约束和语义稀疏性约束下,通过直推式学习填充样本标签,构建样本的近似语义平衡邻域.鉴于邻域中存在噪声干扰,通过多标签语义嵌入的邻域最大边际学习获得距离测度和图像语义的一致性,使得近邻处于同一语义子空间.然后,以近邻为局部坐标基,通过邻域非负稀疏编码获得目标图像和近邻的部分相关性,并构建局部语义一致邻域.以邻域内的语义近邻为指导并结合语境相关信息,进行迭代式降噪与标签预测.实验结果表明了方法的有效性.     

基于多标签判别字典学习的图像自动标注

针对图像自动标注中底层视觉特征与高层语义之间的语义鸿沟问题,在传统字典学习的基础上,提出一种基于多标签判别字典学习的图像自动标注方法。首先,为每幅图像提取多种类型特征,将多种特征组合作为字典学习输入特征空间的输入信息;然后,设计一个标签一致性正则化项,将原始样本的标签信息融入到初始的输入特征数据中,结合标签一致性判别字典和标签一致性正则化项进行字典学习;最后,通过得到的字典和稀疏编码矩阵求解标签稀疏编向量,实现未知图像的语义标注。在Corel 5K数据集上测试其标注性能,所提标注方法平均查准率和平均查全率分别可达到35%和48%;与传统的稀疏编码方法（MSC）相比,分别提高了10个百分点和16个百分点;与距离约束稀疏/组稀疏编码方法（DCSC/DCGSC）相比,分别提高了3个百分点和14个百分点。实验结果表明,所提方法能够较好地预测未知图像的语义信息,与当前几种流行的图像标注方法进行比较,所提方法具有较好的标注性能。     

基于物体间支撑语义关系的室内场景彩色深度图像分割

场景理解是智能自主机器人领域的一个重要研究方向,而图像分割是场景理解的基础.但是,不完备的训练数据集,以及真实环境中的罕见情形,会导致在图像分割时存在先验知识不完备的情况,进而影响图像分割的效果.因此,提出在彩色深度(RGB–D)图像上使用抽象的支撑语义关系来解决多样的物体形态所面对的先验知识不完备问题.在先验知识不完备情况下,针对自底向上的图像分割过程中被过度分割出的物体块,首先对物体块间的支撑语义关系进行建模并计算其支撑概率,然后构造能够度量场景总体稳定性的能量函数,最后通过Swendsen-Wang割(SWC)随机图分割算法最小化该能量函数的值,将物体块间的支撑概率转化为强支撑语义关系并完成物体块合并,实现先验知识不完备情况下的图像分割.实验结果证明,结合支撑语义关系的图像分割能够在先验知识不完备的情况下,将同一物体被过度分割的部分重新合并起来,从而提升了图像分割的准确性.     

Bilinear sparse coding for invariant vision

Recent algorithms for sparse coding and independent component analysis (ICA) have demonstrated how localized features can be learned from natural images. However, these approaches do not take image transformations into account. We describe an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model. We show that from an arbitrary set of natural images, the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformations. The learned generative model can be used to translate features to different locations, thereby reducing the need to learn the same feature at multiple locations, a limitation of previous approaches to sparse coding and ICA. Our results suggest that by explicitly modeling the interaction between local image features and their transformations, the sparse bilinear approach can provide a basis for achieving transformation-invariant vision.     

渐进式前景更新和区域关联的图像协同分割

目的 图像协同分割技术是通过多幅参考图像以实现前景目标与背景区域的分离，并已被广泛应用于图像分类和目标识别等领域中。不过，现有多数的图像协同分割算法只适用于背景变化较大且前景几乎不变的环境。为此，提出一种新的无监督协同分割算法。方法 本文方法是无监督式的，在分级图像分割的基础上通过渐进式优化框架分别实现前景和背景模型的更新估计，同时结合图像内部和不同图像之间的分级区域相似度关联进一步增强上述模型估计的鲁棒性。该无监督的方法不需要进行预先样本学习，能够同时处理两幅或多幅图像且适用于同时存在多个前景目标的情况，并且能够较好地适应前景物体类的变化。结果 通过基于iCoseg和MSRC图像集的实验证明，该算法无需图像间具有显著的前景和背景差异这一约束，与现有的经典方法相比更适用于前景变化剧烈以及同时存在多个前景目标等更为一般化的图像场景中。结论 该方法通过对分级图像分割得到的超像素外观分布分别进行递归式估计来实现前景和背景的有效区分，并同时融合了图像内部以及不同图像区域之间的区域关联性来增加图像前景和背景分布估计的一致性。实验表明当前景变化显著时本文方法相比于现有方法具有更为鲁棒的表现。     

Weakly-supervised region annotation for understanding scene images

Scene image understanding has drawn much attention for its intriguing applications in the past years. In this paper, we propose a unified probabilistic graphical model called Topic-based Coherent Region Annotation (TCRA) for weakly-supervised scene region annotation. The multiscale over-segmented regions within a scene image are considered as the “words” of our topic model, which impose neighborhood contextual constraints on topic level through spatial MRF modeling, and incorporate an annotation reasoning mechanism for learning and inferring region labels automatically. Mean field variational inference is provided for model learning. The proposed TCRA has the following two main advantages for understanding natural scene images. First, spatial information of multiscale over-segmented regions is explicitly modeled to obtain coherent region annotations. Second, only image-level labels are needed for automatically inferring the label of every region within the scene. This is particularly helpful in reducing human burden on manually labeling pixel-level semantics in the scene understanding research. Thus, given a scene image that has no textual prior, the regions in it can be automatically labeled using the learned TCRA model. The experimental results conducted on three benchmarks consisting of the MSRCORID image dataset, the UIUC Events image dataset and the SIFT FLOW dataset show that the proposed model outperforms the recent state-of-the-art methods.     

Transductive 3D Shape Segmentation using Sparse Reconstruction

We propose a transductive shape segmentation algorithm, which can transfer prior segmentation results in database to new shapes without explicitly specification of prior category information. Our method first partitions an input shape into a set of segmentations as a data preparation, and then a linear integer programming algorithm is used to select segments from them to form the final optimal segmentation. The key idea is to maximize the segment similarity between the segments in the input shape and the segments in database, where the segment similarity is computed through sparse reconstruction error. The segment‐level similarity enables to handle a large amount of shapes with significant topology or shape variations with a small set of segmented example shapes. Experimental results show that our algorithm can generate high quality segmentation and semantic labeling results in the Princeton segmentation benchmark.     

一种基于支持向量机和半监督期望最大化算法的分级图像标识方法

自动图像标识就是自动识别图像中的有意义目标并赋予其相应的语义关键词, 该过程虽然对于人类来说并不难, 但是对于计算机而言却是一项艰巨而有挑战性的任务. 鉴于人类识别物体通常是一个由粗到细的过程, 本文提出一种层次标识方案. 首先, 输入图像被自动分割成多个区域, 每个区域由支持向量机进行粗分类. 由于粗分类结果会直接影响后续细分类, 本文建立统计的上下文语义关系以修订不正确的粗标识. 接着为了对每个获得粗标识的区域进行细分类, 本文提出一种半监督期望最大化算法, 该算法不仅能为每一粗类别下的细类找到代表模式, 而且能对粗分类区域进行二次分类, 使其获得细标识. 最后我们再次应用上下文语义关系修订不合适的细标识. 为了证明上述识别方案的有效性, 我们开发了一个原型图像标识系统, 实验结果证明该层次标识方案是有效的.     

基于稀疏卷积的前景实时双目深度估计算法

为了提高立体匹配算法处理前景视差估计任务的计算效率,针对一般网络采用完全双目图像作为输入,场景内前景空间占比小而导致输入信息冗余度大的缺点,提出了一种基于稀疏卷积的目标实时立体匹配算法。为实现和改善算法对稀疏前景的视差估计,首先利用分割算法同时获得稀疏前景掩膜和场景语义特征;其次通过稀疏卷积提取稀疏前景区域的空间特征后与场景语义特征相融合,并将融合特征输入到解码模块进行视差回归;最后以前景真值图作为损失生成视差图。在ApolloScape数据集上的测试结果表明,所提算法的准确性和实时性均优于先进算法PSMNet和GANet,且算法的单次运行时间低至60.5 ms,对目标遮挡具有一定的鲁棒性,可用于目标实时深度估计。     

基于EM的非监督图像多标签区域标定算法

提出一个基于EM迭代的非监督图像多标签区域标定算法,它能够非常有效地将基于全图的标签自动标定到图像的对应局部区域上。首先对所有图像进行SIFT特征点的密集采样,然后对所有的SIFT特征点进行K-m eans聚类,获得词典,再构造EM迭代过程计算出每幅图像中每个标签对每个存在WORD的置信度,最后选择那些置信度较高的WORD,确定每幅图像中每个标签置信度最高的对应区域。实验表明,在样本数据充分的情况下,该算法在解决非监督自动标定、标签表观的多样性以及多标签等问题上都取得了不错的效果。     

Image categorization using a semantic hierarchy model with sparse set of salient regions

Image categorization in massive image database is an important problem. This paper proposes an approach for image categorization, using sparse set of salient semantic information and hierarchy semantic label tree (HSLT) model. First, to provide more critical image semantics, the proposed sparse set of salient regions only at the focuses of visual attention instead of the entire scene was formed by our proposed saliency detection model with incorporating low and high level feature and Shotton’s semantic texton forests (STFs) method. Second, we also propose a new HSLT model in terms of the sparse regional semantic information to automatically build a semantic image hierarchy, which explicitly encodes a general to specific image relationship. And last, we archived image dataset using image hierarchical semantic, which is help to improve the performance of image organizing and browsing. Extension experimental results showed that the use of semantic hierarchies as a hierarchical organizing framework provides a better image annotation and organization, improves the accuracy and reduces human’s effort.     

融合颜色与纹理的复杂场景下的服装图像分割算法

针对复杂场景下拍摄到的服装图像的分割问题,提出一种基于先验知识的融合颜色和纹理特征的无监督分割算法。首先利用块截断编码思想将传统的三维颜色空间截断成为六维空间,得到更为精细的颜色特征,并结合改进的局部二值模式纹理特征实现对图像的特征描述;然后根据目标区域和背景区域在图像中出现的统计规律,提出了一种基于先验知识的两分法来对图像进行分割。由于对图像做了分块处理,因此在子图像块的基础上进行的图像分割将更加高效。实验表明,设计的算法能快速有效地将目标区域从各类不同的复杂场景中分割出来,且整个过程无须人工设定任何参数,对后续的图像理解和图像检索具有重要意义。     

Semantic image segmentation using low-level features and contextual cues

Semantic image segmentation aims to partition an image into non-overlapping regions and assign a pre-defined object class label to each region. In this paper, a semantic method combining low-level features and high-level contextual cues is proposed to segment natural scene images. The proposed method first takes the gist representation of an image as its global feature. The image is then over-segmented into many super-pixels and histogram representations of these super-pixels are used as local features. In addition, co-occurrence and spatial layout relations among object classes are exploited as contextual cues. Finally the features and cues are integrated into the inference framework based on conditional random field by defining specific potential terms and introducing weighting functions. The proposed method has been compared with state-of-the-art methods on the MSRC database, and the experimental results show its effectiveness.     

场景语义树图像标注方法

自动图像标注是一项具有挑战性的工作,它对于图像分析理解和图像检索都有着重要的意义.在自动图像标注领域,通过对已标注图像集的学习,建立语义概念空间与视觉特征空间之间的关系模型,并用这个模型对未标注的图像集进行标注.由于低高级语义之间错综复杂的对应关系,使目前自动图像标注的精度仍然较低.而在场景约束条件下可以简化标注与视觉特征之间的映射关系,提高自动标注的可靠性.因此提出一种基于场景语义树的图像标注方法.首先对用于学习的标注图像进行自动的语义场景聚类,对每个场景语义类别生成视觉场景空间,然后对每个场景空间建立相应的语义树.对待标注图像,确定其语义类别后,通过相应的场景语义树,获得图像的最终标注.在Corel5K图像集上,获得了优于TM(translation model)、CMRM(cross media relevance model)、CRM(continous-space relevance model)、PLSA-GMM(概率潜在语义分析-高期混合模型)等模型的标注结果.     

融合T节点线索的图像物体分割

目的 图像分割是计算机视觉、数字图像处理等应用领域首要解决的关键问题。针对现有的单幅图像物体分割算法广泛存在的过分割和过合并现象,提出基于图像T型节点线索的图像物体分割算法。方法 首先,利用L0梯度最小化方法平滑目标图像,剔除细小纹理的干扰;其次,基于Graph-based分割算法对平滑后图像进行适度分割,得到粗糙分割结果;最后,借助于图像中广泛存在的T型节点线索对初始分割块进行区域合并得到最终优化分割结果。结果 将本文算法分别与Grabcut算法及Graph-based算法在不同场景类型下进行了实验与对比。实验结果显示,Grabcut算法需要人工定位边界且一次只能分割单个物体,Graph-based算法综合类内相似度和类间差异性,可以有效保持图像边界,但无法有效控制分割块数量,且分割结果对阈值参数过分依赖,极易导致过分割和过合并现象。本文方法在降低过分割和过合并现象、边界定位精确性和分割准确率方面获得明显改进,几组不同类型的图片分割准确率平均值达到91.16%,明显由于其他算法。处理图像尺寸800×600像素的图像平均耗时3.5 s,较之其他算法略有增加。结论 与各种算法对比结果表明,该算法可有效解决过分割和过合并问题,对比实验结果验证了该方法的有效性,能够取得具有一定语义的图像物体分割结果。     

模糊相关图割的非监督层次化彩色图像分割

目的 基于阈值的分割方法能根据像素的信息将图像划分为同类的区域,其中常用的最大模糊相关分割方法,因能利用模糊相关度量划分的适当性,得到较好的分割结果,而广受关注。然而该算法存在划分数需预先确定,阈值的分割结果存在孤立噪声,无法对彩色图像实施分割的问题。为此,提出基于模糊相关图割的非监督层次化分割策略来解决该问题。方法 算法首先将图像划分为若干超像素,以提高层次化图像分割的效率;随后将快速模糊相关算法与图割结合,构成模糊相关图割2-划分算子,在确保分割效率的基础上,解决单一阈值分割存在孤立噪声的问题;最后设计了自顶向下层次化分割策略,利用构建的2-划分算子选择合适的区域及通道,迭代地对超像素实施层次化分割,直到算法收敛,划分数自动确定。结果 对Berkeley分割数据库上300幅图像进行了测试,结果表明算法能有效分割彩色图像,分割精度优于Ncut、JSEG方法,运行时间较这两种方法也提高了近20%。结论 本文算法为最大模糊相关算法在非监督彩色图像分割领域的应用提供指导依据,能用于目标检测和识别领域。     

基于全卷积神经网络与低秩稀疏分解的显著性检测

为了准确检测复杂背景下的显著区域,提出一种全卷积神经网络与低秩稀疏分解相结合的显著性检测方法,将图像分解为代表背景的低秩矩阵和对应显著区域的稀疏噪声,结合利用全卷积神经网络学习得到的高层语义先验知识,检测图像中的显著区域.首先,对原图像进行超像素聚类,并提取每个超像素的颜色、纹理和边缘特征,据此构成特征矩阵;然后,在MSRA数据库中,基于梯度下降法学习得到特征变换矩阵,利用全卷积神经网络学习得到高层语义先验知识;接着,利用特征变换矩阵和高层语义先验知识矩阵对特征矩阵进行变换;最后,利用鲁棒主成分分析算法对变换后的矩阵进行低秩稀疏分解,并根据分解得到的稀疏噪声计算显著图.在公开数据集上进行实验验证,并与当前流行的方法进行对比,实验结果表明,本文方法能够准确地检测感兴趣区域,是一种有效的自然图像目标检测与分割的预处理方法.