Urban land cover multi‐level region‐based classification of VHR data by selecting relevant features

The limited spatial resolution of satellite images used to be a problem for the adequate definition of the urban environment. This problem was expected to be solved with the availability of very high spatial resolution satellite images (IKONOS, QuickBird, OrbView‐3). However, these space‐borne sensors are limited to four multi‐spectral bands and may have specific limitations as far as detailed urban area mapping is concerned. It is therefore essential to combine spectral information with other information, such as the features used in visual interpretation (e.g. the degree and kind of texture and the shape) transposed to digital analysis. In this study, a feature selection method is used to show which features are useful for particular land‐cover classes. These features are used to improve the land‐cover classification of very high spatial resolution satellite images of urban areas. The useful features are compared with a visual feature selection. The features are calculated after segmentation into regions that become analysis units and ease the feature calculation.     

A 2-D wavelet decomposition-based bag-of-visual-words model for land-use scene classification

Previous works about spatial information incorporation into a traditional bag-of-visual-words (BOVW) model mainly consider the spatial arrangement of an image, ignoring the rich textural information in land-use remote-sensing images. Hence, this article presents a 2-D wavelet decomposition (WD)-based BOVW model for land-use scene classification, since the 2-D wavelet decomposition method does well not only in textural feature extraction, but also in the multi-resolution representation of an image, which is favourable for the use of both spatial arrangement and textural information in land-use images. The proposed method exploits the textural structures of an image with colour information transformed into greyscale. Moreover, it works first by decomposing the greyscale image into different sub-images using 2-D discrete wavelet transform (DWT) and then by extracting local features of the greyscale image and all the decomposed images with dense regions in which a given image is evenly sampled by a regular grid with a specified grid space. After that, the method generates the corresponding visual vocabularies and computes histograms of visual word occurrences of local features found in each former image. Specifically, the soft-assignment or multi-assignment (MA) technique is employed, accounting for the impact of clustering on visual vocabulary creation that two similar image patches may be clustered into different clusters when increasing the size of visual vocabulary. The proposed method is evaluated on a ground truth image dataset of 21 land-use classes manually extracted from high-resolution remote-sensing images. Experimental results demonstrate that the proposed method significantly outperforms previous methods, such as the traditional BOVW model, the spatial pyramid representation-based BOVW method, the multi-resolution representation-based BOVW method, and so on, and even exceeds the best result obtained from the creator of the land-use dataset. Therefore, the proposed approach is very suitable for land-use scene classification tasks.     

基于上下文分析的无监督分层迭代算法用于SAR图像分割

基于聚类的分割算法能够有效地分析目标特征在特征域的分布结构,进而准确判断目标的所属类别,但难以利用图像的空间和边缘信息,而基于区域增长的分割算法能够在空间域利用多种图像信息计算目标之间的相似性,但缺乏对特征结构本身的深层挖掘,容易出现欠分割或过分割的结果. 本文结合这两种算法各自的优势,针对合成孔径雷达（Synthetic aperture radar,SAR）图像的特点,提出了一种基于上下文分析的无监督分层迭代算法. 该算法使用过分割区域作为操作单元,以提高分割速度,降低SAR图像相干斑噪声的影响. 在合并过分割区域时,该算法采用了分层迭代的策略:首先,设计了一种改进的模糊C均值聚类算法,对过分割区域的外观特征进行聚类分析,获得其类别标记,该类别标记包含了特征的分布结构信息. 然后,利用多种SAR图像特征对同类区域的空域上下文进行分析,使用区域迭代增长算法对全局范围内的相似区域进行合并,直到不存在满足合并条件的过分割区域对为止,再重新执行聚类算法. 这两种子算法分层交替迭代,扬长避短,实现了一种有效的方法来组织和利用多种信息对SAR图像进行分割. 对模拟和真实SAR图像的实验表明,本文提出的算法能够在区域一致性和细节保留之间做到很好的平衡,准确地分割出各类目标区域,对相干斑噪声具有很强的鲁棒性.     

An effective hybrid approach to remote-sensing image classification

This article presents a hybrid fuzzy classifier for effective land-use/land-cover (LULC) mapping. It discusses a Bayesian method of incorporating spatial contextual information into the fuzzy noise classifier (FNC). The FNC was chosen as it detects noise using spectral information more efficiently than its fuzzy counterparts. The spatial information at the level of the second-order pixel neighbourhood was modelled using Markov random fields (MRFs). Spatial contextual information was added to the MRF using different adaptive interaction functions. These help to avoid over-smoothing at the class boundaries. The hybrid classifier was applied to advanced wide-field sensor (AWiFS) and linear imaging self-scanning sensor-III (LISS-III) images from a rural area in India. Validation was done with a LISS-IV image from the same area. The highest increase in accuracy among the adaptive functions was 4.1% and 2.1% for AWiFS and LISS-III images, respectively. The paper concludes that incorporation of spatial contextual information into the fuzzy noise classifier helps in achieving a more realistic and accurate classification of satellite images.     

No-reference image quality assessment using interval type 2 fuzzy sets

Image quality assessment of distorted or decompressed images without any reference to the original image is challenging from computational point of view. Quality of an image is best judged by human observers without any reference image, and evaluated using subjective measures. The paper aims at designing a generic no-reference image quality assessment (NR-IQA) method by incorporating human visual perception in assigning quality class labels to the images. Using fuzzy logic approach, we consider information theoretic entropies of visually salient regions of images as features and assess quality of the images using linguistic values. The features are transformed into fuzzy feature space by designing an algorithm based on interval type-2 (IT2) fuzzy sets. The algorithm measures uncertainty present in the input–output feature space to predict image quality accurately as close to human observations. We have taken a set of training images belonging to five different pre-assigned quality class labels for calculating foot print of uncertainty (FOU) corresponding to each class. To assess the quality class label of the test images, maximum of T-conorm applied on the lower and upper membership functions of the test images belonging to different classes is calculated. Our proposed image quality metric is compared with other no-reference quality metrics demonstrating more accurate results and compatible with subjective mean opinion score metric.     

构建近邻上下文的拷贝图像检索

目的 以词袋模型为基础的拷贝图像检索方法是当前最有效的方法。然而,由于局部特征量化存在信息损失,导致视觉词汇区别能力不足和视觉词汇误匹配增加,从而影响了拷贝图像检索效果。针对视觉词汇的误匹配问题,提出一种基于近邻上下文的拷贝图像检索方法。该方法通过局部特征的上下文关系消除视觉词汇歧义,提高视觉词汇的区分度,进而提高拷贝图像的检索效果。方法 首先,以距离和尺度关系选择图像中某局部特征点周围的特征点作为该特征点的上下文,选取的上下文中的局部特征点称为近邻特征点;再以近邻特征点的信息以及与该局部特征的关系为该局部特征构建上下文描述子;然后,通过计算上下文描述子的相似性对局部特征匹配对进行验证;最后,以正确匹配特征点的个数衡量图像间的相似性,并以此相似性选取若干候选图像作为返回结果。结果 在Copydays图像库进行实验,与Baseline方法进行比较。在干扰图像规模为100 k时,相对于Baseline方法,mAP提高了63%。当干扰图像规模从100 k增加到1 M时,Baseline的mAP值下降9%,而本文方法下降3%。结论 本文拷贝图像检索方法对图像编辑操作,如旋转、图像叠加、尺度变换以及裁剪有较高的鲁棒性。该方法可以有效地应用到图像防伪、图像去重等领域。     

多专家注释的视杯和视盘不确定性量化

现有基于深度学习的视杯和视盘分割方法在模型训练时,仅使用图像的单个注释或从多个注释中获取唯一的注释信息,忽略原始多专家标注中嵌入的一致性或差异性信息,从而导致模型和预测结果过度自信等问题。提出一种基于多解码器不确定性感知体系的模型MUA-Net。通过引入专业知识推断模块,将各个专家注释的专业知识水平作为先验知识嵌入编码器和解码器的瓶颈中,以形成包含专家线索的高级语义特征。利用可同时学习多个注释的多解码器结构调节多专家之间的分歧,重构多专家注释过程,并对不确定或分歧区域进行量化。提出一种双分支软注意机制,增强多解码器分割预测的模糊区域,得到最终校准的分割结果。实验结果表明,该模型在RIGA数据集上能以较高的不确定性预测合理的区域,与MRNet模型相比,该模型在视杯分割中的平均精度、Dice系数、交并比分别提升了0.75、0.39、0.41个百分点。     

基于随机点积图的图像标注改善算法

针对自动图像标注中底层特征和高层语义之间的鸿沟问题,提出一种基于随机点积图的图像标注改善算法。该算法首先采用图像底层特征对图像候选标注词建立语义关系图,然后利用随机点积图对其进行随机重构,从而挖掘出训练图像集中丢失的语义关系,最后采用重启式随机游走算法,实现图像标注改善。该算法结合了图像的底层特征与高层语义,有效降低了图像集规模变小对标注的影响。在3种通用图像库上的实验证明了该算法能够有效改善图像标注,宏F值与微平均F值最高分别达到0.784与0.743。     

融合全局与空间多尺度上下文信息的车辆重识别

目的 车辆重识别指判断不同摄像设备拍摄的车辆图像是否属于同一辆车的检索问题。现有车辆重识别算法使用车辆的全局特征或额外的标注信息,忽略了对多尺度上下文信息的有效抽取。对此,本文提出了一种融合全局与空间多尺度上下文信息的车辆重识别模型。方法 首先,设计一个全局上下文特征选择模块,提取车辆的细粒度判别信息,并且进一步设计了一个多尺度空间上下文特征选择模块,利用多尺度下采样的方式,从全局上下文特征选择模块输出的判别特征中获得其对应的多尺度特征。然后,选择性地集成来自多级特征的空间上下文信息,生成车辆图像的前景特征响应图,以此提升模型对于车辆空间位置特征的感知能力。最后,模型组合了标签平滑的交叉熵损失函数和三元组损失函数,以提升模型对强判别车辆特征的整体学习能力。结果 在VeRi-776(vehicle re-idendification-776)数据集上,与模型PNVR(part-regularized near-duplicate vehicle re-identification)相比,本文模型的mAP(mean average precision)和rank-1 (cumulative...     

Image Segmentation Based on Adaptive Cluster Prototype Estimation

An image segmentation algorithm based on adaptive fuzzy c-means (FCM) clustering is presented in this paper. In the conventional FCM clustering algorithm, cluster assignment is based solely on the distribution of pixel attributes in the feature space, and does not take into consideration the spatial distribution of pixels in an image. By introducing a novel dissimilarity index in the modified FCM objective function, the new adaptive fuzzy clustering algorithm is capable of utilizing local contextual information to impose local spatial continuity, thus exploiting the high inter-pixel correlation inherent in most real-world images. The incorporation of local spatial continuity allows the suppression of noise and helps to resolve classification ambiguity. To account for smooth intensity variation within each homogenous region in an image, a multiplicative field is introduced to each of the fixed FCM cluster prototype. The multiplicative field effectively makes the fixed cluster prototype adaptive to slow smooth within-cluster intensity variation, and allows homogenous regions with slow smooth intensity variation to be segmented as a whole. Experimental results with synthetic and real color images have shown the effectiveness of the proposed algorithm.     

A fast and robust image segmentation using FCM with spatial information

Automated segmentation of images has been considered an important intermediate processing task to extract semantic meaning from pixels. In general, the fuzzy c-means approach (FCM) is highly effective for image segmentation. But for the conventional FCM image segmentation algorithm, cluster assignment is based solely on the distribution of pixel attributes in the feature space, and the spatial distribution of pixels in an image is not taken into consideration. In this paper, we present a novel FCM image segmentation scheme by utilizing local contextual information and the high inter-pixel correlation inherent. Firstly, a local spatial similarity measure model is established, and the initial clustering center and initial membership are determined adaptively based on local spatial similarity measure model. Secondly, the fuzzy membership function is modified according to the high inter-pixel correlation inherent. Finally, the image is segmented by using the modified FCM algorithm. Experimental results showed the proposed method achieves competitive segmentation results compared to other FCM-based methods, and is in general faster.     

基于多重注意结构的图像密集描述生成方法研究EI北大核心CSCD

图像密集描述旨在为复杂场景图像提供细节描述语句.现有研究方法虽已取得较好成绩,但仍存在以下两个问题:1)大多数方法仅将注意力聚焦在网络所提取的深层语义信息上,未能有效利用浅层视觉特征中的几何信息;2)现有方法致力于改进感兴趣区域间上下文信息的提取,但图像内物体空间位置信息尚不能较好体现.为解决上述问题,提出一种基于多重注意结构的图像密集描述生成方法—MAS-ED (Multiple attention structure-encoder decoder). MAS-ED通过多尺度特征环路融合(Multi-scale feature loop fusion, MFLF)机制将多种分辨率尺度的图像特征进行有效集成,并在解码端设计多分支空间分步注意力(Multi-branch spatial step attention, MSSA)模块,以捕捉图像内物体间的空间位置关系,从而使模型生成更为精确的密集描述文本.实验在Visual Genome数据集上对MAS-ED进行评估,结果表明MASED能够显著提升密集描述的准确性,并可在文本中自适应加入几何信息和空间位置信息.基于长短期记忆网络(Longshort term memory, LSTM)解码网络框架, MAS-ED方法性能在主流评价指标上优于各基线方法.     

自适应非局部空间约束与K-L信息的模糊C-均值噪声图像分割算法

针对传统模糊C-均值聚类(FCM)算法对噪声鲁棒性差的问题, 提出一种自适应非局部空间约束与K-L信息 的模糊C-均值噪声图像分割算法. 首先, 通过定义平滑度, 设计自适应匹配函数, 实现非局部空间信息项搜索窗口和 邻域窗口的自适应计算, 克服非局部空间信息窗口大小固定的问题. 其次, 将K-L信息引入目标函数, 利用隐马尔可 夫模型计算图像像素的上下文信息, 减少分割的模糊性. 最后, 利用原始图像和非局部空间信息项局部方差的绝对 差和其倒数自适应约束原始图像和非局部空间信息项, 实现约束项参数的自适应选择, 提高算法的灵活性. 含噪合 成图像和彩色图像分割实验表明, 该算法在分割精准度、平均交互比、归一化互信息、模糊分割系数和模糊划分熵 等性能方面均优于其他几种FCM算法. 例如, 在混合噪声密度为15%的条件下, 算法的模糊分割系数和模糊划分熵 分别达到99.92%和0.14%.     

区域语义多样性密度的图像标注

随着Web2.0下海量共享图像的出现,如何获取图像具有描述力的精准区域标注具有重要研究意义。本文提出一种基于区域语义多样性密度的图像标注框架,重点考虑区域间的视觉特征差异和空间结构差异。具体来说,基于距离相似度的特征多样性密度实现了区域特征语义标注;引入负相关示例的惩罚作用实现了区域空间关系语义及属性语义标注。在NUS-WIDE和MSRC数据集上验证了方法的有效性,区域属性标注的正确率在80%以上,同时基于属性标注的图像检索的平均查准率达到82%。     

基于K-L变换和模糊集理论的彩色字符图像分割

根据彩色印刷字符图像的特点,在Lab颜色空间下提取a分量,将彩色图像转换为灰度图像。根据模糊逻辑和阈值分割方法将图像分为目标区域、背景区域以及模糊区域。用K-L变换组合邻域的区域隶属信息和灰度信息,将灰度域换成模糊域,在该模糊域上进行分割。经实践,该算法在工业环境中对复杂背景的彩色印刷图像可以得到较好的分割效果,其时间复杂度不高于传统的阈值分割算法,并且在分割的精确度上要优于传统的阈值分割算法。     

A novel context sensitive multilevel thresholding for image segmentation

Most of the traditional histogram-based thresholding techniques are effective for bi-level thresholding and unable to consider spatial contextual information of the image for selecting optimal threshold. In this article a novel thresholding technique is presented by proposing an energy function to generate the energy curve of an image by taking into an account the spatial contextual information of the image. The behavior of this energy curve is very much similar to the histogram of the image. To incorporate spatial contextual information of the image for threshold selection process, this energy curve is used as an input of our technique instead of histogram. Moreover, to mitigate multilevel thresholding problem the properties of genetic algorithm are exploited. The proposed algorithm is evaluated on the number of different types of images using a validity measure. The results of the proposed technique are compared with those obtained by using histogram of the image and also with an existing genetic algorithm based context sensitive technique. The comparisons confirmed the effectiveness of the proposed technique.     

M-FFN: multi-scale feature fusion network for image captioning

In this work, we present a novel multi-scale feature fusion network (M-FFN) for image captioning task to incorporate discriminative features and scene contextual information of an image. We construct multi-scale feature fusion network by leveraging spatial transformation and multi-scale feature pyramid networks via feature fusion block to enrich spatial and global semantic information. In particular, we take advantage of multi-scale feature pyramid network to incorporate global contextual information by employing atrous convolutions on top layers of convolutional neural network (CNN). And, the spatial transformation network is exploited on early layers of CNN to remove intra-class variability caused by spatial transformations. Further, the feature fusion block integrates both global contextual information and spatial features to encode the visual information of an input image. Moreover, spatial-semantic attention module is incorporated to learn attentive contextual features to guide the captioning module. The efficacy of the proposed model is evaluated on the COCO dataset.

     

基于多分辨率分析及QFCM算法的图像分割方法研究

在图像的多分辨率小波分析的基础上,采用高斯－马尔可夫随机场模型来描述图像的局部特征,利用ＬＭＳ算法（ｔｈｅ ｌｅａｓｔ－ｍｅａｎ－ｓｑｕａｒｅ ａｌｇｏｒｉｔｈｍ）求得模型的参数估计,构造出图像的特征集,再利用快速模糊Ｃ－均值聚类方法（ＱＦＣＭ）对该特征集进行模糊划分,从而完成图像的分割,实验证明,这种方法具有较强的适应性,尤其对于景物－背景对比度差以及信噪比较低的一类图像,具有良好的分割效果。     

Fuzzy clustering algorithms incorporating local information for change detection in remotely sensed images

In this paper we have used two fuzzy clustering algorithms, namely fuzzy c-means (FCM) and Gustafson–Kessel clustering (GKC) along with local information for unsupervised change detection in multitemporal remote sensing images. In conventional FCM and GKC no spatio-contextual information is taken into account and thus the result is not so much robust to small changes. Since the pixels are highly correlated with their neighbors in image space (spatial domain), incorporation of local information enhances the performance of the algorithms. In this work we have introduced a new technique for incorporation of local information. Change detection maps are obtained by separating the pixel-patterns of the difference image into two groups. Hybridization of FCM and GKC with two other optimization techniques, genetic algorithm (GA) and simulated annealing (SA), is made to further enhance the performance. To show the effectiveness of the proposed technique, experiments are conducted on two multispectral and multitemporal remote sensing images. Two fuzzy cluster validity measures (Xie–Beni and fuzzy hypervolume) have been used to quantitatively evaluate the performance. Results are compared with those of existing state of the art Markov random field (MRF) and neural network based algorithms and found to be superior. The proposed technique is less time consuming and unlike MRF does not require any a priori knowledge of distributions of changed and unchanged pixels.