Modeling textured images using generalized long correlation models

The long correlation (LC) models are a general class of random field (RF) models which are able to model correlations, extending over large image regions with few model parameters. The LC models have seen limited use, due to lack of an effective method for estimating the model parameters. In this work, we develop an estimation scheme for a very general form of this model and demonstrate its applicability to texture modeling applications. The relationship of the generalized LC models to other classes of RF models, namely the simultaneous autoregressive (SAR) and Markov random field (MRF) models, is shown. While it is known that the SAR model is a special case of the LC model, we show that the MRF model is also encompassed by this model. Consequently, the LC model may be considered as a generalization of the SAR and MRF models     

Incorporating higher-order point distribution model priors into MRFs using convex quadratic programming

Recently, with the advent of powerful optimisation algorithms for Markov random fields (MRFs), priors of high arity (more than two) have been put into practice more widely. The statistical relationship between object parts encoding shape in a covariant space, also known as the point distribution model (PDM), is a widely employed technique in computer vision which has been largely overlooked in the context of higher-order MRF models. This paper focuses on such higher-order statistical shape priors and illustrates that in a spatial transformation invariant space, these models can be formulated as convex quadratic programmes. As such, the associated energy of a PDM may be optimised efficiently using a variety of different dedicated algorithms. Moreover, it is shown that such an approach in the context of graph matching can be utilised to incorporate both a global rigid and a non-rigid deformation prior into the problem in a parametric form, a problem which has been rarely addressed in the literature. The paper then illustrates an application of PDM priors for different tasks using graphical models incorporating factors of different cardinalities.     

Discriminative Random Fields

In this research we address the problem of classification and labeling of regions given a single static natural image. Natural images exhibit strong spatial dependencies, and modeling these dependencies in a principled manner is crucial to achieve good classification accuracy. In this work, we present Discriminative Random Fields (DRFs) to model spatial interactions in images in a discriminative framework based on the concept of Conditional Random Fields proposed by lafferty et al.(2001). The DRFs classify image regions by incorporating neighborhood spatial interactions in the labels as well as the observed data. The DRF framework offers several advantages over the conventional Markov Random Field (MRF) framework. First, the DRFs allow to relax the strong assumption of conditional independence of the observed data generally used in the MRF framework for tractability. This assumption is too restrictive for a large number of applications in computer vision. Second, the DRFs derive their classification power by exploiting the probabilistic discriminative models instead of the generative models used for modeling observations in the MRF framework. Third, the interaction in labels in DRFs is based on the idea of pairwise discrimination of the observed data making it data-adaptive instead of being fixed a priori as in MRFs. Finally, all the parameters in the DRF model are estimated simultaneously from the training data unlike the MRF framework where the likelihood parameters are usually learned separately from the field parameters. We present preliminary experiments with man-made structure detection and binary image restoration tasks, and compare the DRF results with the MRF results. Sanjiv Kumar is currently with Google Research, Pittsburgh, PA, USA. His contact email is: sanjivk@google.com.     

一种新的SAR图像无监督分割方法

马尔可夫随机场（MRF）在SAR图像分割中有着广泛的应用。由于合成孔径雷达（SAR）图像本身所固有的相干斑噪声的影响,传统方法很难获得准确的分割,因此提出了一种新的基于MRF（Markov Random Field）融合Gaussian-Hermite矩（GHM）的SAR图像无监督分割算法。利用Gaussian-Hermite矩的不同阶矩作为SAR图像特征得到初始分割;将得到的初始分割结果作为MRF随机场的先验模型,通过引入一个基于两成分权重参数的能量函数,利用最大后验概率（MAP）得到最终的分割结果。通过对合成图像及SAR图像分割实验结果的比较,表明了该方法在误分率、抗噪性以及视觉效果上具有更好的效果。     

Spectral–spatial classification of hyperspectral images by algebraic multigrid based multiscale information fusion

In this work, we present a novel spectral-spatial classification framework of hyperspectral images (HSIs) by integrating the techniques of algebraic multigrid (AMG), hierarchical segmentation (HSEG) and Markov random field (MRF). The proposed framework manifests two main contributions. First, an effective HSI segmentation method is developed by combining the AMG-based marker selection approach and the conventional HSEG algorithm to construct a set of unsupervised segmentation maps in multiple scales. To improve the computational efficiency, the fast Fish Markov selector (FMS) algorithm is exploited for feature selection before image segmentation. Second, an improved MRF energy function is proposed for multiscale information fusion (MIF) by considering both spatial and inter-scale contextual information. Experiments were performed using two airborne HSIs to evaluate the performance of the proposed framework in comparison with several popular classification methods. The experimental results demonstrated that the proposed framework can provide superior performance in terms of both qualitative and quantitative analysis.     

基于MRF模型和EM算法的多源图像融合方法

提出了两种图像融合方法.该方法首先利用EM-MRF算法与模糊分类方法的等价性,将EM-MRF算法引入到图像融合领域.在此基础上,利用统计模型对图像进行非监督分类的模型参数估计转化通过EM算法从不完全数据中估计模型参数的问题,并利用Markov随机场模型建立类别的先验概率、EM迭代算法进行图像分类的方法有较高的分类精度和鲁棒性,导出了基于分布式和集中式多传感器图像融合模型的两种融合方法.最后仿真试验表明,这两种融合方法既可以提高分类精度,又可以加强对噪声的抗干扰能力.     

IRGS: image segmentation using edge penalties and region growing

This paper proposes an image segmentation method named iterative region growing using semantics (IRGS), which is characterized by two aspects. First, it uses graduated increased edge penalty (GIEP) functions within the traditional Markov random field (MRF) context model in formulating the objective functions. Second, IRGS uses a region growing technique in searching for the solutions to these objective functions. The proposed IRGS is an improvement over traditional MRF based approaches in that the edge strength information is utilized and a more stable estimation of model parameters is achieved. Moreover, the IRGS method provides the possibility of building a hierarchical representation of the image content, and allows various region features and even domain knowledge to be incorporated in the segmentation process. The algorithm has been successfully tested on several artificial images and synthetic aperture radar (SAR) images.     

深度置信网络在极化SAR图像分类中的应用

目的 深度置信网络能够从数据中自动学习、提取特征,在特征学习方面具有突出优势。极化SAR图像分类中存在海量特征利用率低、特征选取主观性强的问题。为了解决这一问题,提出一种基于深度置信网络的极化SAR图像分类方法。方法 首先进行海量分类特征提取,获得极化类、辐射类、空间类和子孔径类四类特征构成的特征集;然后在特征集基础上选取样本并构建特征矢量,用以输入到深度置信网络模型之中;最后利用深度置信网络的方法对海量分类特征进行逐层学习抽象,获得有效的分类特征进行分类。结果 采用AIRSAR数据进行实验,分类结果精度达到91.06%。通过与经典Wishart监督分类、逻辑回归分类方法对比,表现了深度置信网络方法在特征学习方面的突出优势,验证了方法的适用性。结论 针对极化SAR图像海量特征的选取与利用,提出了一种新的分类方法,为极化SAR图像分类提供了一种新思路,为深度置信网络获得更广泛地应用进行有益的探索和尝试。     

Study on segmentation and interpretation of multi-look polarimetric SAR images

The segmentation and interpretation of multi-look polarimetric synthetic aperture radar (SAR) images is studied. We first introduce a multi-look polarimetric whitening filter (MPWF) to reduce the speckle in multi-look polarimetric SAR images. Then, by utilizing the wavelet multiresolution approach to extract the texture information in different scales and the Markov random field (MRF) model to characterize the spatial constraints between pixels in each scale level, a multiresolution segmentation algorithm (MSA) to segment the speckle-reduced SAR images is presented. The MSA first segments the image at the lowest resolution level and then proceeds to progressively higher resolutions until individual pixels are well classified. An unsupervised step to estimate both the optimal number of texture classes and their model parameters is also included in the MSA so that the segmentation can be implemented without supervision. Finally, in order to interpret the results of the unsupervised segmentation and to understand the whole polarimetric SAR image, we develop an image interpretation approach which jointly utilizes the scattering mechanism identification and target decomposition approaches. Experimental results with the real-world multi-look polarimetric SAR image demonstrate the effectiveness of the segmentation and interpretation approaches.     

联合显著性与MRF的SAR建筑物分割算法

针对经典马尔可夫随机场（MRF）在进行高分辨率SAR图像分割时存在容易受到斑点噪声干扰等问题,提出一种基于建筑物指数相似度距离及MRF模型（BISD-MRF）的高分辨率SAR建筑物分割算法。基于较复杂SAR场景下建筑物目标可能呈现多种形态结构的问题,设计一种多尺度显著性建筑物指数（MSBI）方案来提取建筑物目标的显著性特征,并通过强度信息重构、纹理显著性提取、频谱显著性信息统计来分别提取不同类型区域的显著性信息,构建适用于SAR建筑物目标的显著性模型。在此基础上,将MSBI值引入到改进的基于改进余弦函数的势函数模型中,利用余弦函数对邻域像素MSBI值进行相似性度量,同时利用特征空间语义信息对像素及其邻域像素标签信息进行有效约束,以提升势函数模型对高分辨率SAR建筑物目标的表征能力。不同平台下的建筑物分割实验结果表明,与MRF、MBI、FRFCM等算法相比,本文算法分割性能平均提升了4.3~10.7个百分点,更适用于较复杂场景下高分辨率SAR建筑物的分割任务。     

A generative framework for fast urban labeling using spatial and temporal context

This paper introduces a multi-level classification framework for the semantic annotation of urban maps as provided by a mobile robot. Environmental cues are considered for classification at different scales. The first stage considers local scene properties using a probabilistic bag-of-words classifier. The second stage incorporates contextual information across a given scene (spatial context) and across several consecutive scenes (temporal context) via a Markov Random Field (MRF). Our approach is driven by data from an onboard camera and 3D laser scanner and uses a combination of visual and geometric features. By framing the classification exercise probabilistically we take advantage of an information-theoretic bail-out policy when evaluating class-conditional likelihoods. This efficiency, combined with low order MRFs resulting from our two-stage approach, allows us to generate scene labels at speeds suitable for online deployment. We demonstrate the virtue of considering such spatial and temporal context during the classification task and analyze the performance of our technique on data gathered over almost 17 km of track through a city.     

一种新的基于大尺寸信息的MRF先验模型

基于马尔可夫随机场理论,Bayesian重建被认为是一种解决图像复原和重建中的病态问题的有效方法。通常,大部分先验模型中的信息都来自小邻域内像素灰度值的简单加权,因此仅能提供给正则化有限的信息。在研究大尺寸信息的过程中,本文提出一种新的非局部先验。在发射断层成像的相关实验表明,该MRF非邻域先验能比传统先验提供更为有效的正则化处理。     

基于区域确定的分层马尔可夫模型及其MPM算法

基于四叉树的分层马尔可夫随机场 (Markov random field, MRF) 模型在层间存在因果性, 不需要像非因果马尔可夫随机场模型那样的迭代算法, 但是传统的分层 MRF 模型常常导致分割结果具有块状现象和非连续边缘. 本文提出一种新的基于区域确定的半树分层 MRF 算法, 并推导出它的最大后验边缘概率 (Maximizer of the posteriori marginal, MPM) 算法. 在流域算法过分割结果的基础上, 该模型将层间的点概率转换为区域概率, 采用区域概率实现各层图像分割. 从 SAR 图像的监督分割实验结果来看, 本文提出的模型较好地克服了基于像素分层模型和单分辨率 MRF 模型带来的块现象和非连续边界, 因而具有更好的分割结果.     

A means for utilizing ancillary information in multispectral classification

A method is presented that allows information from ancillary data sources to be incorporated into the results of an existing classification of remotely sensed data. Based upon probabilistic label relaxation procedures, which are used for imbedding spatial context data in image-labeling problems, the method utilizes the source of ancillary information in the form of a set of probabilities. These are injected into a modified relaxation method called supervised relaxation labeling which, on application, develops a labeling for remotely sensed data that strikes a balance in consistency between spectral, spatial, and ancillary data sources of information. Results are presented of a forestry classification in which accuracy is improved from 68% to 81% by incorporating topographic elevation in the manner outlined.     

Unsupervised classification of SAR images using normalized gamma process mixtures

We propose an image prior for the model-based nonparametric classification of synthetic aperture radar (SAR) images that allows working with infinite number of mixture components. In order to enclose the spatial interactions of the pixel labels, the prior is derived by incorporating a conditional multinomial auto-logistic random field into the Normalized Gamma Process prior. In this way, we obtain an image classification prior that is free from the limitation on the number of classes and includes the smoothing constraint into classification problem. In this model, we introduced a hyper-parameter that can control the preservation of the important classes and the extinction of the weak ones. The recall rates reported on the synthetic and the real TerraSAR-X images show that the proposed model is capable of accurately classifying the pixels. Unlike the existing methods, it applies a simple iterative update scheme without performing a hierarchical clustering strategy. We demonstrate that the estimation accuracy of the proposed method in number of classes outperforms the conventional finite mixture models.     

Change detection method for remote sensing images based on an improved Markov random field

The fixed weights between the center pixel and neighboring pixels are used in the traditional Markov random field for change detection, which will easily cause the overuse of spatial neighborhood information. Besides the traditional label field cannot accurately identify the spatial relations between neighborhood pixels. To solve these problems, this study proposes a change detection method based on an improved MRF. Linear weights are designed for dividing unchanged, uncertain and changed pixels of the difference image, and spatial attraction model is introduced to refine the spatial neighborhood relations, which aims to enhance the accuracy of spatial information in MRF. The experimental results indicate that the proposed method can effectively enhance the accuracy of change detection.     

Tree approximations to Markov random fields

Methods for approximately computing the marginal probability mass functions and means of a Markov random field (MRF) by approximating the lattice by a tree are described. Applied to the a posteriori MRF these methods solve Bayesian spatial pattern classification and image restoration problems. The methods are described, several theoretical results concerning fixed-point problems are proven, and four numerical examples are presented, including comparison with optimal estimators and the iterated conditional mode estimator and including two agricultural optical remote sensing problems     

Using local transition probability models in Markov random fields for forest change detection

Change detection based on the comparison of independently classified images (i.e. post-classification comparison) is well-known to be negatively affected by classification errors of individual maps. Incorporating spatial-temporal contextual information in the classification helps to reduce the classification errors, thus improving change detection results. In this paper, spatial-temporal Markov Random Fields (MRF) models were used to integrate spatial-temporal information with spectral information for multi-temporal classification in an attempt to mitigate the impacts of classification errors on change detection. One important component in spatial-temporal MRF models is the specification of transition probabilities. Traditionally, a global transition probability model is used that assumes spatial stationarity of transition probabilities across an image scene, which may be invalid if areas have varying transition probabilities. By relaxing the stationarity assumption, we developed two local transition probability models to make the transition model locally adaptive to spatially varying transition probabilities. The first model called locally adjusted global transition model adapts to the local variation by multiplying a pixel-wise probability of change with the global transition model. The second model called pixel-wise transition model was developed as a fully local model based on the estimation of the pixel-wise joint probabilities. When applied to the forest change detection in Paraguay, the two local models showed significant improvements in the accuracy of identifying the change from forest to non-forest compared with traditional models. This indicates that the local transition probability models can present temporal information more accurately in change detection algorithms based on spatial-temporal classification of multi-temporal images. The comparison between the two local transition models showed that the fully local model better captured the spatial heterogeneity of the transition probabilities and achieved more stable and consistent results over different regions of a large image scene.     

Spatio-temporal context for robust multitarget tracking

In multitarget tracking, the main challenge is to maintain the correct identity of targets even under occlusions or when differences between the targets are small. The paper proposes a new approach to this problem by incorporating the context information. The context of a target in an image sequence has two components: the spatial context including the local background and nearby targets and the temporal context including all appearances of the targets that have been seen previously. The paper considers both aspects. We propose a new model for multitarget tracking based on the classification of each target against its spatial context. The tracker searches a region similar to the target while avoiding nearby targets. The temporal context is included by integrating the entire history of target appearance based on probabilistic principal component analysis (PPCA). We have developed a new incremental scheme that can learn the full set of PPCA parameters accurately online. The experiments show robust tracking performance under the condition of severe clutter, occlusions, and pose changes     

离散ADMM方法下像素与对象基元协同优化的遥感影像无监督语义分割

语义分割是遥感影像分析中的重要技术之一。现有的方法（如基于深度卷积神经网络的方法等）虽然在语义分割中取得了显著进展,但往往需要大量训练数据。基于图模型的马尔可夫随机场模型（Markov random field model,MRF）提出了一种不依赖训练数据的无监督语义分割思路,可以有效地刻画地物空间关系,并对地物空间分布的统计规律进行建模。但现有的MRF模型方法通常建立在基于像素或对象的单一粒度基元上,难以充分利用影像信息,语义分割效果不佳。针对上述问题,引入交替方向乘子法 （alternative direction method of multiplier,ADMM）并将其离散化,提出了一种像素与对象基元协同的MRF模型无监督语义分割方法（MRF-ADMM）。首先构建像素基元和对象基元两个概率图,其中像素基元概率图用于刻画影像的细节信息,保持语义分割的边界;对象基元概率图用于描述较大范围的空间关系,以应对遥感影像地物内部的高异质性,使分割结果中地物内部具有良好的区域完整性。在模型求解过程中,针对像素和对象基元的特点,提出了一种离散化的ADMM方法,并将其用于两种基元类别标记的传递与更新,实现像素基元细节信息和对象基元区域信息的协同优化。高分二号和航拍影像等不同数据库不同类型遥感影像的语义分割实验结果表明,相较于现有的MRF模型,提出的MRF-ADMM方法能有效地协同不同粒度基元的优点,优化语义分割结果。