Bayes smoothing algorithms for segmentation of binary images modeled by markov random fields

A new image segmentation algorithm is presented, based on recursive Bayes smoothing of images modeled by Markov random fields and corrupted by independent additive noise. The Bayes smoothing algorithm yields the a posteriori distribution of the scene value at each pixel, given the total noisy image, in a recursive way. The a posteriori distribution together with a criterion of optimality then determine a Bayes estimate of the scene. The algorithm presented is an extension of a 1-D Bayes smoothing algorithm to 2-D and it gives the optimum Bayes estimate for the scene value at each pixel. Computational concerns in 2-D, however, necessitate certain simplifying assumptions on the model and approximations on the implementation of the algorithm. In particular, the scene (noiseless image) is modeled as a Markov mesh random field, a special class of Markov random fields, and the Bayes smoothing algorithm is applied on overlapping strips (horizontal/vertical) of the image consisting of several rows (columns). It is assumed that the signal (scene values) vector sequence along the strip is a vector Markov chain. Since signal correlation in one of the dimensions is not fully used along the edges of the strip, estimates are generated only along the middle sections of the strips. The overlapping strips are chosen such that the union of the middle sections of the strips gives the whole image. The Bayes smoothing algorithm presented here is valid for scene random fields consisting of multilevel (discrete) or continuous random variables.     

A Feature-based Approach for Dense Segmentation and Estimation of Large Disparity Motion

We present a novel framework for motion segmentation that combines the concepts of layer-based methods and feature-based motion estimation. We estimate the initial correspondences by comparing vectors of filter outputs at interest points, from which we compute candidate scene relations via random sampling of minimal subsets of correspondences. We achieve a dense, piecewise smooth assignment of pixels to motion layers using a fast approximate graphcut algorithm based on a Markov random field formulation. We demonstrate our approach on image pairs containing large inter-frame motion and partial occlusion. The approach is efficient and it successfully segments scenes with inter-frame disparities previously beyond the scope of layer-based motion segmentation methods. We also present an extension that accounts for the case of non-planar motion, in which we use our planar motion segmentation results as an initialization for a regularized Thin Plate Spline fit. In addition, we present applications of our method to automatic object removal and to structure from motion.     

Image flow segmentation and estimation by constraint lineclustering

Image flow is the velocity field in the image plane caused by the motion of the observer, objects in the scene, or apparent motion, and can contain discontinuities due to object occlusion in the scene. An algorithm that can estimate the image flow velocity field when there are discontinuities due to occlusions is described. The constraint line clustering algorithm uses a statistical test to estimate the image flow velocity field in the presence of step discontinuities in the image irradiance or velocity field. Particular emphasis is placed on motion estimation and segmentation in situations such as random dot patterns where motion is the only cue to segmentation. Experimental results on a demanding synthetic test case and a real image are presented. A smoothing algorithm for improving the velocity field estimate is also described. The smoothing algorithm constructs a smooth estimate of the velocity field by approximating a surface between step discontinuities. It is noted that the velocity field estimate can be improved using surface reconstruction between velocity field boundaries     

On the Spatial Statistics of Optical Flow

We present an analysis of the spatial and temporal statistics of “natural” optical flow fields and a novel flow algorithm that exploits their spatial statistics. Training flow fields are constructed using range images of natural scenes and 3D camera motions recovered from hand-held and car-mounted video sequences. A detailed analysis of optical flow statistics in natural scenes is presented and machine learning methods are developed to learn a Markov random field model of optical flow. The prior probability of a flow field is formulated as a Field-of-Experts model that captures the spatial statistics in overlapping patches and is trained using contrastive divergence. This new optical flow prior is compared with previous robust priors and is incorporated into a recent, accurate algorithm for dense optical flow computation. Experiments with natural and synthetic sequences illustrate how the learned optical flow prior quantitatively improves flow accuracy and how it captures the rich spatial structure found in natural scene motion.     

Motion segmentation and qualitative dynamic scene analysis from an image sequence

This article deals with analysis of the dynamic content of a scene from an image sequence irrespective of the static or dynamic nature of the camera. The tasks involved can be the detection of moving objects in a scene observed by a mobile camera, or the identification of the movements of some relevant components of the scene relatively to the camera. This problem basically requires a motion-based segmentation step. We present a motion-based segmentation method relying on 2-D affine motion models and a statistical regularization approach which ensures stable motion-based partitions. This can be done without the explicit estimation of optic flow fields. Besides these partitions are linked in time. Therefore, the motion interpretation process can be performed on more than two successive frames. The ability to follow a given coherently moving region within an interval of several images of the sequence makes the interpretation process more robust and more comprehensive. Identification of the kinematic components of the scene is induced from an intermediate layer accomplishing a generic qualitative motion labeling. No 3-D measurements are required. Results obtained on several real-image sequences corresponding to complex outdoor situations are reported.     

Multiresolution video object extraction fitted to scalable wavelet-based object coding

To enable content based functionalities in video processing algorithms, decomposition of scenes into semantic objects is necessary. A semi-automatic Markov random field based multiresolution algorithm is presented for video object extraction in a complex scene. In the first frame, spatial segmentation and user intervention determine objects of interest. The specified objects are subsequently tracked in successive frames and newly appeared objects/regions are also detected. The video object extraction algorithm includes discrete wavelet transform decomposition multiresolution Markov random field (MRF)-based spatial segmentation with emphasis on border smoothness at different resolutions, and an MRF-based backward region classification that determines the tracked objects in the scene. Finally, a motion constraint, embedded in the region classifier, determines the newly appeared objects/regions and completes the proposed algorithm towards an efficient video segmentation algorithm. The results are applicable for generic segmentation applications, however the proposed multiresolution video segmentation algorithm supports scalable object-based wavelet coding in particular. Moreover, compared to traditional object extraction algorithms, it produces smoother and more visually pleasing shape masks at different resolutions. The proposed effective multiresolution video object extraction method allows for larger motion, better noise tolerance and less computational complexity     

基于证据马尔可夫随机场模型的图像分割

图像分割是计算机视觉中的经典问题,在许多领域都有重要应用.由于图像信息存在不确定性,难以获得精确的分割结果,为应对图像分割中的不确定性问题,将证据理论这一不确定性建模与推理工具与马尔可夫随机场相结合,提出证据马尔可夫随机场(EMRF)模型,并基于此提出新的图像分割算法.EMRF利用证据标号场描述像素标号的含混性,以证据距离描述相邻像素间的标号关系,利用条件迭代模型(ICM)算法进行优化.实验结果表明,EMRF相较于传统马尔可夫随机场、模糊马尔可夫随机场和传统的基于证据理论的方法,能获得更好的分割效果.     

Optic flow field segmentation and motion estimation using a robustgenetic partitioning algorithm

Optic flow motion analysis represents an important family of visual information processing techniques in computer vision. Segmenting an optic flow field into coherent motion groups and estimating each underlying motion is a very challenging task when the optic flow field is projected from a scene of several independently moving objects. The problem is further complicated if the optic flow data are noisy and partially incorrect. In this paper, the authors present a novel framework for determining such optic flow fields by combining the conventional robust estimation with a modified genetic algorithm. The baseline model used in the development is a linear optic flow motion algorithm due to its computational simplicity. The statistical properties of the generalized linear regression (GLR) model are thoroughly explored and the sensitivity of the motion estimates toward data noise is quantitatively established. Conventional robust estimators are then incorporated into the linear regression model to suppress a small percentage of gross data errors or outliers. However, segmenting an optic flow field consisting of a large portion of incorrect data or multiple motion groups requires a very high robustness that is unattainable by the conventional robust estimators. To solve this problem, the authors propose a genetic partitioning algorithm that elegantly combines the robust estimation with the genetic algorithm by a bridging genetic operator called self-adaptation     

基于改进马尔可夫随机场的钢轨缺陷分割

为提高钢轨缺陷分割对噪声的鲁棒性,提出一种基于改进马尔可夫随机场(MRF)的钢轨缺陷分割方法。利用背景差分法对灰度进行预处理,消除灰度分布不均的干扰。对模糊if-then规则的前提部分采用马尔可夫随机场来利用图像中的空间约束,结果部分指定像素距离图算法,通过使用马尔可夫随机场(MRF)在相邻像素图像之间并入局部空间信息,推导出新的自适应模糊集和MRF相结合的钢轨表面缺陷自动分割方法。建立标准的FCM、GMM和该方法的钢轨缺陷分割对比实验,验证了算法的有效性和优越性。     

基于混合高斯模型MRF场的CT图像分割

提出了一种基于混合高斯模型的马尔可夫随机场CT图像分割方法.此方法根据工业CT图像的特点,建立混合高斯逼近的图像灰度统计模型;用混合高斯模型作为Markov随机场的先验模型,提出混合高斯Markov随机场分割模型.实验表明,该方法较单高斯模型有很大的改善,对工业CT图像分割效果好.     

Unsupervised statistical segmentation of nonstationary images using triplet Markov fields

Recent developments in statistical theory and associated computational techniques have opened new avenues for image modeling as well as for image segmentation techniques. Thus, a host of models have been proposed and the ones which have probably received considerable attention are the hidden Markov fields (HMF) models. This is due to their simplicity of handling and their potential for providing improved image quality. Although these models provide satisfying results in the stationary case, they can fail in the nonstationary one. In this paper, we tackle the problem of modeling a nonstationary hidden random field and its effect on the unsupervised statistical image segmentation. We propose an original approach, based on the recent triplet Markov field (TMF) model, which enables one to deal with nonstationary class fields. Moreover, the noise can be correlated and possibly non-Gaussian. An original parameter estimation method which uses the Pearson system to find the natures of the noise margins, which can vary with the class, is also proposed and used to perform unsupervised segmentation of such images. Experiments indicate that the new model and related processing algorithm can improve the results obtained with the classical ones.     

Dynamic Shape and Appearance Modeling via Moving and Deforming Layers

We propose a model of the shape, motion and appearance of a scene, seen through a sequence of images, that captures occlusions, scene deformations, unconstrained viewpoint variations and changes in its radiance. This model is based on a collection of overlapping layers that can move and deform, each supporting an intensity function that can change over time. We discuss the generality and limitations of this model in relation to existing ones such as traditional optical flow or motion segmentation, layers, deformable templates and deformotion. We then illustrate how this model can be used for inference of shape, motion, deformation and appearance of the scene from a collection of images. The layering structure allows for automatic inpainting of partially occluded regions. We illustrate the model on synthetic and real sequences where existing schemes fail, and show how suitable choices of constants in the model yield existing schemes, from optical flow to motion segmentation and inpainting.     

A review and evaluation of methods estimating ego-motion

If a visual observer moves through an environment, the patterns of light that impinge its retina vary leading to changes in sensed brightness. Spatial shifts of brightness patterns in the 2D image over time are called optic flow. In contrast to optic flow visual motion fields denote the displacement of 3D scene points projected onto the camera’s sensor surface. For translational and rotational movement through a rigid scene parametric models of visual motion fields have been defined. Besides ego-motion these models provide access to relative depth, and both ego-motion and depth information is useful for visual navigation.In the past 30 years methods for ego-motion estimation based on models of visual motion fields have been developed. In this review we identify five core optimization constraints which are used by 13 methods together with different optimization techniques.¹ In the literature methods for ego-motion estimation typically have been evaluated by using an error measure which tests only a specific ego-motion. Furthermore, most simulation studies used only a Gaussian noise model. Unlike, we test multiple types and instances of ego-motion. One type is a fixating ego-motion, another type is a curve-linear ego-motion. Based on simulations we study properties like statistical bias, consistency, variability of depths, and the robustness of the methods with respect to a Gaussian or outlier noise model. In order to achieve an improvement of estimates for noisy visual motion fields, part of the 13 methods are combined with techniques for robust estimation like m-functions or RANSAC. Furthermore, a realistic scenario of a stereo image sequence has been generated and used to evaluate methods of ego-motion estimation provided by estimated optic flow and depth information.     

Markov random fields for catadioptric image processing

Images obtained with catadioptric sensors contain significant deformations which prevent the direct use of classical image treatments. Thus, Markov random fields (MRF) whose usefulness is now obvious for projective image processing, cannot be used directly on catadioptric images because of the inadequacy of the neighborhood. In this paper, we propose to define a new neighborhood for MRF by using the equivalence theorem developed for central catadioptric sensors. We show the importance of this adaptation for segmentation, image restoration and motion detection.     

Estimating optical flow in segmented images using variable-orderparametric models with local deformations

This paper presents a new model for estimating optical flow based on the motion of planar regions plus local deformations. The approach exploits brightness information to organize and constrain the interpretation of the motion by using segmented regions of piecewise smooth brightness to hypothesize planar regions in the scene. Parametric flow models are estimated in these regions in a two step process which first computes a coarse fit and then estimates the appropriate parametrization of the motion of the region. The initial fit is refined using a generalization of the standard area-based regression approaches. Since the assumption of planarity is likely to be violated, we allow local deformations from the planar assumption in the same spirit as physically-based approaches which model shape using coarse parametric models plus local deformations. This parametric plus deformation model exploits the strong constraints of parametric approaches while retaining the adaptive nature of regularization approaches. Experimental results on a variety of images model produces accurate flow estimates while the incorporation of brightness segmentation boundaries     

Bayesian estimation of motion vector fields

A stochastic approach to the estimation of 2D motion vector fields from time-varying images is presented. The formulation involves the specification of a deterministic structural model along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vector-binary Markov random fields. Two estimation criteria are studied. In the maximum a posteriori probability (MAP) estimation, the a posteriori probability of motion given data is maximized, whereas in the minimum expected cost (MEC) estimation, the expectation of a certain cost function is minimized. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs sampler. Two versions are developed: one for a discrete state space and the other for a continuous state space. The MAP estimation is incorporated into a hierarchical environment to deal efficiently with large displacements     

Motion field and optical flow: qualitative properties

It is shown that the motion field the 2-D vector field which is the perspective projection on the image plane of the 3-D velocity field of a moving scene, and the optical flow, defined as the estimate of the motion field which can be derived from the first-order variation of the image brightness pattern, are in general different, unless special conditions are satisfied. Therefore, dense optical flow is often ill-suited for computing structure from motion and for reconstructing the 3-D velocity field by algorithms which require a locally accurate estimate of the motion field. A different use of the optical flow is suggested. It is shown that the (smoothed) optical flow and the motion field can be interpreted as vector fields tangent to flows of planar dynamical systems. Stable qualitative properties of the motion field, which give useful informations about the 3-D velocity field and the 3-D structure of the scene, usually can be obtained from the optical flow. The idea is supported by results from the theory of structural stability of dynamical systems     

Registration and interactive planar segmentation for stereo images of polyhedral scenes

We introduce a two-step iterative segmentation and registration method to find coplanar surfaces among stereo images of a polyhedral environment. The novelties of this paper are: (i) to propose a user-defined initialization easing the image matching and segmentation, (ii) to incorporate color appearance and planar projection information into a Bayesian segmentation scheme, and (iii) to add consistency to the projective transformations related to the polyhedral structure of the scenes. The method utilizes an assisted Bayesian color segmentation scheme. The initial user-assisted segmentation is used to define search regions for planar homography image registration. The two reliable methods cooperate to obtain probabilities for coplanar regions with similar color information that are used to get a new segmentation by means of quadratic Markov measure fields (QMMF). We search for the best regions by iterating both steps: registration and segmentation.     

形态小波域多尺度马尔可夫模型在纹理图像分割中的应用

针对图像分割中小波域多尺度马尔可夫模型(MRMRF-W)无法有效描述图像非线性特征,提出了一种在形态小波域下的多尺度MRF模型（MRMRF-MW）,实现纹理图像分割。该模型结合了形态小波和MRF各自的优势,能够对图像进行非线性多尺度分解,并在各尺度上进行空间关系建模。通过对两个纹理图像库（Brodatz纹理库、Prague纹理库）中图像的分割实验,验证了该模型的有效性。     

Fuzzy Markov random fields versus chains for multispectral image segmentation

This paper deals with a comparison of recent statistical models based on fuzzy Markov random fields and chains for multispectral image segmentation. The fuzzy scheme takes into account discrete and continuous classes which model the imprecision of the hidden data. In this framework, we assume the dependence between bands and we express the general model for the covariance matrix. A fuzzy Markov chain model is developed in an unsupervised way. This method is compared with the fuzzy Markovian field model previously proposed by one of the authors. The segmentation task is processed with Bayesian tools, such as the well-known MPM (mode of posterior marginals) criterion. Our goal is to compare the robustness and rapidity for both methods (fuzzy Markov fields versus fuzzy Markov chains). Indeed, such fuzzy-based procedures seem to be a good answer, e.g., for astronomical observations when the patterns present diffuse structures. Moreover, these approaches allow us to process missing data in one or several spectral bands which correspond to specific situations in astronomy. To validate both models, we perform and compare the segmentation on synthetic images and raw multispectral astronomical data