A colon video analysis framework for polyp detection

This paper presents an automated video analysis framework for the detection of colonic polyps in optical colonoscopy. Our proposed framework departs from previous methods in that we include spatial frame-based analysis and temporal video analysis using time-course image sequences. We also provide a video quality assessment scheme including two measures of frame quality. We extract colon-specific anatomical features from different image regions using a windowing approach for intraframe spatial analysis. Anatomical features are described using an eigentissue model. We apply a conditional random field to model interframe dependences in tissue types and handle variations in imaging conditions and modalities. We validate our method by comparing our polyp detection results to colonoscopy reports from physicians. Our method displays promising preliminary results and shows strong invariance when applied to both white light and narrow-band video. Our proposed video analysis system can provide objective diagnostic support to physicians by locating polyps during colon cancer screening exams. Furthermore, our system can be used as a cost-effective video annotation solution for the large backlog of existing colonoscopy videos.     

Bayesian Separation of Images Modeled With MRFs Using MCMC

We investigate the source separation problem of random fields within a Bayesian framework. The Bayesian formulation enables the incorporation of prior image models in the estimation of sources. Due to the intractability of the analytical solution, we resort to numerical methods for the joint maximization of the a posteriori distribution of the unknown variables and parameters. We construct the prior densities of pixels using Markov random fields based on a statistical model of the gradient image, and we use a fully Bayesian method with modified-Gibbs sampling. We contrast our work to approximate Bayesian solutions such as iterated conditional modes (ICM) and to non-Bayesian solutions of ICA variety. The performance of the method is tested on synthetic mixtures of texture images and astrophysical images under various noise scenarios. The proposed method is shown to outperform significantly both its approximate Bayesian and non-Bayesian competitors.     

GAFFE: a gaze-attentive fixation finding engine.

The ability to automatically detect visually interesting regions in images has many practical applications, especially in the design of active machine vision and automatic visual surveillance systems. Analysis of the statistics of image features at observers' gaze can provide insights into the mechanisms of fixation selection in humans. Using a foveated analysis framework, we studied the statistics of four low-level local image features: luminance, contrast, and bandpass outputs of both luminance and contrast, and discovered that image patches around human fixations had, on average, higher values of each of these features than image patches selected at random. Contrast-bandpass showed the greatest difference between human and random fixations, followed by luminance-bandpass, RMS contrast, and luminance. Using these measurements, we present a new algorithm that selects image regions as likely candidates for fixation. These regions are shown to correlate well with fixations recorded from human observers.     

一种基于区域Gibbs势能函数的视频运动对象分割算法

提出了一种基于时空联合分析框架的视频对象分割算法,通过改进的分水岭变换对视频图像进行帧内空间区域划分,并根据帧间运动信息和区域的空间特性得到初步的分割掩模;然后建立基于区域的马尔可夫随机场分布模型,并定义对应的Gibbs势能函数,通过迭代条件模式(ICM)方法求解得到最小化能量,从而获得稳定的分割标记场,准确地提取视频对象。实验结果表明,提出的分割算法性能优于欧洲COST211研究组所得到的分割结果。     

Estimating Gaussian Markov random field parameters in anonstationary framework: application to remote sensing imaging

In this paper, we tackle the problem of estimating textural parameters. We do not consider the problem of texture synthesis, but the problem of extracting textural features for tasks such as image segmentation. We take into account nonstationarities occurring in the local mean. We focus on Gaussian Markov random fields for which two estimation methods are proposed, and applied in a nonstationary framework. The first one consists of extracting conditional probabilities and performing a least square approximation. This method is applied to a nonstationary framework, dealing with the piecewise constant local mean. This framework is adapted to practical tasks when discriminating several textures on a single image. The blurring effect affecting edges between two different textures is thus reduced. The second proposed method is based on renormalization theory. Statistics involved only concern variances of Gaussian laws, leading to Cramer-Rao estimators. This method is thus especially robust with respect to the size of sampling. Moreover, nonstationarities of the local mean do not affect results. We then demonstrate that the estimated parameters allow texture discrimination for remote sensing data. The first proposed estimation method is applied to extract urban areas from SPOT images. Since discontinuities of the local mean are taken into account, we obtain an accurate urban areas delineation. Finally, we apply the renormalization based on method to segment ice in polar regions from AVHRR data.     

Conditional Random Field based salient proposal set generation and its application in content aware seam carving

Most of the existing generic object localization algorithms usually give the plausible object locations without taking into consideration the saliency ordering of the proposal set. This paper presents a novel object proposal generation which ranks the key objects according to their saliency score in the proposal pool. First, we formulate a Bayesian framework for generating a probabilistic edgemap which is used to assign a saliency value to the edgelets. A conditional random field is then learnt for edge-labeling by effectively combining the edge features with the relative spatial layout of the edge segments. Lastly, we propose an objectness score for the generated proposal set by analyzing the salient object edge density completely lying within the candidate boxes. Extensive experiments on the benchmark PASCAL VOC 2007 and 2012 datasets demonstrate that the proposed method provides competitive performance against popular generic object detection techniques while using fewer number of proposals. Additionally, we demonstrate the applicability of the generated proposal set for content aware image retargeting.     

A joint inter- and intrascale statistical model for Bayesian wavelet based image denoising

This paper presents a new wavelet-based image denoising method, which extends a "geometrical" Bayesian framework. The new method combines three criteria for distinguishing supposedly useful coefficients from noise: coefficient magnitudes, their evolution across scales and spatial clustering of large coefficients near image edges. These three criteria are combined in a Bayesian framework. The spatial clustering properties are expressed in a prior model. The statistical properties concerning coefficient magnitudes and their evolution across scales are expressed in a joint conditional model. The three main novelties with respect to related approaches are (1) the interscale-ratios of wavelet coefficients are statistically characterized and different local criteria for distinguishing useful coefficients from noise are evaluated, (2) a joint conditional model is introduced, and (3) a novel anisotropic Markov random field prior model is proposed. The results demonstrate an improved denoising performance over related earlier techniques.     

GAL: A global-attributes assisted labeling system for outdoor scenes

An approach that extracts global attributes from outdoor images to facilitate geometric layout labeling is investigated in this work. The proposed Global-attributes Assisted Labeling (GAL) system exploits both local features and global attributes. First, by following a classical method, we use local features to provide initial labels for all super-pixels. Then, we develop a set of techniques to extract global attributes from 2D outdoor images. They include sky lines, ground lines, vanishing lines, etc. Finally, we propose the GAL system that integrates global attributes in the conditional random field (CRF) framework to improve initial labels so as to offer a more robust labeling result. The performance of the proposed GAL system is demonstrated and benchmarked with several state-of-the-art algorithms against a popular outdoor scene layout dataset.     

An EM Approach to MAP Solution of Segmenting Tissue Mixtures: A Numerical Analysis

This work presents an iterative expectation-maximization (EM) approach to the maximum a posteriori (MAP) solution of segmenting tissue mixtures inside each image voxel. Each tissue type is assumed to follow a normal distribution across the field-of-view (FOV). Furthermore, all tissue types are assumed to be independent from each other. Under these assumptions, the summation of all tissue mixtures inside each voxel leads to the image density mean value at that voxel. The summation of all the tissue mixtures' unobservable random processes leads to the observed image density at that voxel, and the observed image density value also follows a normal distribution (image data are observed to follow a normal distribution in many applications). By modeling the underlying tissue distributions as a Markov random field across the FOV, the conditional expectation of the posteriori distribution of the tissue mixtures inside each voxel is determined, given the observed image data and the current-iteration estimation of the tissue mixtures. Estimation of the tissue mixtures at next iteration is computed by maximizing the conditional expectation. The iterative EM approach to a MAP solution is achieved by a finite number of iterations and reasonable initial estimate. This MAP-EM framework provides a theoretical solution to the partial volume effect, which has been a major cause of quantitative imprecision in medical image processing. Numerical analysis demonstrated its potential to estimate tissue mixtures accurately and efficiently.      

Automatic detection of gadolinium-enhancing multiple sclerosis lesions in brain MRI using conditional random fields

Gadolinium-enhancing lesions in brain magnetic resonance imaging of multiple sclerosis (MS) patients are of great interest since they are markers of disease activity. Identification of gadolinium-enhancing lesions is particularly challenging because the vast majority of enhancing voxels are associated with normal structures, particularly blood vessels. Furthermore, these lesions are typically small and in close proximity to vessels. In this paper, we present an automatic, probabilistic framework for segmentation of gadolinium-enhancing lesions in MS using conditional random fields. Our approach, through the integration of different components, encodes different information such as correspondence between the intensities and tissue labels, patterns in the labels, or patterns in the intensities. The proposed algorithm is evaluated on 80 multimodal clinical datasets acquired from relapsing-remitting MS patients in the context of multicenter clinical trials. The experimental results exhibit a sensitivity of 98% with a low false positive lesion count. The performance of the proposed algorithm is also compared to a logistic regression classifier, a support vector machine and a Markov random field approach. The results demonstrate superior performance of the proposed algorithm at successfully detecting all of the gadolinium-enhancing lesions while maintaining a low false positive lesion count.     

A unified framework for MAP estimation in remote sensing image segmentation

A complete framework is proposed for applying the maximum a posteriori (MAP) estimation principle in remote sensing image segmentation. The MAP principle provides an estimate for the segmented image by maximizing the posterior probabilities of the classes defined in the image. The posterior probability can be represented as the product of the class conditional probability (CCP) and the class prior probability (CPP). In this paper, novel supervised algorithms for the CCP and the CPP estimations are proposed which are appropriate for remote sensing images where the estimation process might to be done in high-dimensional spaces. For the CCP, a supervised algorithm which uses the support vector machines (SVM) density estimation approach is proposed. This algorithm uses a novel learning procedure, derived from the main field theory, which avoids the (hard) quadratic optimization problem arising from the traditional formulation of the SVM density estimation. For the CPP estimation, Markov random field (MRF) is a common choice which incorporates contextual and geometrical information in the estimation process. Instead of using predefined values for the parameters of the MRF, an analytical algorithm is proposed which automatically identifies the values of the MRF parameters. The proposed framework is built in an iterative setup which refines the estimated image to get the optimum solution. Experiments using both synthetic and real remote sensing data (multispectral and hyperspectral) show the powerful performance of the proposed framework. The results show that the proposed density estimation algorithm outperforms other algorithms for remote sensing data over a wide range of spectral dimensions. The MRF modeling raises the segmentation accuracy by up to 10% in remote sensing images.     

Initialization of Markov random field clustering of large remote sensing images

Markov random field (MRF) clustering, utilizing both spectral and spatial interpixel dependency information, often improves classification accuracy for remote sensing images, such as multichannel polarimetric synthetic aperture radar (SAR) images. However, it is heavily sensitive to initial conditions such as the choice of the number of clusters and their parameters. In this paper, an initialization scheme for MRF clustering approaches is suggested for remote sensing images. The proposed method derives suitable initial cluster parameters from a set of homogeneous regions, and estimates the number of clusters using the pseudolikelihood information criterion (PLIC). The method works best for an image consisting of many large homogeneous regions, such as agricultural crops areas. It is illustrated using a well-known polarimetric SAR image of Flevoland in the Netherlands. The experiment shows a superior performance compared to several other methods, such as fuzzy C-means and iterated conditional modes (ICM) clustering.     

Image completion using efficient belief propagation via priority scheduling and dynamic pruning.

In this paper, a new exemplar-based framework is presented, which treats image completion, texture synthesis, and image inpainting in a unified manner. In order to be able to avoid the occurrence of visually inconsistent results, we pose all of the above image-editing tasks in the form of a discrete global optimization problem. The objective function of this problem is always well-defined, and corresponds to the energy of a discrete Markov random field (MRF). For efficiently optimizing this MRF, a novel optimization scheme, called priority belief propagation (BP), is then proposed, which carries two very important extensions over the standard BP algorithm: "priority-based message scheduling" and "dynamic label pruning." These two extensions work in cooperation to deal with the intolerable computational cost of BP, which is caused by the huge number of labels associated with our MRF. Moreover, both of our extensions are generic, since they do not rely on the use of domain-specific prior knowledge. They can, therefore, be applied to any MRF, i.e., to a very wide class of problems in image processing and computer vision, thus managing to resolve what is currently considered as one major limitation of the BP algorithm: its inefficiency in handling MRFs with very large discrete state spaces. Experimental results on a wide variety of input images are presented, which demonstrate the effectiveness of our image-completion framework for tasks such as object removal, texture synthesis, text removal, and image inpainting.     

Bayesian estimation of ventricular contours in angiographic images

A method for left ventricular contour determination in digital angiographic images is presented. The problem is formulated in a Bayesian framework, adopting as the estimation criterion the maximum a posterior probability (MAP). The true contour is modeled as a one-dimensional noncausal Gauss-Markov random field and the observed image is described as the superposition of an ideal image (deterministic function of the real contour) with white Gaussian noise. The proposed algorithm estimates simultaneously the contour and the model parameters by implementing an adaptive version of the iterated conditional modes algorithm. The convergence of this scheme is proved and its performance evaluated on both synthetic and real angiographic images. The method exhibits robustness against image artifacts and the contours obtained are considered good by expert clinicians. Being completely data-driven and fast, the proposed algorithm is suitable for routine clinical use.     

Urban scene recognition by graphical model and 3D geometry

This paper proposes a simple and discriminative framework,using graphical model and 3D geometry to understand the diversity of urban scenes with varying viewpoints.Our algorithm constructs a conditional random field (CRF) network using over-segmented superpixels and learns the appearance model from different set of features for specific classes of our interest.Also,we introduce a training algorithm to learn a model for edge potential among these superpixel areas based on their feature difference.The proposed algorithm gives competitive and visually pleasing results for urban scene segmentation.We show the inference from our trained network improves the class labeling performance compared to the result when using the appearance model solely.     

基于EM参数“软”估计的SAR图像分割

借助EM算法和模糊理论,提出了一种基于参数"软"估计和Markov随机场的SAR图像无监督分割方法。首先利用多维空间的EM算法估计随机场的模型参数,并根据随机场模型参数分别计算观测数据的条件概率和标记图像的先验概率,继而根据最大后验概率准则将图像分成具有相似统计特性的同质区域,重复以上步骤直至收敛。通过与传统的参数"硬"估计分割算法的实验比较,该算法能更好保持图像边缘细节,区域连通性更好。     

Robust Multithreaded Object Tracker through Occlusions for Spatial Augmented Reality

A spatial augmented reality (SAR) system enables a virtual image to be projected onto the surface of a real-world object and the user to intuitively control the image using a tangible interface. However, occlusions frequently occur, such as a sudden change in the lighting environment or the generation of obstacles. We propose a robust object tracker based on a multithreaded system, which can track an object robustly through occlusions. Our multithreaded tracker is divided into two threads: the detection thread detects distinctive features in a frame-to-frame manner, and the tracking thread tracks features periodically using an optical-flow-based tracking method. Consequently, although the speed of the detection thread is considerably slow, we achieve real-time performance owing to the multithreaded configuration. Moreover, the proposed outlier filtering automatically updates a random sample consensus distance threshold for eliminating outliers according to environmental changes. Experimental results show that our approach tracks an object robustly in real-time in an SAR environment where there are frequent occlusions occurring from augmented projection images.     

Image forgery detection algorithm based on U-shaped detection network

Aiming at the defects of traditional image tampering detection algorithm relying on single image attribute,low applicability and current high time-complexity detection algorithm based on deep learning,an U-shaped detection network image forgery detection algorithm was proposed.Firstly,the multi-stage feature information in the image by using the continuous convolution layers and the max-pooling layers was extracted by U-shaped detection network,and then the obtained feature information to the resolution of the input image through the upsampling operation was restored.At the same time,in order to ensure higher detection accuracy while extracting high-level semantic information of the image,the output features of each stage in U-shaped detection network would be merged with the corresponding output features through the upsampling layer.Further the hidden feature information between tampered and un-tampered regions in the image upon the characteristics of the general network was explored by U-shaped detection network,which could be realized quickly by using its end-to-end network structure and extracting the attributes of strong correlation information among image contexts that could ensure high-precision detection results.Finally,the conditional random field was used to optimize the output of the U-shaped detection network to obtain a more exact detection results.The experimental results show that the proposed algorithm outperforms those traditional forgery detection algorithms based on single image attribute and the current deep learning-based detection algorithm,and has good robustness.     

Hot spot detection based on feature space representation of visual search

This paper presents a new framework for capturing intrinsic visual search behavior of different observers in image understanding by analysing saccadic eye movements in feature space. The method is based on the information theory for identifying salient image features based on which visual search is performed. We demonstrate how to obtain feature space fixation density functions that are normalized to the image content along the scan paths. This allows a reliable identification of salient image features that can be mapped back to spatial space for highlighting regions of interest and attention selection. A two-color conjunction search experiment has been implemented to illustrate the theoretical framework of the proposed method including feature selection, hot spot detection, and back-projection. The practical value of the method is demonstrated with computed tomography image of centrilobular emphysema, and we discuss how the proposed framework can be used as a basis for decision support in medical image understanding.