Detection and correction of specular reflections for automatic surgical tool segmentation in thoracoscopic images

This paper presents an algorithm that automatically detects and corrects specular reflections in thoracoscopic images and its application in the context of automatic segmentation of surgical tools. The detection is done by isolating the spike component of the specular reflection which is characterized by a bump at the end of the histogram of thoracoscopic images. The specular lobe is then extracted in the neighborhood of the spike component of the reflection. The result is a mask of the reflections positions in the image. Thereafter, the image is corrected using Oliveira et al.’s digital inpainting method. The automatic segmentation of surgical tools using the corrected images is then demonstrated. Results of the segmentation with and without the specular reflection elimination technique are compared. Moreover, 108 images extracted from 5 different surgeries performed under various conditions were considered to demonstrate the effectiveness of the proposed technique.     

Multiresolution color image segmentation

Image segmentation is the process by which an original image is partitioned into some homogeneous regions. In this paper, a novel multiresolution color image segmentation (MCIS) algorithm which uses Markov random fields (MRF's) is proposed. The proposed approach is a relaxation process that converges to the MAP (maximum a posteriori) estimate of the segmentation. The quadtree structure is used to implement the multiresolution framework, and the simulated annealing technique is employed to control the splitting and merging of nodes so as to minimize an energy function and therefore, maximize the MAP estimate. The multiresolution scheme enables the use of different dissimilarity measures at different resolution levels. Consequently, the proposed algorithm is noise resistant. Since the global clustering information of the image is required in the proposed approach, the scale space filter (SSF) is employed as the first step. The multiresolution approach is used to refine the segmentation. Experimental results of both the synthesized and real images are very encouraging. In order to evaluate experimental results of both synthesized images and real images quantitatively, a new evaluation criterion is proposed and developed     

Rapid classification of specular and diffuse reflection from image velocities

We propose a method for rapidly classifying surface reflectance directly from the output of spatio-temporal filters applied to an image sequence of rotating objects. Using image data from only a single frame, we compute histograms of image velocities and classify these as being generated by a specular or a diffusely reflecting object. Exploiting characteristics of material-specific image velocities we show that our classification approach can predict the reflectance of novel 3D objects, as well as human perception.     

Removal of specular reflections from image sequences using feature correspondences

The presence of specular highlights can hide underlying features of a scene within an image and can be problematic in many application scenarios. In particular, this poses a significant challenge for applications where image stitching is used to create a single static image of a scene from inspection footage of pipes, gas tubes, train tracks and concrete structures. Furthermore, they can hide small defects in the images causing them to be missed during inspection. We present a method which exploits additional information in neighbouring frames from video footage to reduce specularity from each frame. The technique first automatically determines frames which contain overlapping regions before the relationship that exists between them is exploited in order to suppress the effects of specular reflections. This results in an image that is free from specular highlights provided there is at least one frame present in the sequence where a given pixel is present in a diffuse form. The method is shown to work well on greyscale as well as colour images and effectively reduces specularity and significantly improves the quality of the stitched image, even in the presence of noise. While applied to the challenge of reducing specularity in inspection videos, the method improves upon the state-of-the-art in specularity removal, and its applications are wide-ranging as a general purpose pre-processing tool.     

一种改进的彩色图像分割算法

文中算法对传统的基于最大隶属度的彩色图像分割算法进行了改进。传统的基于最大隶属度原则的分割方法根据彩色直方图中的色彩矢量来确定目标和背景色模糊集,而绘制彩色直方图时比较繁琐。为了克服这一缺点,先绘制彩色图像对应的灰度直方图。对应于每一个灰度级,取属于该灰度级的像素中的一个像素的颜色来代表这一灰度级所对应的颜色,并保存其R、G、B值。据此建立一组色彩模糊集。计算图像中的所有色彩在各个模糊集中的隶属度,并基于最大隶属度原则确定色彩的归属。建立一种基于最大隶属度原则的神经网络,实现对彩色图像快速有效的分割。通过灰度直方图判断目标和背景色时的误差很小,并不影响判断。在建立色彩模糊集和计算隶属度实现色彩分类的时候都是根据像素色彩的R、G、B值来实现的,因而没有颜色损失。实验结果证明,该算法对原有的基于最大隶属度原则的彩色图像分割方法进行了改进,速度快,效果好。     

一种基于区域的彩色图像分割方法

介绍了一种基于区域的彩色图像分割方法。该方法首先提取图像像素点的颜色、纹理等特征,然后采用Gaussian混合模型,通过EM算法学习,根据提出的选择最佳高斯混合模型参数K的准则,确定K,利用图像像素点特征的相似度在特征空间中粗略的将像素点划分为不同的组,最后在各个组内依据其位置信息对图像再进一步划分,得到图像的区域分割。实验结果表明,该分割方法具有较好的分割性能。     

SOM and fuzzy based color image segmentation

Spatial information enhances the quality of clustering which is not utilized in the conventional FCM. Normally fuzzy c-mean (FCM) algorithm is not used for color image segmentation and also it is not robust against noise. In this paper, we presented a modified version of fuzzy c-means (FCM) algorithm that incorporates spatial information into the membership function for clustering of color images A progressive technique based on SOM is used to automatically find the number of optimal clusters. The results show that our technique outperforms state-of-the art methods.     

改进的MRF彩色图像分割算法

为了解决彩色图像中实现不同目标、不同背景的分割问题,提出了一种基于RGB色彩统计分布的MRF分割算法。在MRF模型的基础上,设定了MRF模型的关键参数并对相关公式进行了推导。对RGB颜色分布模型进行了介绍,重写了能量函数,大大降低了算法的计算难度。由于该算法在RGB彩色空间下充分表达了像素值的信息,使得该算法在分割准确性、适应性及快速性上均有提高。通过与其他算法在同等环境下进行分割实验定量对比,验证了该算法的有效性。     

一种织物彩色图像的分割算法

提出了一种针对织物彩色图像的分割算法。该算法在颜色聚类的基础上,把每种颜色用 不同的序号来表示,生成一个与原图同大小的序号矩阵,然后根据矩阵中序号的分布进行初始分割并 产生种子区域,再把初始分割未能确定的像素指派到种子区域中,最后根据颜色信息合并过分割的区 域,并对区域边缘进行平滑,得到最终的分割结果。实验表明,该算法能较有效地实现各类彩色织物 图像的分割。     

Pixel clustering for color image segmentation

Programming and Computer Software - Image segmentation using a hierarchical sequence of piecewise constant approximations that minimally differ from the original image in terms of the total squared...     

融合多颜色空间分量的自适应彩色图像分割

提出了一种新的简单有效的融合多颜色分量的分割方法,首先在六个不同的颜色空间中选择最佳的待分割颜色分量,然后应用直方图和空间模糊C均值（SFCM）技术对不同颜色分量进行自适应初始分割,最后融合分割结果并进行区域合并。利用该算法在Berkeley图像库上进行了大量实验,实验结果表明,与当前一些经典分割算法Mean-shift、FCR、CTM等相比,利用该方法能够获得更好的分割结果以及更优的性能指标。     

语义分割和HSV色彩空间引导的低光照图像增强

目的 低光照图像增强是图像处理中的基本任务之一。虽然已经提出了各种方法,但它们往往无法在视觉上产生吸引人的结果,这些图像存在细节不清晰、对比度不高和色彩失真等问题,同时也对后续目标检测、语义分割等任务有不利影响。针对上述问题,提出一种语义分割和HSV（hue,saturation and value）色彩空间引导的低光照图像增强方法。方法 首先提出一个迭代图像增强网络,逐步学习低光照图像与增强图像之间像素级的最佳映射,同时为了在增强过程中保留语义信息,引入一个无监督的语义分割网络并计算语义损失,该网络不需要昂贵的分割注释。为了进一步解决色彩失真问题,在训练时利用HSV色彩空间设计HSV损失;为了解决低光照图像增强中出现细节不清晰的问题,设计了空间一致性损失,使增强图像与对应的低光照图像尽可能细节一致。最终,本文的总损失函数由5个损失函数组成。结果 将本文方法与LIME（low-light image enhancement）、RetinexNet（deep retinex decomposition）、EnlightenGAN（deep light enhancement using generative adversarial networks）、Zero-DCE（zero-reference deep curve estimation）和SGZ（semantic-guided zero-shot learning）5种方法进行了比较。在峰值信噪比（peak signal-to noise ratio,PSNR）上,本文方法平均比Zero-DCE（zero-reference deep curve estimation）提高了0.32dB;在自然图像质量评价（natural image quality evaluation,NIQE）方面,本文方法比EnlightenGAN提高了6%。从主观上看,本文方法具有更好的视觉效果。结论 本文所提出的低光照图像增强方法能有效解决细节不清晰、色彩失真等问题,具有一定的应用价值。     

示温漆彩色图像分割与温度识别系统

通过对示温漆颜色温度特性的分析,提出一种基于标记区域生长的彩色图像分割方法,并给出温度识别的方案。试验结果显示该方法对示温漆图像分割及温度识别具有较好的效果。     

Detection and localization of specular surfaces using image motion cues

Successful identification of specularities in an image can be crucial for an artificial vision system when extracting the semantic content of an image or while interacting with the environment. We developed an algorithm that relies on scale and rotation invariant feature extraction techniques and uses motion cues to detect and localize specular surfaces. Appearance change in feature vectors is used to quantify the appearance distortion on specular surfaces, which has previously been shown to be a powerful indicator for specularity (Doerschner et al. in Curr Biol, 2011). The algorithm combines epipolar deviations (Swaminathan et al. in Lect Notes Comput Sci 2350:508–523, 2002) and appearance distortion, and succeeds in localizing specular objects in computer-rendered and real scenes, across a wide range of camera motions and speeds, object sizes and shapes, and performs well under image noise and blur conditions.     

Robust image segmentation against complex color distribution

Color distribution is the most effective cue that is widely adopted in previous interactive image segmentation methods. However, it also may introduce additional errors in some situations, for example, when the foreground and background have similar colors. To address this problem, this paper proposes a novel method to learn the segmentation likelihoods. The proposed method is designed for high reliability, for which purpose it may choose to discard some unreliable likelihoods that may cause segmentation error. The reliability of likelihoods is estimated in a few Expectation–Maximization iterations. In each iteration, a novel multi-class transductive learning algorithm, namely, the Constrained Mapping, is proposed to learn likelihoods and identify unreliable likelihoods simultaneously. The resulting likelihoods then can be used as the input of any segmentation methods to improve their robustness. Experiments show that the proposed method is an effective way to improve both segmentation quality and efficiency, especially when the input image has complex color distribution.     

Interactive color image segmentation with linear programming

Image segmentation is an important and fundamental task for image and vision understanding. This paper describes a linear programming (LP) approach for segmenting a color image into multiple regions. Compared with the recently proposed semi-definite programming (SDP)-based approach, our approach has a simpler mathematical formulation, and a far lower computational complexity. In particular, to segment an image of M × N pixels into k classes, our method requires only O((M N k) ^m ) complexity—a sharp contrast to the complexity of O((M N k)²ⁿ ) if the SDP method is adopted, where m and n are the polynomial complexity of the corresponding LP solver and SDP solver, respectively (in general we have m≪ n). Such a significant reduction in computation readily enables our algorithm to process color images of reasonable sizes. For example, while the existing SDP relaxation algorithm is only able to segment a toy-size image of, e.g., 10 × 10 to 30 × 30 pixels in hours time, our algorithm can process larger color image of, say, 100 × 100 to 500 × 500 image in much shorter time.     

Removal of achromatic reflections from a single color image

In this paper we consider the problem of removing achromatic reflections from a picture of a scene taken through a semi-transparent medium, assuming that the reflection pattern is due to a light source or another object located in front of the object of interest. While other works assume the availability of multiple observations, we consider the more challenging problem of having as data a single color image. We suppose a data model where the virtual reflected image combines additively with the real transmitted image of the object, through unknown coefficients. This highly underdetermined problem is handled by means of a blind estimation technique that exploits the strict dependence of the gradients of the three color channels of the ideal image, and their independence from the gradient of the grayscale reflected image. The model parameters are estimated through independent component analysis, and then the component images are estimated through a regularization technique. The whole algorithm is very fast, and its performance is quantitatively evaluated on numerically generated images, and qualitatively tested on real images.     

基于半监督的超像素谱聚类彩色图像分割算法

近年来谱聚类算法被广泛应用于图像分割领域,而相似性矩阵的构造是谱聚类算法的关键步骤。 针对传统谱聚类算法计算复杂度高难以应用到大规模图像分割处理的问题,提出了基于半监督的超像素谱聚类彩色图像分割算法。该算法利用超像素将彩色图像进行预分割,利用用户提供的少量标记信息构造预分割区域的基于半监督的模糊相似性测度,利用该相似性测度构造预分隔区域的相似性矩阵并通过规范切图谱划分准则对预分割区域进行划分得到最终的图像分割结果。由于少量标记信息和模糊理论的引入,提高了传统谱聚类的分割性能,对比实验也表明该算法在分割效果和计算复杂度上都有较大的改善。