Segmentation of confocal microscope images of cell nuclei in thick tissue sections

Segmentation of intact cell nuclei from three-dimensional (3D) images of thick tissue sections is an important basic capability necessary for many biological research studies. However, segmentation is often difficult because of the tight clustering of nuclei in many specimen types. We present a 3D segmentation approach that combines the recognition capabilities of the human visual system with the efficiency of automatic image analysis algorithms. The approach first uses automatic algorithms to separate the 3D image into regions of fluorescence-stained nuclei and unstained background. This includes a novel step, based on the Hough transform and an automatic focusing algorithm to estimate the size of nuclei. Then, using an interactive display, each nuclear region is shown to the analyst, who classifies it as either an individual nucleus, a cluster of multiple nuclei, partial nucleus or debris. Next, automatic image analysis based on morphological reconstruction and the watershed algorithm divides clusters into smaller objects, which are reclassified by the analyst. Once no more clusters remain, the analyst indicates which partial nuclei should be joined to form complete nuclei. The approach was assessed by calculating the fraction of correctly segmented nuclei for a variety of tissue types: Caenorhabditis elegans embryos (839 correct out of a total of 848), normal human skin (343/362), benign human breast tissue (492/525), a human breast cancer cell line grown as a xenograft in mice (425/479) and invasive human breast carcinoma (260/335). Furthermore, due to the analyst's involvement in the segmentation process, it is always known which nuclei in a population are correctly segmented and which not, assuming that the analyst's visual judgement is correct.     

Overlapping nuclei segmentation based on Bayesian networks and stepwise merging strategy

This paper presents a new approach to the segmentation of fluorescence in situ hybridization images. First, to segment the cell nuclei from the background, a threshold is estimated using a Gaussian mixture model and maximizing the likelihood function of the grey values for the cell images. After the nuclei segmentation, the overlapping and isolated nuclei are classified to facilitate a more accurate nuclei analysis. To do this, the morphological features of the nuclei, such their compactness, smoothness and moments, are extracted from training data to generate three probability distribution functions that are then applied to a Bayesian network as evidence. Following the nuclei classification, the overlapping nuclei are segmented into isolated nuclei using an intensity gradient transform and watershed algorithm. A new stepwise merging strategy is also proposed to merge fragments into a major nucleus. Experimental results using fluorescence in situ hybridization images confirm that the proposed system produced better segmentation results when compared to previous methods, because of the nuclei classification before separating the overlapping nuclei.     

Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections

We present a region‐based segmentation method in which seeds representing both object and background pixels are created by combining morphological filtering of both the original image and the gradient magnitude of the image. The seeds are then used as starting points for watershed segmentation of the gradient magnitude image. The fully automatic seeding is done in a generous fashion, so that at least one seed will be set in each foreground object. If more than one seed is placed in a single object, the watershed segmentation will lead to an initial over‐segmentation, i.e. a boundary is created where there is no strong edge. Thus, the result of the initial segmentation is further refined by merging based on the gradient magnitude along the boundary separating neighbouring objects. This step also makes it easy to remove objects with poor contrast. As a final step, clusters of nuclei are separated, based on the shape of the cluster. The number of input parameters to the full segmentation procedure is only five. These parameters can be set manually using a test image and thereafter be used on a large number of images created under similar imaging conditions. This automated system was verified by comparison with manual counts from the same image fields. About 90% correct segmentation was achieved for two‐ as well as three‐dimensional images.     

Empirical gradient threshold technique for automated segmentation across image modalities and cell lines

New microscopy technologies are enabling image acquisition of terabyte‐sized data sets consisting of hundreds of thousands of images. In order to retrieve and analyze the biological information in these large data sets, segmentation is needed to detect the regions containing cells or cell colonies. Our work with hundreds of large images (each 21 000×21 000 pixels) requires a segmentation method that: (1) yields high segmentation accuracy, (2) is applicable to multiple cell lines with various densities of cells and cell colonies, and several imaging modalities, (3) can process large data sets in a timely manner, (4) has a low memory footprint and (5) has a small number of user‐set parameters that do not require adjustment during the segmentation of large image sets. None of the currently available segmentation methods meet all these requirements. Segmentation based on image gradient thresholding is fast and has a low memory footprint. However, existing techniques that automate the selection of the gradient image threshold do not work across image modalities, multiple cell lines, and a wide range of foreground/background densities (requirement 2) and all failed the requirement for robust parameters that do not require re‐adjustment with time (requirement 5). We present a novel and empirically derived image gradient threshold selection method for separating foreground and background pixels in an image that meets all the requirements listed above. We quantify the difference between our approach and existing ones in terms of accuracy, execution speed, memory usage and number of adjustable parameters on a reference data set. This reference data set consists of 501 validation images with manually determined segmentations and image sizes ranging from 0.36 Megapixels to 850 Megapixels. It includes four different cell lines and two image modalities: phase contrast and fluorescent. Our new technique, called Empirical Gradient Threshold (EGT), is derived from this reference data set with a 10‐fold cross‐validation method. EGT segments cells or colonies with resulting Dice accuracy index measurements above 0.92 for all cross‐validation data sets. EGT results has also been visually verified on a much larger data set that includes bright field and Differential Interference Contrast (DIC) images, 16 cell lines and 61 time‐sequence data sets, for a total of 17 479 images. This method is implemented as an open‐source plugin to ImageJ as well as a standalone executable that can be downloaded from the following link: https://isg.nist.gov/ .     

Segmentation of nuclei and cells using membrane related protein markers

Segmenting individual cell nuclei from microscope images normally involves volume labelling of the nuclei with a DNA stain. However, this method often fails when the nuclei are tightly clustered in the tissue, because there is little evidence from the images on where the borders of the nuclei are. In this paper we present a method which solves this limitation and furthermore enables segmentation of whole cells. Instead of using volume stains, we used stains that specifically label the surface of nuclei or cells: lamins for the nuclear envelope and alpha-6 or beta-1 integrins for the cellular surface. The segmentation is performed by identifying unique seeds for each nucleus/cell and expanding the boundaries of the seeds until they reach the limits of the nucleus/cell, as delimited by the lamin or integrin staining, using gradient-curvature flow techniques. We tested the algorithm using computer-generated objects to evaluate its robustness against noise and applied it to cells in culture and to tissue specimens. In all the cases that we present the algorithm gave accurate results.     

Segmentation of intestinal gland images with iterative region growing

A region growing algorithm for segmentation of human intestinal gland images is presented. The initial seeding regions are identified based on the large vacant regions (lumen) inside the intestinal glands by fitting with a very large moving window. The seeding regions are then expanded by repetitive application of a morphological dilate operation with a much smaller round window structure set. False gland regions (nongland regions initially misclassified as gland regions) are removed based on either their excessive ages of active growth or inadequate thickness of dams formed by the strings of goblet cell nuclei sitting immediately outside the grown regions. The goblet cell nuclei are then identified and retained in the image. The gland contours are detected by applying a large moving round window fitting to the enormous empty exterior of the goblet cell nucleus chains in the image. The assumptions based on real intestinal gland images include the closed chain structured goblet cell nuclei that sit side-by-side with only small gaps between the neighbouring nuclei and that the lumens enclosed by the goblet cell nucleus chains are most vacant with only occasional run-away nuclei. The method performs well for most normal and abnormal intestinal gland images although it is less applicable to cancer cases. The experimental results show that the segmentations of the real microscopic intestinal gland images are satisfactorily accurate based on the visual evaluations.     

Segmentation and tracking of live cells in phase-contrast images using directional gradient vector flow for snakes

Cell shape is an important characteristic of the physiological state of a cell and is used as a primary read-out of cell behaviour in various assays. Automated accurate segmentation of cells in microscopy images is hence of large practical importance in cell biology. We report a simple algorithm for automated cell segmentation in high-magnification phase-contrast images, which takes advantage of the characteristic directionality of the local image intensity gradient at cellular boundaries due to the 'halo-effect'. We employ a two-step algorithm in which a gradient vector flow (GVF) field is first used to direct active contours to an approximate cell boundary. A directional GVF (DGVF) field is then calculated by considering only edges for which the image intensity gradient is directed outwards with respect to the approximate cell contour. Subsequently, the DGVF field is used to refine the cell contour, by directing active contours to edges with the desired gradient directionality. This method allows us to accurately segment cells in an image series, as well as follow the dynamics of cell shape over time in an automated fashion.     

A particle segmentation method based on nucleation and growth of the background

A new algorithm for image segmentation is proposed, which is capable of extracting particles in the presence of noise and background fluctuations. It begins with the detection of small regions belonging to the background, called background nuclei, and then lets these nuclei grow to become the entire background. Edge information and region information of the image are used simultaneously.     

Segmenting time‐lapse phase contrast images of adjacent NIH 3T3 cells

We present a new method for segmenting phase contrast images of NIH 3T3 fibroblast cells that is accurate even when cells are physically in contact with each other. The problem of segmentation, when cells are in contact, poses a challenge to the accurate automation of cell counting, tracking and lineage modelling in cell biology. The segmentation method presented in this paper consists of (1) background reconstruction to obtain noise‐free foreground pixels and (2) incorporation of biological insight about dividing and nondividing cells into the segmentation process to achieve reliable separation of foreground pixels defined as pixels associated with individual cells. The segmentation results for a time‐lapse image stack were compared against 238 manually segmented images (8219 cells) provided by experts, which we consider as reference data. We chose two metrics to measure the accuracy of segmentation: the ‘Adjusted Rand Index’ which compares similarities at a pixel level between masks resulting from manual and automated segmentation, and the ‘Number of Cells per Field’ (NCF) which compares the number of cells identified in the field by manual versus automated analysis. Our results show that the automated segmentation compared to manual segmentation has an average adjusted rand index of 0.96 (1 being a perfect match), with a standard deviation of 0.03, and an average difference of the two numbers of cells per field equal to 5.39% with a standard deviation of 4.6%.     

Automated nuclei segmentation of malignant using level sets

Segmentation of objects from a noisy and complex image is still a challenging task that needs to be addressed. This article proposed a new method to detect and segment nuclei to determine whether they are malignant or not (determination of the region of interest, noise removal, enhance the image, candidate detection is employed on the centroid transform to evaluate the centroid of each object, the level set [LS] is applied to segment the nuclei). The proposed method consists of three main stages: preprocessing, seed detection, and segmentation. Preprocessing stage involves the preparation of the image conditions to ensure that they meet the segmentation requirements. Seed detection detects the seed point to be used in the segmentation stage, which refers to the process of segmenting the nuclei using the LS method. In this research work, 58 H&E breast cancer images from the UCSB Bio‐Segmentation Benchmark dataset are evaluated. The proposed method reveals the high performance and accuracy in comparison to the techniques reported in literature. The experimental results are also harmonized with the ground truth images.     

Multi-image based method to correct vignetting effect in light microscopy images

Vignetting is the radial attenuation effect of the image's brightness intensity from the center of the optical axis to the edges. To perform quantitative image analyses it is mandatory to take into account this effect, intrinsic of the acquisition system. Many image processing steps, such as segmentation and object tracking, are strongly affected by vignetting and the effect becomes particularly evident in mosaicing. The most common approach to compensate the attenuation of the image's brightness intensity is to estimate the vignetting function from a homogeneous reference object, typically an empty field, and to use it to normalize the images acquired under the same microscope set-up conditions. However, several reasons lead to the use of image-based methods to estimate the vignetting function from the images themselves. In this work, we propose an effective multi-image based method suitable for real-time applications. It is designed to correct vignetting in wide field light microscopy images. The vignetting function is computed stemming from a background built incrementally from the proposed background segmentation algorithm, validated on several manually segmented images. The extensive experiments carried out using cell cultures, histological samples and synthetic images prove that our method almost always yields the best results and in worst cases are comparable to those achieved by using homogeneous reference objects.     

An automated feedback system with the hybrid model of scoring and classification for solving over-segmentation problems in RNAi high content screening

Background: High content screening (HCS) via automated fluorescence microscopy is a powerful technology for generating cellular images that are rich in phenotypic information. RNA interference is a revolutionary approach for silencing gene expression and has become an important method for studying genes through RNA interference‐induced cellular phenotype analysis. The convergence of the two technologies has led to large‐scale, image‐based studies of cellular phenotypes under systematic perturbations of RNA interference. However, existing high content screening image analysis tools are inadequate to extract content regarding cell morphology from the complex images, thus they limit the potential of genome‐wide RNA interference high content screening screening for simple marker readouts. In particular, over‐segmentation is one of the persistent problems of cell segmentation; this paper describes a new method to alleviate this problem. Methods: To solve the issue of over‐segmentation, we propose a novel feedback system with a hybrid model for automated cell segmentation of images from high content screening. A Hybrid learning model is developed based on three scoring models to capture specific characteristics of over‐segmented cells. Dead nuclei are also removed through a statistical model. Results: Experimental validation showed that the proposed method had 93.7% sensitivity and 94.23% specificity. When applied to a set of images of F‐actin‐stained Drosophila cells, 91.3% of over‐segmented cells were detected and only 2.8% were under‐segmented. Conclusions: The proposed feedback system significantly reduces over‐segmentation of cell bodies caused by over‐segmented nuclei, dead nuclei, and dividing cells. This system can be used in the automated analysis system of high content screening images.     

多尺度区域生长与去粘连模型的乳腺细胞分割

乳腺癌已经成为女性最常见的恶性肿瘤,组织切片显微图像的病理分析是诊断的主要手段,细胞的准确分割是病理分析的重要环节。该文提出了一种新的乳腺细胞显微图像的自动分割算法:首先结合小波分解和多尺度区域生长算法分离细胞和背景,实现对细胞的精确定位;然后采用改进的数学形态学对粘连细胞进行一次细分割;接着再采用基于曲率尺度空间(CSS)的角点检测分割算法对粘连细胞进行二次细分割;两次细分割方法构成了一个双策略去粘连模型,保证了去粘连的准确性和鲁棒性。将算法应用到22幅乳腺细胞显微图像上,可以对不同类型的乳腺细胞图像进行全自动分割,有较高的分割灵敏度(0.944±0.024)和特异度(0.937±0.038),且具有较好的普适性。     

基于标识线导航的自动导引车跟踪控制

为实现自动导引车对标识线路径的识别和跟踪,提出一种原理简单的视觉图像处理方法,包括阈值分割、滤波、边缘检测、变形矫正和改进的车体相对位置参数提取等。此方法能有效降低噪声干扰、图像变形对参数提取的影响以及增加图像处理的实时性;还提出一种适于自动导引车跟踪控制的最优控制策略。为验证理论分析的正确性,先对标识线图像进行处理,然后进行自动导引车的跟踪控制试验。结果表明,采用此种图像处理方法和最优控制策略的自动导引车,具有较为准确和可靠的路径跟踪效果。     

基于轮廓的血细胞图像拼接算法的设计与实现

针对显微镜下血细胞切片图视野较小的问题,提出了一种基于轮廓的拼接算法。首先通过改进的迭代法分割出白细胞胞核;采用边界跟踪法提取胞核轮廓后,计算各轮廓图像的不变矩,并构建不变矩的欧式距离矩阵;根据最近邻原则找出匹配轮廓,以匹配轮廓的重心为控制点计算仿射坐标变换系数。实验结果表明,该方法具有旋转、平移、缩放（RTS）不变性,且精度高,自动化程度好。     

An automatic segmentation algorithm for 3D cell cluster splitting using volumetric confocal images

With the rapid advance of three-dimensional (3D) confocal imaging technology, more and more 3D cellular images will be available. Segmentation of intact cells is a critical task in automated image analysis and quantification of cellular microscopic images. One of the major complications in the automatic segmentation of cellular images arises due to the fact that cells are often closely clustered. Several algorithms are proposed for segmenting cell clusters but most of them are 2D based. In other words, these algorithms are designed to segment 2D cell clusters from a single image. Given 2D segmentation methods developed, they can certainly be applied to each image slice with the 3D cellular volume to obtain the segmented cell clusters. Apparently, in such case, the 3D depth information with the volumetric images is not really used. Often, 3D reconstruction is conducted after the individualized segmentation to build the 3D cellular models from segmented 2D cellular contours. Such 2D native process is not appropriate as stacking of individually segmented 2D cells or nuclei do not necessarily form the correct and complete 3D cells or nuclei in 3D. This paper proposes a novel and efficient 3D cluster splitting algorithm based on concavity analysis and interslice spatial coherence. We have taken the advantage of using the 3D boundary points detected using higher order statistics as an input contour for performing the 3D cluster splitting algorithm. The idea is to separate the touching or overlapping cells or nuclei in a 3D native way. Experimental results show the efficiency of our algorithm for 3D microscopic cellular images.     

An automatic algorithm for the segmentation and morphological analysis of microvessels in immunostained histological tumour sections

A fully automatic segmentation and morphological analysis algorithm for the analysis of microvessels from CD31 immunostained histological tumour sections is presented. Development of the algorithm exploited the distinctive hues of stained vascular endothelial cells, cell nuclei and background, to provide the seeds for a 'region-growing' method for object segmentation in the 3D hue, saturation, value (HSV) colour model. The segmented objects, identified as microvessels by CD31 immunostaining, were post-processed with three morphological tasks: joining separate objects that were likely to belong to a single vessel, closing objects that had a narrow gap around their periphery, and splitting objects with multiple lumina into individual vessels. The automatic segmentation was validated against a hand-segmented set of 44 images from three different SW1222 human colorectal carcinomas xenografted into mice. 96.3 ± 0.9% of pixels were found to be correctly classified. Automated segmentation was carried out on a further 53 images from three histologically distinct mouse fibrosarcomas (MFs) for morphological comparison with the SW1222 tumours. Four morphometric measurements were calculated for each segmented vessel: vascular area (VA), ratio of lumen area to vascular area (lu/VA), eccentricity (e), and roundness (ro). In addition, the total vascular area relative to tumour tissue area (rVA) was calculated. lu/VA, e and ro were found to be significantly smaller in MF tumours than in SW1222 tumours (p < 0.05; unpaired t-test). The algorithm is available through the website http://www.caiman.org.uk where images can be uploaded, processed and sent back to users. The output from CAIMAN consists of the original image with boundaries of segmented vessels overlaid, the calculated parameters and a Matlab file, which contains the segmentation that the user can use to derive further results.     

Segmentation of 3D microtomographic images of granular materials with the stochastic watershed

Segmentation of 3D images of granular materials obtained by microtomography is not an easy task. Because of the conditions of acquisition and the nature of the media, the available images are not exploitable without a reliable method of extraction of the grains. The high connectivity in the medium, the disparity of the object's shape and the presence of image imperfections make classical segmentation methods (using image gradient and watershed constrained by markers) extremely difficult to perform efficiently. In this paper, we propose a non‐parametric method using the stochastic watershed, allowing to estimate a 3D probability map of contours. Procedures allowing to extract final segmentation from this function are then presented.     

Cell tracking using phase‐adaptive shape prior

Automated tracking of cell population is very crucial for quantitative measurements of dynamic cell‐cycle behaviour of individual cells. This problem involves several subproblems and a high accuracy of each step is essential to avoid error propagation. In this paper, we propose a holistic three‐component system to tackle this problem. For each phase, we first learn a mean shape as well as a model of the temporal dynamics of transformation, which are used for estimating a shape prior for the cell in the current frame. We then segment the cell using a level set‐based shape prior model. Finally, we identify its phase based on the goodness‐of‐fit of the data to the segmentation model. This phase information is also used for fine‐tuning the segmentation result. We evaluate the performance of our method empirically in various aspects and in tracking individual cells from HeLa H2B‐GFP cell population. Highly accurate validation results confirm the robustness of our method in many realistic scenarios and the essentiality of each component of our integrating system.     

Embryo zebrafish segmentation using an improved hybrid method

Zebrafish is an invaluable vertebrate model in life science research and has been widely used in biological pathway analysis, molecular screening and disease modelling, among others. As a result, microscopic imaging has become an essential step in zebrafish phenotype analysis, and image segmentation thus plays an important role in the zebrafish microscopy analysis. Due to the nonuniform distribution of intensity and weak boundary in zebrafish microscope images, the traditionally used segmentation methods may lead to unsatisfactory result. Here, a novel hybrid method that integrates region and boundary information into active contour model is proposed to segment zebrafish embryos from the background, which performs better than traditional segmentation models. Meanwhile, how to utilize the gradient information effectively in image segmentation is still an open problem. In this paper, we propose to improve the aforementioned hybrid method in two aspects. Firstly, the mean grey value of background is estimated by the expectation maximization (EM) algorithm to constrain the active curve evolution. Secondly, an edge stopping function sensitive to gradient information is designed to stop curve evolution when the active curve reaches the embryo boundary. Experimental results show that the proposed methods can provide superior segmentation results compared to existing algorithms.