Segmentation of confocal microscope images of cell nuclei in thick tissue sections

Segmentation of intact cell nuclei from three-dimensional (3D) images of thick tissue sections is an important basic capability necessary for many biological research studies. However, segmentation is often difficult because of the tight clustering of nuclei in many specimen types. We present a 3D segmentation approach that combines the recognition capabilities of the human visual system with the efficiency of automatic image analysis algorithms. The approach first uses automatic algorithms to separate the 3D image into regions of fluorescence-stained nuclei and unstained background. This includes a novel step, based on the Hough transform and an automatic focusing algorithm to estimate the size of nuclei. Then, using an interactive display, each nuclear region is shown to the analyst, who classifies it as either an individual nucleus, a cluster of multiple nuclei, partial nucleus or debris. Next, automatic image analysis based on morphological reconstruction and the watershed algorithm divides clusters into smaller objects, which are reclassified by the analyst. Once no more clusters remain, the analyst indicates which partial nuclei should be joined to form complete nuclei. The approach was assessed by calculating the fraction of correctly segmented nuclei for a variety of tissue types: Caenorhabditis elegans embryos (839 correct out of a total of 848), normal human skin (343/362), benign human breast tissue (492/525), a human breast cancer cell line grown as a xenograft in mice (425/479) and invasive human breast carcinoma (260/335). Furthermore, due to the analyst's involvement in the segmentation process, it is always known which nuclei in a population are correctly segmented and which not, assuming that the analyst's visual judgement is correct.     

Automatic thresholding of three-dimensional microvascular structures from confocal microscopy images

We have combined confocal microscopy, image processing, and optimization techniques to obtain automated, accurate volumetric measurements of microvasculature. Initially, we made tissue phantoms containing 15-μm FocalCheck™ microspheres suspended in type I collagen. Using these phantoms we obtained a stack of confocal images and examined the accuracy of various thresholding schemes. Thresholding algorithms from the literature that utilize a unimodal histogram, a bimodal histogram, or an intensity and edge-based algorithm all significantly overestimated the volume of foreground structures in the image stack. Instead, we developed a heuristic technique to automatically determine good-quality threshold values based on the depth, intensity, and (optionally) gradient of each voxel. This method analyzed intensity and gradient threshold methods for each individual image stack, taking into account the intensity attenuation that is seen in deeper images of the stack. Finally, we generated a microvascular construct comprised of rat fat microvessel fragments embedded in collagen I gels and obtained stacks of confocal images. Using our new thresholding scheme we were able to obtain automatic volume measurements of growing microvessel fragments.     

Three-dimensional volume reconstruction of extracellular matrix proteins in uveal melanoma from fluorescent confocal laser scanning microscope images

The distribution of looping patterns of laminin in uveal melanomas and other tumours has been associated with adverse outcome. Moreover, these patterns are generated by highly invasive tumour cells through the process of vasculogenic mimicry and are not therefore blood vessels. Nevertheless, these extravascular matrix patterns conduct plasma. The three‐dimensional (3D) configuration of these laminin‐rich patterns compared with blood vessels has been the subject of speculation and intensive investigation. We have developed a method for the 3D reconstruction of volume for these extravascular matrix proteins from serial paraffin sections cut at 4 µm thicknesses and stained with a fluorescently labelled antibody to laminin ( Maniotis et al., 2002 ). Each section was examined via confocal laser‐scanning focal microscopy (CLSM) and 13 images were recorded in the Z‐dimension for each slide. The input CLSM imagery is composed of a set of 3D subvolumes (stacks of 2D images) acquired at multiple confocal depths, from a sequence of consecutive slides. Steps for automated reconstruction included (1) unsupervised methods for selecting an image frame from a subvolume based on entropy and contrast criteria, (2) a fully automated registration technique for image alignment and (3) an improved histogram equalization method that compensates for spatially varying image intensities in CLSM imagery due to photo‐bleaching. We compared image alignment accuracy of a fully automated method with registration accuracy achieved by human subjects using a manual method. Automated 3D volume reconstruction was found to provide significant improvement in accuracy, consistency of results and performance time for CLSM images acquired from serial paraffin sections.     

3-D image formation in high-aperture fluorescence confocal microscopy: a numerical analysis

The imaging properties of a confocal fluorescence microscope are considered on the basis of a theoretical model. The model takes into account high-aperture objectives, the polarization state of the excitation light and a finite detector pinhole. Electromagnetic diffraction theory of the field near focus as developed by Richards and Wolf is used to compute the optical properties of the model. These are shown to be dependent on the polarization of the light. With the resulting three-dimensional point spread function we have studied the imaging of point, line and plane objects as a function of their orientation with respect to the confocal plane. In addition, the effect of the pinhole size on the image formation of these objects is discussed. The results indicate the necessity to take object orientation into account during image processing activities such as segmentation or analysis.     

Enhancement of confocal microscopy images using Mueller-matrix polarimetry

A simplified procedure based on Mueller-matrix polarimetry has recently been reported as a method of retinal image improvement in a confocal ophthalmoscope [J. M. Bueno et al ., J. Opt. Soc. Am. A 24, 1337 (2007)]. Here, we have applied the technique to imaging static samples providing well-defined reflection properties. The method uses a generator of polarization states in the illumination pathway of a confocal scanning laser system. From the calculated four elements of the Mueller matrix of any sample and instrument combination, the best images defined by different metrics were constructed. For samples with specular, diffuse and mixed reflections, the best-constructed images showed an enhancement in both objective and subjective image quality compared to the original images and those obtained from frame averaging. This technique could improve microscopic imaging in many diverse fields, particularly in biomedical imaging.     

Study of Golgi-impregnated material using the confocal tandem scanning reflected light microscope

The tandem scanning reflected-light microscope (TSM) is a real-time, direct-view confocal microscope. Only those points in the specimen situated in the focal plane contribute information to the image. A Tracor Northern TMS with piezo-electric control of the objective lens was used to generate 3-D images from Golgi-impregnated hamster cerebral cortex. Stereoscopic pairs of images were recorded as 35-mm colour film transparencies by photographing while automatically through-focusing along inclined axes. Transferring the image via a TV camera to the computer, stereo-pairs were obtained by oblique through-focusing and summing, displaying maximum intensity data in each line of sight. Pseudocolour topographic displays were generated by assigning the pixel value in a z map image as the focal depth at which the back-scattered light signal was maximal. The TSM was also modified so that a conventional transmitted-light image with a large depth of field could be obtained simultaneously as the very shallow depth of field confocal back-scattered-light image seen at any focus level. The conventional image is a silhouette of the impregnated neurons: the top surface of the cell is not visible and the relationships of processes that cross over cell bodies cannot be discerned. TSM gives a high-contrast image. The Golgi precipitate over the neuronal surface is resolved as globular or ovoid, coloured particles. The smaller particles also cover the dendritic spines. All the confocal range (extended focus) image display methods satisfactorily demonstrated the 3-D arrangement of cell bodies and processes in the chosen volume.     

Three-dimensional imaging of corneal cells using in vivo confocal microscopy

Confocal microscopy is a unique and powerful imaging paradigm which allows optical sectioning through intact tissue. Real-time tandem scanning confocal microscopy has previously been used to generate high-magnification two-dimensional (2-D) images of cells in living organ systems. Inherent problems with movement, however, have prevented the in vivo acquisition of complete 3-D datasets. The development of a new objective lens, used in combination with specialized real-time image acquisition procedures, has allowed sequential serial sections to be obtained in vivo from the rabbit cornea for the first time. These sections can be digitially registered and stacked on the computer to provide a 3-D reconstruction of the corneal cells. This technique should serve as a useful method for studying 3-D structures and analysing 4-D phenomena at the cellular level in living animals. Three-dimensional images of a stromal nerve in normal rabbit cornea and of fibroblasts within a rabbit corneal wound are presented as examples of current capabilities.     

A confocal video-rate laser-beam scanning reflected-light microscope with no moving parts

A no-moving-parts, 30 frames/s, laser-beam scanning confocal reflected-light microscope has been developed. In principle, the technique can be extended to fluorescence and transmission light microscopy. Acousto-optic beam deflectors controlled by digital electronics move a laser beam in a 512-line interlaced 8·5 times 8·5-mm raster. The light passes through a beam splitter, enters an inverted microscope through the side camera port, and is imaged at the object by the microscope objective. Reflected light returns through the objective, exits the camera port, is reflected off the beam splitter, and is imaged on to the photocathode of an image dissector tube (IDT). Confocality is provided by raster scanning the IDT aperture coincident with the congruent image of the laser beam incident on the object. Real-time jitter-free reflected light images of a variety of biological objects have been produced. Computer-controlled alignment of the laser scan and IDT is performed in several seconds.     

The use of single image random dot stereograms for presenting 3D microscopic confocal images

Presenting 3D images produced by confocal optical slicing techniques in a static and easily publishable form can be difficult. Here, we demonstrate the presentation of the data as a single image random dot stereogram (SIRD), which can be viewed as a 3D object by 'defocusing' the eyes. The production of the SIRD employs three steps: (i) acquisition of the optical slices using confocal techniques, (ii) allocation of a suitable grey level to the object in each slice to provide depth-encoding information for the final image and (iii) calculation of the SIRD from the composite depth-encoded image. The technique is demonstrated with a limited number of optical slices through an acridine-orange-stained neutrophil (diameter =10 μm), in order to show the relative positions of the nuclear lobes in the cell.     

Three-dimensional image restoration and super-resolution in fluorescence confocal microscopy

Confocal scanning laser microscopy (CSLM) provides optical sectioning of a fluorescent sample and improved resolution with respect to conventional optical microscopy. As a result, three-dimensional (3-D) imaging of biological objects becomes possible. A difficulty is that the lateral resolution is better than the axial resolution and, thus, the microscope provides orientation-dependent images. However, a theoretical investigation of the process of image formation in CSLM shows that it must be possible to improve the resolution obtained in practice. We present two methods for achieving such a result in the case of 3-D fluorescent objects. The first method applies to conventional CSLM, where the image is detected only on the optical axis for any scanning position. Since the resulting 3-D image is the convolution of the object with the impulse-response function of the instrument, the problem of image restoration is a deconvolution problem and is affected by numerical instability. A short introduction to the linear methods developed for obtaining stable solutions of these problems (the so-called regularization theory of ill-posed problems) is given and an application to a real image is discussed. The second method applies to a new version of CSLM proposed in recent years. In such a case the full image must be measured by a suitable array of detectors. For each scanning position the data are not single numbers but vectors. Then, in order to recover the object, one must solve a Fredholm integral equation of the first kind. A method for the solution of this equation is presented and the possibility of achieving super-resolution is demonstrated. More precisely, we show that it is possible to improve by about a factor of 2 the resolution of conventional CSLM both in the lateral and axial directions.     

Comparison and accuracy of methods to determine the confocal volume for quantitative fluorescence correlation spectroscopy

Single molecule detection based on fluorescent labels offers the possibility to gain not only qualitative but also quantitative insight into specific functions of complex biological systems. Fluorescence correlation spectroscopy is one of the favourite techniques to determine concentrations and diffusion constants as well as molecular brightness of molecules in the pico‐ to nano‐molar concentration range, with broad applications in biology and chemistry. Although fluorescence correlation spectroscopy in principle has the potential to measure absolute concentrations and diffusion coefficients, the necessity to know the exact size and shape of the confocal volume very often hampers the possibility to obtain quantitative results and restricts fluorescence correlation spectroscopy to relative measurements mainly. The determination of the confocal volume in situ is difficult because it is sensitive to optical alignment and aberrations, optical saturation and variations of the index of refraction as observed in biological specimen. In the present contribution, we compare different techniques to characterize the confocal volume and to obtain the confocal parameters by fluorescence correlation spectroscopy curve fitting, a fluorescence correlation spectroscopy dilution series and confocal scanning of fluorescent beads. The results are compared in the view of quantitative fluorescence correlation spectroscopy measurement and analysis. We investigate how unavoidable artefacts caused by a non‐ideal confocal volume can be experimentally determined and validated.     

Methods for large data volumes from confocal scanning laser microscopy of lung

Confocal scanning laser microscopy makes it possible to obtain series of optical sections in precise registration. Certain studies of lung parenchyma, however, require both the fine resolution obtainable with high-numerical-aperture (NA) objectives and the extensive fields of view that usually would be achieved only with low-NA objectives. This article presents a technique that resolves this conflict by using a sequence of operations: (i) to correct intensity variations on individual sections due to non-uniform illumination/detection characteristics of the microscope; (ii) to correct intensity variations between successive sections in a series due to, for example, depth-related absorption or step changes in detector sensitivity; (iii) to adjust adjacent, overlapping stacks of sections to a common intensity level; and (iv) to fuse a group of such overlapping stacks into a single series of larger sections. This resulting stack may contain, for example, a complete cross-section of an alveolar ductal unit about 500 μm or more in diameter at about 1-μm pixel resolution.     

Detection and quantitative assessment of image aberrations from single HRTEM lattice images

For utilizing the full capabilities of quantitative high-resolution transmission electron microscopy in materials characterization, a precise knowledge of the various aberrations blurring the object information is essential. Here, we describe an extended approach to the detection and quantitative assessment of image aberrations from lattice images of crystal samples. The approach is based on a theoretical analysis of five-beam lattice images in the presence of all relevant optical aberrations and for partial beam coherence. Compact analytical expressions for linear and nonlinear image Fourier coefficients as explicit functions of the aberration parameters are derived. In particular, a fundamental relationship between the occurrence of erroneous image symmetries and the simultaneous presence of optical misalignments and partial beam coherence is established. An image analysis procedure is proposed which allows for the detection of even- and odd-order residual aberrations and for the quantitative determination of defocus, two-fold astigmatism and axial coma if the three-fold astigmatism is known. For coma-free images, the three-fold astigmatism can also be determined quantitatively. Moreover, the procedure allows for a reliable detection of crystal misalignment for images of wedge-shaped crystal samples.     

A comparison of image restoration approaches applied to three-dimensional confocal and wide-field fluorescence microscopy

We have compared different image restoration approaches for fluorescence microscopy. The most widely used algorithms were classified with a Bayesian theory according to the assumed noise model and the type of regularization imposed. We considered both Gaussian and Poisson models for the noise in combination with Tikhonov regularization, entropy regularization, Good's roughness and without regularization (maximum likelihood estimation). Simulations of fluorescence confocal imaging were used to examine the different noise models and regularization approaches using the mean squared error criterion. The assumption of a Gaussian noise model yielded only slightly higher errors than the Poisson model. Good's roughness was the best choice for the regularization. Furthermore, we compared simulated confocal and wide-field data. In general, restored confocal data are superior to restored wide-field data, but given sufficient higher signal level for the wide-field data the restoration result may rival confocal data in quality. Finally, a visual comparison of experimental confocal and wide-field data is presented.     

Measurement of co-localization of objects in dual-colour confocal images

A method to measure the degree of co-localization of objects in confocal dual-colour images has been developed. This image analysis produced two coefficients that represent the fraction of co-localizing objects in each component of a dual-channel image. The generation of test objects with a Gaussian intensity distribution, at well-defined positions in both components of dual-channel images, allowed an accurate investigation of the reliability of the procedure. To do that, the co-localization coefficients were determined before degrading the image with background, cross-talk and Poisson noise. These synthesized sources of image deterioration represent sources of deterioration that must be dealt with in practical confocal imaging, namely dark current, non-specific binding and cross-reactivity of fluorescent probes, optical cross-talk and photon noise. The degraded images were restored by filtering and cross-talk correction. The co-localization coefficients of the restored images were not significantly different from those of the original undegraded images. Finally, we tested the procedure on images of real biological specimens. The results of these tests correspond with data found in the literature. We conclude that the co-localization coefficients can provide relevant quantitative information about the positional relation between biological objects or processes.     

Digital image acquisition in in vivo confocal microscopy

A flexible system for the real-time acquisition of in vivo images has been developed. Images are generated using a tandem scanning confocal microscope interfaced to a low-light-level camera. The video signal from the camera is digitized and stored using a Gould image processing system with a real-time digital disk (RTDD). The RTDD can store up to 3200 512 times 512 pixel images at video rates (30 images s^?1). Images can be input directly from the camera during the study, or off-line from a Super VHS video recorder. Once a segment of experimental interest is digitized onto the RTDD, the user can interactively step through the images, average stable sequences, and identify candidates for further processing and analysis. Examples of how this system can be used to study the physiology of various organ systems in vivo are presented.     

Automated three-dimensional analysis of particle measurements using an optical profilometer and image analysis software

The automated collection of topographic images from an optical profilometer coupled with existing image analysis software offers the unique ability to quantify three‐dimensional particle morphology. Optional software available with most optical profilers permits automated collection of adjacent topographic images of particles dispersed onto a suitable substrate. Particles are recognized in the image as a set of continuous pixels with grey‐level values above the grey level assigned to the substrate, whereas particle height or thickness is represented in the numerical differences between these grey levels. These images are loaded into remote image analysis software where macros automate image processing, and then distinguish particles for feature analysis, including standard two‐dimensional measurements (e.g. projected area, length, width, aspect ratios) and third‐dimensional measurements (e.g. maximum height, mean height). Feature measurements from each calibrated image are automatically added to cumulative databases and exported to a commercial spreadsheet or statistical program for further data processing and presentation. An example is given that demonstrates the superiority of quantitative three‐dimensional measurements by optical profilometry and image analysis in comparison with conventional two‐dimensional measurements for the characterization of pharmaceutical powders with plate‐like particles.     

Visualization and coding in three-dimensional image processing

A system for calculating orthographic views of three-dimensional objects from a confocal microscope has been implemented in a high-level language. It is used on a regular basis in a number of projects and on different computers. The system enables the user to filter the original data and make a selection of which points and parts of the objects to show in a projective view. The information to be shown is coded in a compact format that is well suited for projection calculations. Several display principles were implemented that enhance different aspects of the objects.     

改善激光共焦扫描显微镜像质的方法

为解决传统光学显微镜样本上每一点的图像都受到邻近点衍射或散射光干扰的问题,研发了一套基于C#WinForm控制平台进行连续扫描方式的激光共焦扫描显微镜(LCSM)系统,并且成功地对生物细胞进行了扫描成像。针对共焦显微镜图像像质不高的问题,提出合理选取探测器针孔直径,并通过高斯低通滤波、盲解卷积的方法,确保实现高像质。实验结果表明,基于上述方法改进后的LCSM具有较高图像质量,该方法简单易行,便于实施。