An open‐source deconvolution software package for 3‐D quantitative fluorescence microscopy imaging

Deconvolution techniques have been widely used for restoring the 3‐D quantitative information of an unknown specimen observed using a wide‐field fluorescence microscope. Deconv , an open‐source deconvolution software package, was developed for 3‐D quantitative fluorescence microscopy imaging and was released under the GNU Public License. Deconv provides numerical routines for simulation of a 3‐D point spread function and deconvolution routines implemented three constrained iterative deconvolution algorithms: one based on a Poisson noise model and two others based on a Gaussian noise model. These algorithms are presented and evaluated using synthetic images and experimentally obtained microscope images, and the use of the library is explained. Deconv allows users to assess the utility of these deconvolution algorithms and to determine which are suited for a particular imaging application. The design of Deconv makes it easy for deconvolution capabilities to be incorporated into existing imaging applications.     

Robust incremental compensation of the light attenuation with depth in 3D fluorescence microscopy

Fluorescent signal intensities from confocal laser scanning microscopes (CLSM) suffer from several distortions inherent to the method. Namely, layers which lie deeper within the specimen are relatively dark due to absorption and scattering of both excitation and fluorescent light, photobleaching and/or other factors. Because of these effects, a quantitative analysis of images is not always possible without correction. Under certain assumptions, the decay of intensities can be estimated and used for a partial depth intensity correction. In this paper we propose an original robust incremental method for compensating the attenuation of intensity signals. Most previous correction methods are more or less empirical and based on fitting a decreasing parametric function to the section mean intensity curve computed by summing all pixel values in each section. The fitted curve is then used for the calculation of correction factors for each section and a new compensated sections series is computed. However, these methods do not perfectly correct the images. Hence, the algorithm we propose for the automatic correction of intensities relies on robust estimation, which automatically ignores pixels where measurements deviate from the decay model. It is based on techniques adopted from the computer vision literature for image motion estimation. The resulting algorithm is used to correct volumes acquired in CLSM. An implementation of such a restoration filter is discussed and examples of successful restorations are given.     

Dispersion, aberration and deconvolution in multi-wavelength fluorescence images

The wavelength dependence of the incoherent point spread function in a wide-field microscope was investigated experimentally. Dispersion in the sample and optics can lead to significant changes in the point spread function as wavelength is varied over the range commonly used in fluorescence microscopy. For a given sample, optical conditions can generally be optimized to produce a point spread function largely free of spherical aberration at a given wavelength. Unfortunately, deviations in wavelength from this value will result in spherically aberrated point spread functions. Therefore, when multiple fluorophores are used to localize different components in the same sample, the image of the distribution of at least one of the fluorophores will be spherically aberrated. This aberration causes a loss of intensity and resolution, thereby complicating the localization and analysis of multiple components in a multi-wavelength image. We show that optimal resolution can be restored to a spherically aberrated image by constrained, iterative deconvolution, as long as the spherical aberration in the point spread function used for deconvolution matches the aberration in the image reasonably well. The success of this method is essentially independent of the initial degree of spherical aberration in the image. Deconvolution of many biological images can be achieved by collecting a small library of spherically aberrated and unaberrated point spread functions, and then choosing a point spread function appropriate for deconvolving each image. The co-localization and relative intensities of multiple components can then be accurately studied in a multi-wavelength image.     

Application of the split-gradient method to 3D image deconvolution in fluorescence microscopy

The methods of image deconvolution are important for improving the quality of the detected images in the different modalities of fluorescence microscopy such as wide‐field, confocal, two‐photon excitation and 4Pi. Because deconvolution is an ill‐posed problem, it is, in general, reformulated in a statistical framework such as maximum likelihood or Bayes and reduced to the minimization of a suitable functional, more precisely, to a constrained minimization, because non‐negativity of the solution is an important requirement. Next, iterative methods are designed for approximating such a solution. In this paper, we consider the Bayesian approach based on the assumption that the noise is dominated by photon counting, so the likelihood is of the Poisson‐type, and that the prior is edge‐preserving, as derived from a simple Markov random field model. By considering the negative logarithm of the a posteriori probability distribution, the computation of the maximum a posteriori (MAP) estimate is reduced to the constrained minimization of a functional that is the sum of the Csiszár I‐divergence and a regularization term. For the solution of this problem, we propose an iterative algorithm derived from a general approach known as split‐gradient method (SGM) and based on a suitable decomposition of the gradient of the functional into a negative and positive part. The result is a simple modification of the standard Richardson–Lucy algorithm, very easily implementable and assuring automatically the non‐negativity of the iterates. Next, we apply this method to the particular case of confocal microscopy for investigating the effect of several edge‐preserving priors proposed in the literature using both synthetic and real confocal images. The quality of the restoration is estimated both by computation of the Kullback–Leibler divergence of the restored image from the detected one and by visual inspection. It is observed that the noise artefacts are considerably reduced and desired characteristics (edges and minute features as islets) are retained in the restored images. The algorithm is stable, robust and tolerant at various noise (Poisson) levels. Finally, by remarking that the proposed method is essentially a scaled gradient method, a possible modification of the algorithm is briefly discussed in view of obtaining fast convergence and reduction in computational time.     

Multicolour analysis and local image correlation in confocal microscopy

Multiparameter fluorescence microscopy is often used to identify cell types and subcellular organelles according to their differential labelling. For thick objects, the quantitative comparison of different multiply labelled specimens requires the three-dimensional (3-D) sampling capacity of confocal laser scanning microscopy, which can be used to generate pseudocolour images. To analyse such 3-D data sets, we have created pixel fluorogram representations, which are estimates of the joint probability densities linking multiple fluorescence distributions. Such pixel fluorograms also provide a powerful means of analysing image acquisition noise, fluorescence cross-talk, fluorescence photobleaching and cell movements. To identify true fluorescence co-localization, we have developed a novel approach based on local image correlation maps. These maps discriminate the coincident fluorescence distributions from the superimposition of noncorrelated fluorescence profiles on a local basis, by correcting for contrast and local variations in background intensity in each fluorescence channel. We believe that the pixel fluorograms are best suited to the quality control of multifluorescence image acquisition. The local image correlation methods are more appropriate for identifying co-localized structures at the cellular or subcellular level. The thresholding of these correlation maps can further be used to recognize and classify biological structures according to multifluorescence attributes.     

An FFT-based method for attenuation correction in fluorescence confocal microscopy

A problem in three-dimensional imaging using a confocal scanning laser microscope (CSLM) in the (epi)fluorescence mode is the darkening of the deeper layers due to absorption and scattering of both the excitation and the fluorescence light. A new method is proposed to correct for these effects. The approach, valid for weak attenuation, consists of multiplying the measured fluorescence intensity by a correction factor involving a convolution integral of the measured signal, which can be computed efficiently by the fast Fourier transform. Analytical and numerical estimates are given for the degree of attenuation under which the method is valid, and the method is applied to various test images. A real CSLM image is restored. Finally, the method is compared with a recent iterative method with regard to numerical accuracy and computational efficiency.     

Application of regularized Richardson-Lucy algorithm for deconvolution of confocal microscopy images

Although confocal microscopes have considerably smaller contribution of out-of-focus light than widefield microscopes, the confocal images can still be enhanced mathematically if the optical and data acquisition effects are accounted for. For that, several deconvolution algorithms have been proposed. As a practical solution, maximum-likelihood algorithms with regularization have been used. However, the choice of regularization parameters is often unknown although it has considerable effect on the result of deconvolution process. The aims of this work were: to find good estimates of deconvolution parameters; and to develop an open source software package that would allow testing different deconvolution algorithms and that would be easy to use in practice. Here, Richardson-Lucy algorithm has been implemented together with the total variation regularization in an open source software package IOCBio Microscope. The influence of total variation regularization on deconvolution process is determined by one parameter. We derived a formula to estimate this regularization parameter automatically from the images as the algorithm progresses. To assess the effectiveness of this algorithm, synthetic images were composed on the basis of confocal images of rat cardiomyocytes. From the analysis of deconvolved results, we have determined under which conditions our estimation of total variation regularization parameter gives good results. The estimated total variation regularization parameter can be monitored during deconvolution process and used as a stopping criterion. An inverse relation between the optimal regularization parameter and the peak signal-to-noise ratio of an image is shown. Finally, we demonstrate the use of the developed software by deconvolving images of rat cardiomyocytes with stained mitochondria and sarcolemma obtained by confocal and widefield microscopes.     

Generalized approach for accelerated maximum likelihood based image restoration applied to three-dimensional fluorescence microscopy

For deconvolution applications in three-dimensional microscopy we derived and implemented a generic, accelerated maximum likelihood image restoration algorithm. A conjugate gradient iteration scheme was used considering either Gaussian or Poisson noise models. Poisson models are better suited to low intensity fluorescent image data; typically, they show smaller restoration errors and smoother results. For the regularization, we modified the standard Tikhonov method. However, the generic design of the algorithm allows for more regularization approaches. The Hessian matrix of the restoration functional was used to determine the step size. We compared restoration error and convergence behaviour between the classical line-search and the Hessian matrix method. Under typical working conditions, the restoration error did not increase over that of the line-search and the speed of convergence did not significantly decrease allowing for a twofold increase in processing speed. To determine the regularization parameter, we modified the generalized cross-validation method. Tests that were done on both simulated and experimental fluorescence wide-field data show reliable results.     

Image adaptive point-spread function estimation and deconvolution for in vivo confocal microscopy

Visualizing deep inside the tissue of a thick biological sample often poses severe constraints on image conditions. Standard restoration techniques (denoising and deconvolution) can then be very useful, allowing one to increase the signal-to-noise ratio and the resolution of the images. In this paper, we consider the problem of obtaining a good determination of the point-spread function (PSF) of a confocal microscope, a prerequisite for applying deconvolution to three-dimensional image stacks acquired with this system. Because of scattering and optical distortion induced by the sample, the PSF has to be acquired anew for each experiment. To tackle this problem, we used a screening approach to estimate the PSF adaptively and automatically from the images. Small PSF-like structures were detected in the images, and a theoretical PSF model reshaped to match the geometric characteristics of these structures. We used numerical experiments to quantify the sensitivity of our detection method, and we demonstrated its usefulness by deconvolving images of the hearing organ acquired in vitro and in vivo.     

Restoration of confocal images for quantitative image analysis

Quantitative studies of three-dimensional (3-D) structure of microscopic objects have been made possible through the introduction of microscopic volume imaging techniques, most notably the confocal fluorescence microscope (CFM). Although the CFM is a true volume imager, its specific imaging properties give rise to distortions in the images and hamper subsequent quantitative analysis. Therefore, it is a prerequisite that confocal images are restored prior to analysis. The distortions can be divided into several categories: attenuation of areas in the image due to self-absorption, bleaching effects, geometrical effects and distortions due to diffraction effects. Of these, absorption and diffraction effects are the most important. This paper describes a method aimed at the correction of diffraction-induced distortions. All the steps necessary in restoring confocal images are discussed, including a novel method to measure instrumental properties on a routine basis. To test the restoration procedure an image of a fluorescent planar object was restored. The results show a considerable improvement in the z-resolution and no ringing artefacts. The relevance of the method for image analysis is demonstrated by a comparison of results of applying 3-D texture analysis to restored and unrestored images of a synthetic object. Furthermore, the method can be successfully applied to noisy fluorescence images of biological objects, such as interphase cell nucei.     

Registration of phase‐contrast images in propagation‐based X‐ray phase tomography

X‐ray phase tomography aims at reconstructing the 3D electron density distribution of an object. It offers enhanced sensitivity compared to attenuation‐based X‐ray absorption tomography. In propagation‐based methods, phase contrast is achieved by letting the beam propagate after interaction with the object. The phase shift is then retrieved at each projection angle, and subsequently used in tomographic reconstruction to obtain the refractive index decrement distribution, which is proportional to the electron density. Accurate phase retrieval is achieved by combining images at different propagation distances. For reconstructions of good quality, the phase‐contrast images recorded at different distances need to be accurately aligned. In this work, we characterise the artefacts related to misalignment of the phase‐contrast images, and investigate the use of different registration algorithms for aligning in‐line phase‐contrast images. The characterisation of artefacts is done by a simulation study and comparison with experimental data. Loss in resolution due to vibrations is found to be comparable to attenuation‐based computed tomography. Further, it is shown that registration of phase‐contrast images is nontrivial due to the difference in contrast between the different images, and the often periodical artefacts present in the phase‐contrast images if multilayer X‐ray optics are used. To address this, we compared two registration algorithms for aligning phase‐contrast images acquired by magnified X‐ray nanotomography: one based on cross‐correlation and one based on mutual information. We found that the mutual information‐based registration algorithm was more robust than a correlation‐based method.     

Fluorescence photobleaching-based image standardization for fluorescence microscopy

A method is presented for the standardization of images acquired with fluorescence microscopy, based on the knowledge of spatial distributions proportional to the microscope's absolute excitation intensity and fluorescence detection efficiency distributions over the image field. These distributions are determined using a thin fluorescent test layer, employed under practically mono-exponential photobleaching conditions. It is demonstrated that these distributions can be used for (i) the quantitative evaluation of differences between both the excitation intensity and the fluorescence detection efficiency of different fluorescence microscopes and (ii) the standardization of images acquired with different microscopes, permitting the deduction of quantitative relationships between images obtained under different imaging conditions.     

Confocal microscopy and image processing in the study of plant nuclear structure

We describe measurements of the point spread function (PSF) for a confocal microscope and compare them with the PSF for a conventional (wide-field) fluorescence microscope. In situ hybridization with probes to telomere and ribosomal rDNA sequences, combined with three-dimensional (3-D) microscopy, has been used to study interphase nuclei in root tissue of Pisum sativum and Vicia faba. Nearly all the telomeres in both species are located at the nuclear envelope, and are highly clustered in the Vicia tissues, suggesting specific binding interactions. rDNA labelling in P. sativum shows four brightly staining knobs, corresponding to condensed regions of the rDNA genes from the two pairs of nucleolar organizer genes in this species, arranged approximately tetrahedrally around each nucleolus. Deconvolution using the measured PSFs can be used to improve these images, revealing a fibrous substructure in the perinucleolar knobs, and a large amount of interconnecting internal structure, which we suggest represents rDNA both in the fibrillar centres and also more diffuse, widely dispersed rDNA. Finally we show that accurate conventional data coupled with deconvolution can produce 3-D reconstructions comparable to those obtainable with confocal microscopy, but that the clearest images are obtained by applying deconvolution to the confocal data.     

Quantification of colocalization and cross-talk based on spectral angles

Common methods for quantification of colocalization in fluorescence microscopy typically require cross-talk free images or images where cross-talk has been eliminated by image processing, as they are based on intensity thresholding. Quantification of colocalization includes not only calculating a global measure of the degree of colocalization within an image, but also a classification of each image pixel as showing colocalized signals or not. In this paper, we present a novel, automated method for quantification of colocalization and classification of image pixels. The method, referred to as SpecDec, is based on an algorithm for spectral decomposition of multispectral data borrowed from the field of remote sensing. Pixels are classified based on hue rather than intensity. The hue distribution is presented as a histogram created by a series of steps that compensate for the quantization noise always present in digital image data, and classification rules are thereafter based on the shape of the angle histogram. Detection of colocalized signals is thus only dependent on the hue, making it possible to classify also low-intensity objects, and decoupling image segmentation from detection of colocalization. Cross-talk will show up as shifts of the peaks of the histogram, and thus a shift of the classification rules, making the method essentially insensitive to cross-talk. The method can also be used to quantify and compensate for cross-talk, independent of the microscope hardware.     

Quantitative image correction and calibration for confocal fluorescence microscopy using thin reference layers and SIPchart-based calibration procedures

The fluorescence intensity image of an axially integrated through-focus series of a thin standardized uniform fluorescent layer can be used for image intensity correction and calibration in sectioning microscopy. This intensity image is in fact available from the earlier introduced Sectioned Imaging Property (SIP) charts ( Brakenhoff et al. , 2005 ). It is shown that the integrated intensity of a z -stack from a biological sample, imaged under identical conditions as the layer, can be calibrated in terms of fluorescence layer units of the calibration layer. The imaging after such calibration becomes, as a first approximation, independent of the microscope system and imaging conditions. This is demonstrated on axially integrated images of standard fluorescent beads and standard BPAE Fluorocells. Corrections on the microscope imaging conditions include shading effects, imaging with different magnifications and objectives, and using different microscope systems. It is also shown that with the present approach the actual underlying three-dimensional (3D) fluorescence data set itself can be corrected for variations in point spread function (PSF) imaging efficiency over the imaging data cube. Realizing such calibration between imaging conditions or systems requires basically only the 2D fluorescer molecule density of the reference layers and the section distances with which the layer data are collected.     

4Pi-confocal images with axial superresolution

We present two-photon excitation 4Pi-confocal images of clustered fluorescence beads demonstrating three-dimensional far-field light microscopy with unprecedented resolution. For an excitation wavelength of 760 nm, the lateral and axial resolution amounts to 200 and 145 nm, respectively. The four-fold improved axial resolution is achieved by engineering the point-spread function through a suitable combination of aperture enlargement, two-photon excitation, confocalization and three-point deconvolution. In contrast to their confocal counterparts, 4Pi-confocal images do not exhibit the typical axial elongation. The axial resolution in the 4Pi-confocal images corresponds to about one-fifth of the wavelength and surpasses the lateral resolution by 25%.     

Noise suppression of point spread functions and its influence on deconvolution of three-dimensional fluorescence microscopy image sets

The point spread function (PSF) is of central importance in the image restoration of three-dimensional image sets acquired by an epifluorescent microscope. Even though it is well known that an experimental PSF is typically more accurate than a theoretical one, the noise content of the experimental PSF is often an obstacle to its use in deconvolution algorithms. In this paper we apply a recently introduced noise suppression method to achieve an effective noise reduction in experimental PSFs. We show with both simulated and experimental three-dimensional image sets that a PSF that is smoothed with this method leads to a significant improvement in the performance of deconvolution algorithms, such as the regularized least-squares algorithm and the accelerated Richardson–Lucy algorithm.     

Comparative evaluation of performance measures for shading correction in time‐lapse fluorescence microscopy

Time‐lapse fluorescence microscopy is a valuable technology in cell biology, but it suffers from the inherent problem of intensity inhomogeneity due to uneven illumination or camera nonlinearity, known as shading artefacts. This will lead to inaccurate estimates of single‐cell features such as average and total intensity. Numerous shading correction methods have been proposed to remove this effect. In order to compare the performance of different methods, many quantitative performance measures have been developed. However, there is little discussion about which performance measure should be generally applied for evaluation on real data, where the ground truth is absent. In this paper, the state‐of‐the‐art shading correction methods and performance evaluation methods are reviewed. We implement 10 popular shading correction methods on two artificial datasets and four real ones. In order to make an objective comparison between those methods, we employ a number of quantitative performance measures. Extensive validation demonstrates that the coefficient of joint variation (CJV) is the most applicable measure in time‐lapse fluorescence images. Based on this measure, we have proposed a novel shading correction method that performs better compared to well‐established methods for a range of real data tested.     

Use of a white light supercontinuum laser for confocal interference-reflection microscopy

Shortly after its development, the white light supercontinuum laser was applied to confocal scanning microscopy as a more versatile substitute for the multiple monochromatic lasers normally used for the excitation of fluorescence. This light source is now available coupled to commercial confocal fluorescence microscopes. We have evaluated a supercontinuum laser as a source for a different purpose: confocal interferometric imaging of living cells and artificial models by interference reflection. We used light in the range 460-700 nm where this source provides a reasonably flat spectrum, and obtained images free from fringe artefacts caused by the longer coherence length of conventional lasers. We have also obtained images of cytoskeletal detail that is difficult to see with a monochromatic laser.