Noise suppression of point spread functions and its influence on deconvolution of three-dimensional fluorescence microscopy image sets

The point spread function (PSF) is of central importance in the image restoration of three-dimensional image sets acquired by an epifluorescent microscope. Even though it is well known that an experimental PSF is typically more accurate than a theoretical one, the noise content of the experimental PSF is often an obstacle to its use in deconvolution algorithms. In this paper we apply a recently introduced noise suppression method to achieve an effective noise reduction in experimental PSFs. We show with both simulated and experimental three-dimensional image sets that a PSF that is smoothed with this method leads to a significant improvement in the performance of deconvolution algorithms, such as the regularized least-squares algorithm and the accelerated Richardson–Lucy algorithm.     

基于边缘预测和稀疏约束的湍流图像盲复原

大气湍流严重影响天文观测图像的成像效果,必须对退化图像进行处理才能获得清晰的图像。经典的湍流退化图像盲复原算法(IBD、NAS-RIF等)使用的先验知识过于简单,导致很多场合不能获得较优的复原效果。近几年提出的稀疏表达理论,使用自然图像边缘的稀疏先验信息指导图像复原,能复原出较多的细节,但它直接使用模糊图像的梯度图像指导点扩散函数复原,而模糊的梯度图像包含很多噪声和伪边缘,无效的梯度会误导点扩散函数的估计,从而使复原图像中出现较多伪迹。针对上述问题,提出了一种基于边缘预测和稀疏比值正则约束的湍流退化图像盲复原算法,该算法首先从当前的复原图像中预测出有效的边缘,然后将边缘预测信息与自然图像边缘的稀疏先验信息相结合指导点扩散函数复原,得到点扩散函数后,再通过一种非盲复原算法恢复出当前的目标图像,并将此复原图像作为下一次边缘预测的输入图像,如此迭代循环直到求出最终清晰的目标图像。所提算法结合了图像的先验信息与退化图像自身包含的有效信息,能有效抑制图像复原过程中产生的伪迹,获得令人满意的结果。针对多幅模拟的湍流退化图像进行仿真测试,验证了算法的有效性。     

Application of an advanced maximum likelihood estimation restoration method for enhanced‐resolution and contrast in second‐harmonic generation microscopy

Second‐harmonic generation (SHG) microscopy has gained popularity because of its ability to perform submicron, label‐free imaging of noncentrosymmetric biological structures, such as fibrillar collagen in the extracellular matrix environment of various organs with high contrast and specificity. Because SHG is a two‐photon coherent scattering process, it is difficult to define a point spread function (PSF) for this modality. Hence, compared to incoherent two‐photon processes like two‐photon fluorescence, it is challenging to apply the various PSF‐engineering methods to improve the spatial resolution to be close to the diffraction limit. Using a synthetic PSF and application of an advanced maximum likelihood estimation (AdvMLE) deconvolution algorithm, we demonstrate restoration of the spatial resolution in SHG images to that closer to the theoretical diffraction limit. The AdvMLE algorithm adaptively and iteratively develops a PSF for the supplied image and succeeds in improving the signal to noise ratio (SNR) for images where the SHG signals are derived from various sources such as collagen in tendon and myosin in heart sarcomere. Approximately 3.5 times improvement in SNR is observed for tissue images at depths of up to ～480 nm, which helps in revealing the underlying helical structures in collagen fibres with an ～26% improvement in the amplitude contrast in a fibre pitch. Our approach could be adapted to noisy and low resolution modalities such as micro‐nano CT and MRI, impacting precision of diagnosis and treatment of human diseases.     

Isotropic high-resolution three-dimensional confocal micro-rotation imaging for non-adherent living cells

Recently, micro-rotation confocal microscopy has enabled the acquisition of a sequence of micro-rotated images of nonadherent living cells obtained during a partially controlled rotation movement of the cell through the focal plane. Although we are now able to estimate the three-dimensional position of every optical section with respect to the cell frame, the reconstruction of the cell from the positioned micro-rotated images remains a last task that this paper addresses. This is not strictly an interpolation problem since a micro-rotated image is a convoluted two-dimensional map of a three-dimensional reality. It is rather a 'reconstruction from projection' problem where the term projection is associated to the PSF of the deconvolution process. Micro-rotation microscopy has a specific difficulty. It does not yield a complete coverage of the volume. In this paper, experiments illustrate the ability of the classical EM algorithm to deconvolve efficiently cell volume despite of the incomplete coverage. This cell reconstruction method is compared to a kernel-based method of interpolation, which does not take account explicitly the point-spread-function (PSF). It is also compared to the standard volume obtained from a conventional z-stack. Our results suggest that deconvolution of micro-rotation image series opens some exciting new avenues for further analysis, ultimately laying the way towards establishing an enhanced resolution 3D light microscopy.     

An application of spatial deconvolution to a capillary-based high-pressure chamber for fluorescence microscopy imaging

Capillary‐based high‐pressure chambers for which the wall serves as both the optical window and mechanical support have been reported for fluorescence microscopy imaging. Although capillary chambers are straightforward and economical to construct, the curved capillary wall introduces image aberrations. The significance of these aberrations in imaging sub‐cellular‐dimension objects has yet to be assessed. Using a capillary chamber that is routinely pressurized to between 20 and 30 MPa, a pressure range suitable for studying a wide variety of cellular processes, we demonstrate sub‐cellular‐dimension spatial resolution in the imaging of fluorescent micro‐spheres. Objectives with a range of numerical apertures (0.5–1.3) and working distances (0.1–7.4 mm) are considered. We show that spatial (or point‐spread function, PSF) deconvolution improves image contrast in capillary‐based images by comparing deconvolution results with those obtained from slide‐mounted controls. Furthermore, similar deconvolution results between a measured PSF and a calculated, flat‐geometry PSF indicate that the capillary wall is optically flat on cellular length scales. Results here facilitate the application of contemporary techniques in fluorescence microscopy to high‐pressure imaging fields.     

Super-Resolution Reconstruction via Multi-frame Defocused Images Based on PSF Estimation and Compressive Sensing

Image super-resolution reconstruction is an effective method to improve image resolution, but most reconstruction methods rely on the clear low resolution images ignoring the blurred images which are also effective observations of the scene. Aiming at the problem, a super-resolution reconstruction (SRR) method via multi-frame defocused images is proposed. Firstly, according to the image degraded model, we establish the cost function of the point spread function (PSF) and utilize the particle swarm optimization algorithm to estimate it. Then, based on the multi-frame defocused images and PSFs, a joint reconstruction model is established to realize SRR by compressive sensing (CS) theory. In the CS framework, only the interpolated version of the low-resolution image is used for training purpose and the K-Singular Value Decomposition method is used for dictionary training. In addition, to solve the edge effect problem, an internal blur matrix is constructed according to the image blurring process, and a weight coefficient is introduced in the patch splicing process. Experiments show that the proposed algorithm can accurately estimate the defocused image PSF and achieve a good reconstruction effect.     

3D restoration with multiple images acquired by a modified conventional microscope

A problem in high magnification microscopy is the blurring in the imaging of an object. In this article, we demonstrate a restoration technique that simultaneously makes use of the confocal image and the wide-field image. These images can be acquired by a modified conventional microscope. In front of the light-source, there is an array of pinholes. There are no pinholes at the detection plane. Instead, one or more pixels from the CCD camera are used, where the pinholes would have been. Using all pixels gives the wide-field image, but using a selected subset can give a confocal image. The array is used to speed up the process of acquiring the image. Note that the speed of acquisition is proportional to the number of pinholes. We show that the restoration from the two images can lead to a better result than using only one of the images. If this is the case, we show that a distance of 5 times the diameter of the pinholes can give the same results as a distance of 20 times after deconvolution. This offers an increase in acquisition time of a factor 16.     

Confocal microscopy and image processing in the study of plant nuclear structure

We describe measurements of the point spread function (PSF) for a confocal microscope and compare them with the PSF for a conventional (wide-field) fluorescence microscope. In situ hybridization with probes to telomere and ribosomal rDNA sequences, combined with three-dimensional (3-D) microscopy, has been used to study interphase nuclei in root tissue of Pisum sativum and Vicia faba. Nearly all the telomeres in both species are located at the nuclear envelope, and are highly clustered in the Vicia tissues, suggesting specific binding interactions. rDNA labelling in P. sativum shows four brightly staining knobs, corresponding to condensed regions of the rDNA genes from the two pairs of nucleolar organizer genes in this species, arranged approximately tetrahedrally around each nucleolus. Deconvolution using the measured PSFs can be used to improve these images, revealing a fibrous substructure in the perinucleolar knobs, and a large amount of interconnecting internal structure, which we suggest represents rDNA both in the fibrillar centres and also more diffuse, widely dispersed rDNA. Finally we show that accurate conventional data coupled with deconvolution can produce 3-D reconstructions comparable to those obtainable with confocal microscopy, but that the clearest images are obtained by applying deconvolution to the confocal data.     

A method of PSF generation for 3D brightfield deconvolution

This paper addresses the problem of 3D deconvolution of through focus widefield microscope datasets (Z‐stacks). One of the most difficult stages in brightfield deconvolution is finding the point spread function. A theoretically calculated point spread function (called a ‘synthetic PSF’ in this paper) requires foreknowledge of many system parameters and still gives only approximate results. A point spread function measured from a sub‐resolution bead suffers from low signal‐to‐noise ratio, compounded in the brightfield setting (by contrast to fluorescence) by absorptive, refractive and dispersal effects. This paper describes a method of point spread function estimation based on measurements of a Z‐stack through a thin sample. This Z‐stack is deconvolved by an idealized point spread function derived from the same Z‐stack to yield a point spread function of high signal‐to‐noise ratio that is also inherently tailored to the imaging system. The theory is validated by a practical experiment comparing the non‐blind 3D deconvolution of the yeast Saccharomyces cerevisiae with the point spread function generated using the method presented in this paper (called the ‘extracted PSF’) to a synthetic point spread function. Restoration of both high‐ and low‐contrast brightfield structures is achieved with fewer artefacts using the extracted point spread function obtained with this method. Furthermore the deconvolution progresses further (more iterations are allowed before the error function reaches its nadir) with the extracted point spread function compared to the synthetic point spread function indicating that the extracted point spread function is a better fit to the brightfield deconvolution model than the synthetic point spread function.     

Blind deconvolution of 3D fluorescence microscopy using depth‐variant asymmetric PSF

The 3D wide‐field fluorescence microscopy suffers from depth‐variant asymmetric blur. The depth‐variance and axial asymmetry are due to refractive index mismatch between the immersion and the specimen layer. The radial asymmetry is due to lens imperfections and local refractive index inhomogeneities in the specimen. To obtain the PSF that has these characteristics, there were PSF premeasurement trials. However, they are useless since imaging conditions such as camera position and refractive index of the specimen are changed between the premeasurement and actual imaging. In this article, we focus on removing unknown depth‐variant asymmetric blur in such an optical system under the assumption of refractive index homogeneities in the specimen. We propose finding few parameters in the mathematical PSF model from observed images in which the PSF model has a depth‐variant asymmetric shape. After generating an initial PSF from the analysis of intensities in the observed image, the parameters are estimated based on a maximum likelihood estimator. Using the estimated PSF, we implement an accelerated GEM algorithm for image deconvolution. Deconvolution result shows the superiority of our algorithm in terms of accuracy, which quantitatively evaluated by FWHM, relative contrast, standard deviation values of intensity peaks and FWHM. Microsc. Res. Tech. 79:480–494, 2016. © 2016 Wiley Periodicals, Inc.     

Dispersion, aberration and deconvolution in multi-wavelength fluorescence images

The wavelength dependence of the incoherent point spread function in a wide-field microscope was investigated experimentally. Dispersion in the sample and optics can lead to significant changes in the point spread function as wavelength is varied over the range commonly used in fluorescence microscopy. For a given sample, optical conditions can generally be optimized to produce a point spread function largely free of spherical aberration at a given wavelength. Unfortunately, deviations in wavelength from this value will result in spherically aberrated point spread functions. Therefore, when multiple fluorophores are used to localize different components in the same sample, the image of the distribution of at least one of the fluorophores will be spherically aberrated. This aberration causes a loss of intensity and resolution, thereby complicating the localization and analysis of multiple components in a multi-wavelength image. We show that optimal resolution can be restored to a spherically aberrated image by constrained, iterative deconvolution, as long as the spherical aberration in the point spread function used for deconvolution matches the aberration in the image reasonably well. The success of this method is essentially independent of the initial degree of spherical aberration in the image. Deconvolution of many biological images can be achieved by collecting a small library of spherically aberrated and unaberrated point spread functions, and then choosing a point spread function appropriate for deconvolving each image. The co-localization and relative intensities of multiple components can then be accurately studied in a multi-wavelength image.     

Blind deconvolution of 3D transmitted light brightfield micrographs

The blind deconvolution algorithm for 3D transmitted light brightfield (TLB) microscopy, published previously ( Holmes et al . Handbook of Biological Confocal Microscopy (1995), is summarized with example images. The main emphasis of this paper is to discuss more thoroughly the importance and usefulness of this method and to provide more detailed evidence, some being quantitative, of its necessity. Samples of horseradish peroxidase (HRP)-stained pyramidal neurones were prepared and evaluated for the ability to see fine structures clearly, including the dendrites and spines. It is demonstrated that the appearance of fine spine structure, and means of identifying spine categories, is made possible by using blind deconvolution. A comparison of images of the same sample from reflected light confocal microscopy, which is the conventional light microscopic way of viewing the 3D structure of these HRP-stained samples, shows that the blind deconvolution method is far superior for clearly showing the structure with less distortion and better resolution of the spines. The main significance of this research is that it is now possible to obtain clear images of 3D structure by light microscopy of absorbing stains. This is important because the TLB microscope is probably the most widely used modality in the life-science laboratory, yet, until now, there has been no reliable means for it to provide visualization of 3D structure clearly. The main importance of the blind deconvolution approach is that it obviates the need to measure the point spread function of the optical system, so that it now becomes realistic to provide a 3D light microscopic deconvolution method that can be pervasively used by microscopists.     

Application of regularized Richardson-Lucy algorithm for deconvolution of confocal microscopy images

Although confocal microscopes have considerably smaller contribution of out-of-focus light than widefield microscopes, the confocal images can still be enhanced mathematically if the optical and data acquisition effects are accounted for. For that, several deconvolution algorithms have been proposed. As a practical solution, maximum-likelihood algorithms with regularization have been used. However, the choice of regularization parameters is often unknown although it has considerable effect on the result of deconvolution process. The aims of this work were: to find good estimates of deconvolution parameters; and to develop an open source software package that would allow testing different deconvolution algorithms and that would be easy to use in practice. Here, Richardson-Lucy algorithm has been implemented together with the total variation regularization in an open source software package IOCBio Microscope. The influence of total variation regularization on deconvolution process is determined by one parameter. We derived a formula to estimate this regularization parameter automatically from the images as the algorithm progresses. To assess the effectiveness of this algorithm, synthetic images were composed on the basis of confocal images of rat cardiomyocytes. From the analysis of deconvolved results, we have determined under which conditions our estimation of total variation regularization parameter gives good results. The estimated total variation regularization parameter can be monitored during deconvolution process and used as a stopping criterion. An inverse relation between the optimal regularization parameter and the peak signal-to-noise ratio of an image is shown. Finally, we demonstrate the use of the developed software by deconvolving images of rat cardiomyocytes with stained mitochondria and sarcolemma obtained by confocal and widefield microscopes.     

基于跨尺度字典学习的图像盲解卷积算法

在模糊核未知情况下利用模糊图像对清晰图像进行复原称为图像盲解卷积问题,这是一个欠定逆问题,现有的大部分算法通过引入模糊核和清晰图像的先验知识来约束问题的解空间。本文提出了一种基于跨尺度字典学习的图像盲解卷积算法,采用降采样图像训练稀疏表示的字典,并将图像纹理区域在该字典下的稀疏表示作为正则化约束引入盲解卷积目标函数中。图像降采样过程减弱了图像的模糊程度,且图像中存在冗余的跨尺度相似块,利用更清晰的图像块训练字典能够更好地对清晰图像进行稀疏表示,减小稀疏表示误差;同时,由于在纹理区域清晰图像的稀疏表示误差小于模糊图像的稀疏表示误差,在该字典下对图像中的纹理块进行稀疏表示,使重建图像偏向清晰图像。本文的算法在Kohler数据集上复原结果的平均峰值信噪比为29.54 dB。在大量模糊图像上的实验验证了本文的算法能够有效解决大尺寸模糊核的复原,并具有良好的鲁棒性。     

Image deconvolution in spherical aberration-corrected high-resolution transmission electron microscopy

The method of image deconvolution developed previously for FEG high-resolution transmission electron microscope (HRTEM) without a spherical aberration (C(s)) corrector was for the first time applied to FEG HRTEM with a C(s)-corrector. The principle and the procedure of image deconvolution are briefly described. Four qualified [1 1 0] images of Si were selected from a through-focus series to perform image deconvolution. The projected potential is successfully derived from all the images, and the obtained "dumbbell" structure maps of Si [1 1 0] are in good agreement with the calculated potential map. The criterion of selecting qualified images for performing image deconvolution is indicated. The possibility of applying image deconvolution to defect study and to ab initio crystal structure determination is discussed.     

Parallel deconvolution of large 3D images obtained by confocal laser scanning microscopy

Various deconvolution algorithms are often used for restoration of digital images. Image deconvolution is especially needed for the correction of three‐dimensional images obtained by confocal laser scanning microscopy. Such images suffer from distortions, particularly in the Z dimension. As a result, reliable automatic segmentation of these images may be difficult or even impossible. Effective deconvolution algorithms are memory‐intensive and time‐consuming. In this work, we propose a parallel version of the well‐known Richardson–Lucy deconvolution algorithm developed for a system with distributed memory and implemented with the use of Message Passing Interface (MPI). It enables significantly more rapid deconvolution of two‐dimensional and three‐dimensional images by efficiently splitting the computation across multiple computers. The implementation of this algorithm can be used on professional clusters provided by computing centers as well as on simple networks of ordinary PC machines. Microsc. Res. Tech., 2010. © 2009 Wiley‐Liss, Inc.     

湍流退化红外图像降晰函数辨识

针对高速湍流场导致的红外成像模糊,提出了一种基于图像质量评价的湍流退化红外图像降晰函数辨识算法。利用先验知识将退化过程简化为可用参数描述的二维高斯函数形式,将退化图像分割为边缘区、纹理区和平坦区,计算不同支持域下复原图像在不同参数时的峰度值;利用曲率最大准则对得到的“峰度-参数”曲线进行相应的降晰函数参数估计,进而由支持域和对应的估计参数得到对应降晰函数并用于复原退化图像;最后对复原图像进行无参考图像质量评价,评价指标最高的复原图像对应的降晰函数即为最终辨识结果。实验结果表明：该算法能较好地辨识降晰函数参数和支持域大小,当退化图像信噪比大于30 dB时,估计参数与真实值的最大偏差小于±5%。该算法所得结果可以作为湍流退化红外图像其他复原算法的降晰函数起始估计。     

Effects of signal diffusion on x-ray phase contrast images

We discuss the problem of signal diffusion among neighbouring pixels in x-ray phase contrast imaging (XPCi) specifically for coded-aperture (CA) XPCi, but many of the discussed observations are directly transferable to other XPCi modalities. CA XPCi exploits the principle of pixel edge illumination by means of two CA masks. The first mask, placed in contact with the detector, creates insensitive regions between adjacent pixels; the second one, placed immediately before the sample, creates individual beams impinging on the boundaries between sensitive and insensitive regions on the detector, as created by the detector mask. In this way, edge illumination is achieved for all pixels of an area detector illuminated by a divergent and polychromatic beam generated by a conventional source. As the detector mask redefines the resolution properties of the detector, sample dithering can be used to effectively increase the system spatial resolution, without having to apply any post-processing procedure (e.g., deconvolution). This however creates artifacts in the form of secondary fringes (which have nothing to do with phase-related secondary fringes) if there is signal diffusion between adjacent pixels. In non-dithered images, signal diffusion between adjacent pixels causes a reduction in image contrast. This effect is investigated both theoretically and experimentally, and its direct implications on image quality are discussed. The interplay with the sample positioning with respect to the detector pixel matrix, which also has an effect on the obtained image contrast, is also discussed.     

改进的奇异值分解法估计图像点扩散函数

为了提高图像复原算法的性能,提出了一种改进的奇异值分解法估计图像的点扩散函数。从图像的退化离散模型出发,对图像进行逐层分块奇异值分解,并自动选取奇异值重组阶数以减少噪声对估计的影响。利用理想图像奇异值向量平均能谱指数模型,估计点扩散函数奇异值向量的频谱,再反傅里叶变换得到其时域结果。实验结果表明,该方法能在不同信噪比情况下估计成像系统的点扩散函数,估计结果比原有估计方法有所提高,有望为图像复原算法的预处理提供一种有效的手段。