Application of the split-gradient method to 3D image deconvolution in fluorescence microscopy

The methods of image deconvolution are important for improving the quality of the detected images in the different modalities of fluorescence microscopy such as wide‐field, confocal, two‐photon excitation and 4Pi. Because deconvolution is an ill‐posed problem, it is, in general, reformulated in a statistical framework such as maximum likelihood or Bayes and reduced to the minimization of a suitable functional, more precisely, to a constrained minimization, because non‐negativity of the solution is an important requirement. Next, iterative methods are designed for approximating such a solution. In this paper, we consider the Bayesian approach based on the assumption that the noise is dominated by photon counting, so the likelihood is of the Poisson‐type, and that the prior is edge‐preserving, as derived from a simple Markov random field model. By considering the negative logarithm of the a posteriori probability distribution, the computation of the maximum a posteriori (MAP) estimate is reduced to the constrained minimization of a functional that is the sum of the Csiszár I‐divergence and a regularization term. For the solution of this problem, we propose an iterative algorithm derived from a general approach known as split‐gradient method (SGM) and based on a suitable decomposition of the gradient of the functional into a negative and positive part. The result is a simple modification of the standard Richardson–Lucy algorithm, very easily implementable and assuring automatically the non‐negativity of the iterates. Next, we apply this method to the particular case of confocal microscopy for investigating the effect of several edge‐preserving priors proposed in the literature using both synthetic and real confocal images. The quality of the restoration is estimated both by computation of the Kullback–Leibler divergence of the restored image from the detected one and by visual inspection. It is observed that the noise artefacts are considerably reduced and desired characteristics (edges and minute features as islets) are retained in the restored images. The algorithm is stable, robust and tolerant at various noise (Poisson) levels. Finally, by remarking that the proposed method is essentially a scaled gradient method, a possible modification of the algorithm is briefly discussed in view of obtaining fast convergence and reduction in computational time.     

Noise suppression of point spread functions and its influence on deconvolution of three-dimensional fluorescence microscopy image sets

The point spread function (PSF) is of central importance in the image restoration of three-dimensional image sets acquired by an epifluorescent microscope. Even though it is well known that an experimental PSF is typically more accurate than a theoretical one, the noise content of the experimental PSF is often an obstacle to its use in deconvolution algorithms. In this paper we apply a recently introduced noise suppression method to achieve an effective noise reduction in experimental PSFs. We show with both simulated and experimental three-dimensional image sets that a PSF that is smoothed with this method leads to a significant improvement in the performance of deconvolution algorithms, such as the regularized least-squares algorithm and the accelerated Richardson–Lucy algorithm.     

基于HSI颜色空间的深度学习单幅图像去雾方法

针对传统的单幅图像去雾算法容易受到雾图先验知识制约导致颜色失真等问题,本文提出了一种基于HSI颜色空间的深度学习多尺度卷积神经网络单幅图像去雾方法,即通过设计深度学习网络结构来直接学习雾天图像与其无雾清晰图像色调、饱和度和亮度之间的映射关系,从而实现图像去雾.该方法首先将有雾图像从RGB颜色空间转换到HSI颜色空间,然后设计了一个端到端的多尺度全卷积神经网络模型,通过色调H、饱和度I、强度S三个不同的去雾子网分别进行多尺度提取,深度学习得到有雾图像与清晰图像之间的映射关系,从而恢复出无雾图像.实验结果表明,本文方法对于雾天图像具有良好的去雾效果,在主观评价和客观评价上均优于其它对比算法.     

Application of regularized Richardson-Lucy algorithm for deconvolution of confocal microscopy images

Although confocal microscopes have considerably smaller contribution of out-of-focus light than widefield microscopes, the confocal images can still be enhanced mathematically if the optical and data acquisition effects are accounted for. For that, several deconvolution algorithms have been proposed. As a practical solution, maximum-likelihood algorithms with regularization have been used. However, the choice of regularization parameters is often unknown although it has considerable effect on the result of deconvolution process. The aims of this work were: to find good estimates of deconvolution parameters; and to develop an open source software package that would allow testing different deconvolution algorithms and that would be easy to use in practice. Here, Richardson-Lucy algorithm has been implemented together with the total variation regularization in an open source software package IOCBio Microscope. The influence of total variation regularization on deconvolution process is determined by one parameter. We derived a formula to estimate this regularization parameter automatically from the images as the algorithm progresses. To assess the effectiveness of this algorithm, synthetic images were composed on the basis of confocal images of rat cardiomyocytes. From the analysis of deconvolved results, we have determined under which conditions our estimation of total variation regularization parameter gives good results. The estimated total variation regularization parameter can be monitored during deconvolution process and used as a stopping criterion. An inverse relation between the optimal regularization parameter and the peak signal-to-noise ratio of an image is shown. Finally, we demonstrate the use of the developed software by deconvolving images of rat cardiomyocytes with stained mitochondria and sarcolemma obtained by confocal and widefield microscopes.     

Digital image tiles: a method for the processing of large sections

Segmentation of large areas of light microscopic slides into N by N fields, and each of these fields into M digital image tiles, allows the scanning, storage and digital processing of large images. Any of the original N² fields or composites of M adjacent tiles can be recalled to the video display for analysis. Developed procedures for use on a microscope equipped with a precision scanning stage allow registration of the image coordinates (X-Y) for any original or composite field and the alignment of one of these fields along the depth (Z) axis by means of external, machined fiducial marks in serial sections. To facilitate work whenever unavoidable, we have incorporated methods for digital image panning and zooming (changes of magnification) and discuss their use and implications.     

A method of PSF generation for 3D brightfield deconvolution

This paper addresses the problem of 3D deconvolution of through focus widefield microscope datasets (Z‐stacks). One of the most difficult stages in brightfield deconvolution is finding the point spread function. A theoretically calculated point spread function (called a ‘synthetic PSF’ in this paper) requires foreknowledge of many system parameters and still gives only approximate results. A point spread function measured from a sub‐resolution bead suffers from low signal‐to‐noise ratio, compounded in the brightfield setting (by contrast to fluorescence) by absorptive, refractive and dispersal effects. This paper describes a method of point spread function estimation based on measurements of a Z‐stack through a thin sample. This Z‐stack is deconvolved by an idealized point spread function derived from the same Z‐stack to yield a point spread function of high signal‐to‐noise ratio that is also inherently tailored to the imaging system. The theory is validated by a practical experiment comparing the non‐blind 3D deconvolution of the yeast Saccharomyces cerevisiae with the point spread function generated using the method presented in this paper (called the ‘extracted PSF’) to a synthetic point spread function. Restoration of both high‐ and low‐contrast brightfield structures is achieved with fewer artefacts using the extracted point spread function obtained with this method. Furthermore the deconvolution progresses further (more iterations are allowed before the error function reaches its nadir) with the extracted point spread function compared to the synthetic point spread function indicating that the extracted point spread function is a better fit to the brightfield deconvolution model than the synthetic point spread function.     

Strategies for optimum use of superposition diffractogram in scanning electron microscopy

Characteristics of the superposition diffractogram used for precisely estimating scanning electron microscopy (SEM) resolution are investigated. It is shown that the choice of pixel density to satisfy the sampling theorem, the direction of scanning, the choice of image shift direction, the properties of the specimen, the effect of external disturbances such as vibration and stray magnetic fields, and the effect of the window function required in the Fourier transform, are all factors which must be considered in order to make the superposition diffractogram a practical technique. An additional important improvement required to exploit fully the ability of the superposition diffractogram, which potentially is very high, is a special scanning mode which employs a digital scan generator, and digital image processing technology with autocorrelation functions.     

An open‐source deconvolution software package for 3‐D quantitative fluorescence microscopy imaging

Deconvolution techniques have been widely used for restoring the 3‐D quantitative information of an unknown specimen observed using a wide‐field fluorescence microscope. Deconv , an open‐source deconvolution software package, was developed for 3‐D quantitative fluorescence microscopy imaging and was released under the GNU Public License. Deconv provides numerical routines for simulation of a 3‐D point spread function and deconvolution routines implemented three constrained iterative deconvolution algorithms: one based on a Poisson noise model and two others based on a Gaussian noise model. These algorithms are presented and evaluated using synthetic images and experimentally obtained microscope images, and the use of the library is explained. Deconv allows users to assess the utility of these deconvolution algorithms and to determine which are suited for a particular imaging application. The design of Deconv makes it easy for deconvolution capabilities to be incorporated into existing imaging applications.     

织物密度的数字图像检测方法

针对织物的纱线密度检测,提出一种对织物图像,信息进行检测的方法。通过数字相机获得织物图像,对织物图像进行最优二值化处理。利用像素黑白颜色区分织物上的纱线和间隙,通过扫描经线和纬线方向的像素颜色值获得纱线处像素颜色特性,从而计算出经线方向和纬线方向的纱线数目。根据各方向最大像素数得织物的大小,即可得知织物的密度。     

A method for personal expertise-independent evaluation of image resolution in scanning electron microscopy

The two conventional methods currently employed for the evaluation of image resolution in scanning electron microscopy are the gap method and a fast Fourier transform (FFT) method. These can be highly dependent on personal expertise on the distinction between signal information and noise contained in a micrograph. Hence, the present paper proposes an alternative method (referred to as a contrast-to-gradient (CG) method) that can determine the image resolution of a micrograph without requiring personal expertise on the judgment of noise. The image resolution in the CG method is defined as a weighted harmonic mean of the local resolution, which is proportional to the quotient of the threshold contrast divided by the local gradient. The local gradient is calculated from the quadratic function that best fits the local pixel intensities over 5 x 5 pixels. It has been shown that the CG method, compared with the FFT method, has a broader range of applications for various types of images, such as low-contrast, noise-containing, filter-processed, highly directional, and quasi-periodic feature images.     

A software tool for tomographic axial superresolution in STED microscopy

A method for generating three‐dimensional tomograms from multiple three‐dimensional axial projections in STimulated Emission Depletion (STED) superresolution microscopy is introduced. Our STED< method, based on the use of a micromirror placed on top of a standard microscopic sample, is used to record a three‐dimensional projection at an oblique angle in relation to the main optical axis. Combining the STED< projection with the regular STED image into a single view by tomographic reconstruction, is shown to result in a tomogram with three‐to‐four‐fold improved apparent axial resolution. Registration of the different projections is based on the use of a mutual‐information histogram similarity metric. Fusion of the projections into a single view is based on Richardson‐Lucy iterative deconvolution algorithm, modified to work with multiple projections. Our tomographic reconstruction method is demonstrated to work with real biological STED superresolution images, including a data set with a limited signal‐to‐noise ratio (SNR); the reconstruction software (SuperTomo) and its source code will be released under BSD open‐source license.     

基于双重模型客观评价微光像增强器的分辨力

以光学调制度模型和归一化互相关模型为基础,以模板匹配和光学调制度双重判据为核心算法,研究了微光像增强器分辨力的客观评价方法,构建了微光像增强器分辨力客观评价系统.光学调制度模型和归一化互相关模型既互相独立又相辅相成.首先,利用归一化互相关模型中的模板匹配系数进行定位和初步评价;然后,用光学调制度模型进行定量分析.这种先定性后定量的图像处理模式提高了微光像增强器分辨力测量的准确性和重复性,实现了对微光像增强器分辨力客观、准确的评价,避免了目视观察法受人的主观因素影响的弊端.实验结果表明,由CCD采集得到的微光像增强器分辨力与人眼观测结果具有较好的一致性,提高了本评价方法的客观性和准确性,该方法也可推广至CCD、ICCD、EMCCD等可见光探测成像系统分辨力的客观评价.     

Application of an advanced maximum likelihood estimation restoration method for enhanced‐resolution and contrast in second‐harmonic generation microscopy

Second‐harmonic generation (SHG) microscopy has gained popularity because of its ability to perform submicron, label‐free imaging of noncentrosymmetric biological structures, such as fibrillar collagen in the extracellular matrix environment of various organs with high contrast and specificity. Because SHG is a two‐photon coherent scattering process, it is difficult to define a point spread function (PSF) for this modality. Hence, compared to incoherent two‐photon processes like two‐photon fluorescence, it is challenging to apply the various PSF‐engineering methods to improve the spatial resolution to be close to the diffraction limit. Using a synthetic PSF and application of an advanced maximum likelihood estimation (AdvMLE) deconvolution algorithm, we demonstrate restoration of the spatial resolution in SHG images to that closer to the theoretical diffraction limit. The AdvMLE algorithm adaptively and iteratively develops a PSF for the supplied image and succeeds in improving the signal to noise ratio (SNR) for images where the SHG signals are derived from various sources such as collagen in tendon and myosin in heart sarcomere. Approximately 3.5 times improvement in SNR is observed for tissue images at depths of up to ～480 nm, which helps in revealing the underlying helical structures in collagen fibres with an ～26% improvement in the amplitude contrast in a fibre pitch. Our approach could be adapted to noisy and low resolution modalities such as micro‐nano CT and MRI, impacting precision of diagnosis and treatment of human diseases.     

基于图像处理的轮对磨耗值检测方法的研究

提出了如何利用图像处理技术进行轮对磨耗值检测的一种方法,详细介绍了系统结构和图像处理的整个过程,从而实现了轮对磨耗值的非接触测量。测量精度可以达到0．1mm。     

基于不同积分时间帧累加的红外图像超帧方法

针对大多数常规红外成像系统单帧成帧时间内探测器组件响应动态范围有限,以及超帧算法需要借助其他复杂装置或专用电路的不足,提出了一种基于不同积分时间帧累加的红外图像超帧处理方法,采用电子学处理方法对整个超帧处理过程进行了研究。分析了限制红外探测器响应动态范围的原因,简单介绍了帧积分方法的原理和利弊。然后对不同积分时间下获得的图像进行了非均匀性校正、帧信息融合和新帧映射。实验结果表明:本文方法提高了系统的噪声等效温差(NETD),捕获了更多的场景原始图像信息,丰富了图像的灰度层次,增加了图像的信息熵。该方法能够等效拓展探测器的响应动态范围,增加场景原始信息的捕获,提高成像系统的输出信号信噪比以及探测灵敏度。     

The point-spread function of a confocal microscope: its measurement and use in deconvolution of 3-D data

We have measured the point-spread function (PSF) for an MRC-500 confocal scanning laser microscope using subresolution fluorescent beads. PSFs were measured for two lenses of high numerical aperture—the Zeiss plan-neofluar 63 × water immersion and Leitz plan-apo 63 × oil immersion—at three different sizes of the confocal detector aperture. The measured PSFs are fairly symmetrical, both radially and axially. In particular there is considerably less axial asymmetry than has been demonstrated in measurements of conventional (non-confocal) PSFs. Measurements of the peak width at half-maximum peak height for the minimum detector aperture gave approximately 0·23 and 0·8 μm for the radial and axial resolution respectively (4·6 and 15·9 in dimensionless optical units). This increased to 0·38 and 1·5 μm (7·5 and 29·8 in dimensionless units) for the largest detector aperture examined. The resulting optical transfer functions (OTFs) were used in an iterative, constrained deconvolution procedure to process three-dimensional confocal data sets from a biological specimen—pea root cells labelled in situ with a fluorescent probe to ribosomal genes. The deconvolution significantly improved the clarity and contrast of the data. Furthermore, the loss in resolution produced by increasing the size of the detector aperture could be restored by the deconvolution procedure. Therefore for many biological specimens which are only weakly fluorescent it may be preferable to open the detector aperture to increase the strength of the detected signal, and thus the signal-to-noise ratio, and then to restore the resolution by deconvolution.     

面向挤出中聚合物形态的层析图像处理方法的研究

依据光学信息论中的多维随机变量的联合熵、条件熵、平均互信息之间关系和互信息的链式法则,提出了面向挤出中聚合物形态的层析图像处理方法.它包括层析图像中的背景与分离、目标标度变换、亚像素插补的图像预处理和多模式目标特征图像的提取和测量.实验表明,通过该方法不仅能将目标形态从目标与背景之比为1∶9的层析图像中分辨出来,而且能提高目标图像的空间分辨率30%,解决了由于目标图像信息量小、对比度差而使得层析图像测量方法难以在工程中应用的问题.     

Formation of a three-dimensional image of the Earth surface by an airborne radio-thermal locator

Methods for forming a three-dimensional image of the Earth surface and objects on this surface with the use of an airborne multichannel radio-thermal location station are proposed. These methods are based on range measurement, image reconstruction, and correlation-extreme combination of frames.     

A method using an on-line digital computer for the improvement of resolution of backscattered electron images

In principle, the resolution of backscattered electron (BSE) images can be little improved, even though an infinitely small beam size is achieved by various improvements in the intrinsic instrument. In order to circumvent this problem, a method is proposed which utilizes an on-line digital computer for the image recording and processing. The major image-processing tools are reduction, expansion, super-imposition with matching of the images, and high-emphasis filtering in the Fourier domain. By using various combinations of these techniques, the resolution of BSE images has been significantly improved. The validity of these improved images has been confirmed. In the case of a BSE image with too wide a dynamic range, both the present method and digital homomorphic filtering provide successful results.     

基于遗传算法的生物组织图像最佳挖取点寻优

生物组织微阵列芯片的制备过程中,需要从包埋有生物组织的蜡块中挖取一块直径为0.5~1.5 mm的组织蜡柱。为了实现自动化作业,研究了基于数字图像处理的组织蜡柱最佳挖取点计算机寻优策略。确定了问题的数学描述为在选定组织图像区域中寻找最大内切圆的圆心,提出了基于遗传算法的寻优算法。重点探讨了坐标点编码,参数选取和遗传操作方法等。研究了个体适应度快速计算方法,初始种群选择等算法改进方案。经过实际编程反复数次实验,结果表明迭代60次算法完全收敛,证明其搜索速度和稳定性都比较好,完全满足自动化作业的在线计算要求。