Electron tomography combining ESEM and STEM: a new 3D imaging technique

This paper presents the development and the application of a new electron tomography technique based on STEM (Scanning Transmission Electron Microscopy) configuration in ESEM (Environmental Scanning Electron Microscopy). This combination provides a new approach for the characterization of the 3D structure of materials, as it optimizes a compromise between the resolution level of a few tens of nm and the large tomogram size due to the high thickness of transparency. The method is well adapted for non-conductive samples, and exhibits good contrast even for materials with low atomic number. The paper describes the development of a dedicated stage for this new tomography technique. Taking advantage of the size of the ESEM chamber, the range of tilt angles is not limited by the space around the sample. The performance of this device is illustrated through the three-dimensional characterization of samples issued from materials science and chosen to illustrate the results in resolution, contrast and thickness.     

A 2D MEMS mirror with sidewall electrodes applied for confocal MACROscope imaging

This paper presents microelectromechanical system micromirrors with sidewall electrodes applied for use as a Confocal MACROscope for biomedical imaging. The MACROscope is a fluorescence and brightfield confocal laser scanning microscope with a very large field of view. In this paper, a microelectromechanical system mirror with sidewall electrodes replaces the galvo-scanner and XYZ-stage to improve the confocal MACROscope design and obtain an image. Two micromirror-based optical configurations are developed and tested to optimize the optical design through scanning angle, field of view and numerical aperture improvement. Meanwhile, the scanning frequency and control waveform of the micromirror are tested. Analysing the scan frequency and waveform becomes a key factor to optimize the micromirror-based confocal MACROscope. When the micromirror is integrated into the MACROscope and works at 40 Hz, the micromirror with open-loop control possesses good repeatability, so that the synchronization among the scanner, XYZ-stage and image acquisition can be realized. A laser scanning microscope system based on the micromirror with 2 μm width torsion bars was built and a 2D image was obtained as well. This work forms the experimental basis for building a practical confocal MACROscope.     

Algorithms for accurate 3D registration of neuronal images acquired by confocal scanning laser microscopy

This paper presents automated and accurate algorithms based on high‐order transformation models for registering three‐dimensional (3D) confocal images of dye‐injected neurons. The algorithms improve upon prior methods in several ways, and meet the more stringent image registration needs of applications such as two‐view attenuation correction recently developed by us. First, they achieve high accuracy (≈ 1.2 voxels, equivalent to 0.4 µm) by using landmarks, rather than intensity correlations, and by using a high‐dimensional affine and quadratic transformation model that accounts for 3D translation, rotation, non‐isotropic scaling, modest curvature of field, distortions and mechanical inconsistencies introduced by the imaging system. Second, they use a hierarchy of models and iterative algorithms to eliminate potential instabilities. Third, they incorporate robust statistical methods to achieve accurate registration in the face of inaccurate and missing landmarks. Fourth, they are fully automated, even estimating the initial registration from the extracted landmarks. Finally, they are computationally efficient, taking less than a minute on a 900‐MHz Pentium III computer for registering two images roughly 70 MB in size. The registration errors represent a combination of modelling, estimation, discretization and neuron tracing errors. Accurate 3D montaging is described; the algorithms have broader applicability to images of vasculature, and other structures with distinctive point, line and surface landmarks.     

Three-dimensional imaging in fluorescence by confocal scanning microscopy

The improved resolution and sectioning capability of a confocal microscope make it an ideal instrument for extracting three-dimensional information especially from extended biological specimens. The imaging properties, also with finite detection pinholes are considered and a number of biological applications demonstrated.     

3D imaging of nanomaterials by discrete tomography

The field of discrete tomography focuses on the reconstruction of samples that consist of only a few different materials. Ideally, a three-dimensional (3D) reconstruction of such a sample should contain only one grey level for each of the compositions in the sample. By exploiting this property in the reconstruction algorithm, either the quality of the reconstruction can be improved significantly, or the number of required projection images can be reduced. The discrete reconstruction typically contains fewer artifacts and does not have to be segmented, as it already contains one grey level for each composition.     

Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution

Confocal laser scanning microscopy is a powerful and popular technique for 3D imaging of biological specimens. Although confocal microscopy images are much sharper than standard epifluorescence ones, they are still degraded by residual out-of-focus light and by Poisson noise due to photon-limited detection. Several deconvolution methods have been proposed to reduce these degradations, including the Richardson-Lucy iterative algorithm, which computes maximum likelihood estimation adapted to Poisson statistics. As this algorithm tends to amplify noise, regularization constraints based on some prior knowledge on the data have to be applied to stabilize the solution. Here, we propose to combine the Richardson-Lucy algorithm with a regularization constraint based on Total Variation, which suppresses unstable oscillations while preserving object edges. We show on simulated and real images that this constraint improves the deconvolution results as compared with the unregularized Richardson-Lucy algorithm, both visually and quantitatively.     

Parallel deconvolution of large 3D images obtained by confocal laser scanning microscopy

Various deconvolution algorithms are often used for restoration of digital images. Image deconvolution is especially needed for the correction of three‐dimensional images obtained by confocal laser scanning microscopy. Such images suffer from distortions, particularly in the Z dimension. As a result, reliable automatic segmentation of these images may be difficult or even impossible. Effective deconvolution algorithms are memory‐intensive and time‐consuming. In this work, we propose a parallel version of the well‐known Richardson–Lucy deconvolution algorithm developed for a system with distributed memory and implemented with the use of Message Passing Interface (MPI). It enables significantly more rapid deconvolution of two‐dimensional and three‐dimensional images by efficiently splitting the computation across multiple computers. The implementation of this algorithm can be used on professional clusters provided by computing centers as well as on simple networks of ordinary PC machines. Microsc. Res. Tech., 2010. © 2009 Wiley‐Liss, Inc.     

Scanning slit aperture confocal microscopy for three-dimensional imaging

A scanning confocal microscope using stationary slit apertures of variable width is described. Scanning is achieved by employing an oscillating mirror galvanometer capable of scanning images at 160 frames/s. The microscope depth-of-focus is characterized for several slit widths and shown to approach diffraction-limited performance as width decreases. Reduction of flare in images taken with narrow slit widths is demonstrated. In addition, the optical sectioning capability of the system is demonstrated using polymer casts of the vasculature of rat kidney.     

Aberration correction for confocal imaging in refractive-index-mismatched media

A major limitation to the use of confocal microscopes to image thick biological tissue lies in the dramatic reduction in both signal level and resolution when focusing deep into a refractive-index-mismatched specimen. This limitation may be overcome by measuring the wavefront aberration and pre-shaping the input beam so as to cancel the effects of aberration. We consider the images of planar and point objects in brightfield, single-photon fluorescence and two-photon fluorescence imaging. In all cases, the specimens are imaged using an oil-immersion objective through various thicknesses of water. The question of finite-sized pinhole is addressed and it is found, in general, that it is sufficient to correct only the first two or three orders of spherical aberration to restore adequate image signal level and optical resolution, at imaging depths of up to 50-100 wavelengths.     

Inexpensive, high-quality optical relay for use in confocal scanning beam imaging

An inexpensive, high optical-quality relay lens made up of two eyepieces arranged in an afocal assembly for use in confocal scanning laser imaging is described. In the past we have used relays, within our confocal microscopes, made up of achromats with long focal lengths (> or = 10 cm), which take up large optical tracks and suffer from significant amounts of astigmatism and curvature of field. We quantify aberrations associated with achromat and eyepiece relays using CODE V optical design and analysis software. The eyepiece relay is found to be more compact, better corrected, and not significantly more expensive than its achromat counterpart. In addition to being used to interconnect two scanning mirrors optically as well as scanning mirrors with microscope objectives, it can form part of the optics in a confocal scanning laser MACROscope-Microscope system (Biomedical Photometrics, Inc., Waterloo, Ontario, Canada). Due to design constraints, the MACROscope-Microscope system cannot incorporate a conventional wide-field microscope into its structure such as is done in most commercial confocal microscopes. The eyepiece relay is used as a stand-alone, compact optical link between the scanning mirrors and the microscope objective. This consequently makes the MACROscope-Microscope system more compact and easier to commercialize.     

Three-dimensional imaging in confocal systems

We discuss the origin of the three-dimensional imaging characteristics of confocal optical systems. Several methods of information display are considered. The important practical question concerning the correct choice of limiting detector aperture is also considered.     

多光谱3D成像方法

已有3D成像方法难以实现单目、单帧图像条件下同时获取场景图像及深度信息,也不能兼具时间效率高、体积紧凑、能耗低等优点。为此,创新地提出多光谱3D成像方法,通过具有纵向色散的光学成像镜头与快照式多光谱图像传感器两部分构成图像采集系统,使用离焦深度还原算法获取深度信息。其基本原理为:首先,增强纵向色差光学镜头使得同一物点在不同光谱波段图像上的成像离焦程度不同;其次,快照式窄带多光谱图像传感器单帧曝光同时获取多幅窄带光谱图像;再通过离焦深度还原算法根据多光谱图像边缘梯度获取3D信息。实验采用纵向色散增强型光学成像系统及快照式多光谱相机捕获450±10 nm、525±10 nm、620±10 nm 3通道光谱图像,对5 m内场景进行3D深度恢复,获得了深度误差不高于5 cm的测量结果。实验结果表明多光谱3D视觉方法可以实现单目、所提单帧图像的深度估计。该方法能同时获得视觉及深度信息且无需空间位置配准及预先深度刻度,单帧图像处理平均耗时0. 186 s,图像采集系统尺寸为120 mm×77 mm×65 mm,其工作功率约为10 W,兼具时间效率高、体积紧凑、能耗低等优点。因此,所提方法有望在无人驾驶及智能机器人等领域获得广泛应用。     

Multipoint scanning dual‐detection confocal microscopy for fast 3D volumetric measurement

We propose a multipoint scanning dual‐detection confocal microscopy (MS‐DDCM) system for fast 3D volumetric measurements. Unlike conventional confocal microscopy, MS‐DDCM can accomplish surface profiling without axial scanning. Also, to rapidly obtain 2D images, the MS‐DDCM employs a multipoint scanning technique, with a digital micromirror device used to produce arrays of effective pinholes, which are then scanned. The MS‐DDCM is composed of two CCDs: one collects the conjugate images and the other collects nonconjugate images. The ratio of the axial response curves, measured by the two detectors, provides a linear relationship between the height of the sample surface and the ratio of the intensity signals. Furthermore, the difference between the two images results in enhanced contrast. The normalising effect of the MS‐DDCM provides accurate sample heights, even when the reflectance distribution of the surface varies. Experimental results confirmed that the MS‐DDCM achieved high‐speed surface profiling with improved image contrast capability.     

Subpixel position measurement using 1D, 2D and 3D centroid algorithms with emphasis on applications in confocal microscopy

The accuracy with which centroid algorithms in 1D, 2D and 3D can estimate an object's position has been investigated. Three factors that can influence the method's accuracy have been investigated: systematic error of the algorithm, influence of photon noise and the influence of perturbations such as scanning nonlinearity. The variation of the accuracy with parameters that are relevant for confocal microscopy, such as object diameter and photon noise/pixel, has been considered. Theory and simulations presented show that the variation of the accuracy with respect to such parameters can differ drastically between the 1D, 2D and 3D cases. Experiments performed using microspheres show that the magnitudes of the three types of error can be approximately the same under normal operating conditions and that it is therefore necessary to take all three into account when assessing the total error.     

An automatic segmentation algorithm for 3D cell cluster splitting using volumetric confocal images

With the rapid advance of three-dimensional (3D) confocal imaging technology, more and more 3D cellular images will be available. Segmentation of intact cells is a critical task in automated image analysis and quantification of cellular microscopic images. One of the major complications in the automatic segmentation of cellular images arises due to the fact that cells are often closely clustered. Several algorithms are proposed for segmenting cell clusters but most of them are 2D based. In other words, these algorithms are designed to segment 2D cell clusters from a single image. Given 2D segmentation methods developed, they can certainly be applied to each image slice with the 3D cellular volume to obtain the segmented cell clusters. Apparently, in such case, the 3D depth information with the volumetric images is not really used. Often, 3D reconstruction is conducted after the individualized segmentation to build the 3D cellular models from segmented 2D cellular contours. Such 2D native process is not appropriate as stacking of individually segmented 2D cells or nuclei do not necessarily form the correct and complete 3D cells or nuclei in 3D. This paper proposes a novel and efficient 3D cluster splitting algorithm based on concavity analysis and interslice spatial coherence. We have taken the advantage of using the 3D boundary points detected using higher order statistics as an input contour for performing the 3D cluster splitting algorithm. The idea is to separate the touching or overlapping cells or nuclei in a 3D native way. Experimental results show the efficiency of our algorithm for 3D microscopic cellular images.     

Quantitative 3D imaging of yeast by hard X-ray tomography

Full-field hard X-ray tomography could be used to obtain three-dimensional (3D) nanoscale structures of biological samples. The image of the fission yeast, Schizosaccharomyces pombe, was clearly visualized based on Zernike phase contrast imaging technique and heavy metal staining method at a spatial resolution better than 50 nm at the energy of 8 keV. The distributions and shapes of the organelles during the cell cycle were clearly visualized and two types of organelle were distinguished. The results for cells during various phases were compared and the ratios of organelle volume to cell volume can be analyzed quantitatively. It showed that the ratios remained constant between growth and division phase and increased strongly in stationary phase, following the shape and size of two types of organelles changes. Our results demonstrated that hard X-ray microscopy was a complementary method for imaging and revealing structural information for biological samples.     

Wavelength and alignment tests for confocal spectral imaging systems

Confocal spectral imaging (CSI) microscope systems now on the market delineate multiple fluorescent proteins, labels, or dyes within biological specimens by performing spectral characterizations. However, we find that some CSI present inconsistent spectral profiles of reference spectra within a particular system as well as between related and unrelated instruments. We also find evidence of instability that, if not diagnosed, could lead to inconsistent data. This variability confirms the need for diagnostic tools to provide a standardized, objective means of characterizing instability, evidence of misalignment, as well as performing calibration and validation functions. Our protocol uses an inexpensive multi-ion discharge lamp (MIDL) that contains Hg+, Ar+, and inorganic fluorophores that emit distinct, stable spectral features, in place of a sample. An MIDL characterization verifies the accuracy and consistency of a CSI system and validates acquisitions of biological samples. We examined a total of 10 CSI systems, all of which displayed spectral inconsistencies, enabling us to identify malfunctioning subsystems. Only one of the 10 instruments met its optimal performance expectations. We have found that using a primary light source that emits an absolute standard "reference spectrum" enabled us to diagnose instrument errors and measure accuracy and reproducibility under normalized conditions. Using this information, a CSI operator can determine whether a CSI system is working optimally and make objective comparisons with the performance of other CSI systems. It is evident that if CSI systems of a similar make and model were standardized to reveal the same spectral profile from a standard light source, then researchers could be confident that real-life experimental findings would be repeatable on any similar system.     

A confocal fibre scanning microscope for magnetic domain imaging

A compact confocal single-mode optical fibre scanning microscope for imaging magnetic domain structures, based on the polar magneto-optic Kerr effect, has been developed. The images obtained correspond to those obtained using single mirror scanning but this design offers a more compact structure and can be made more immune from system depolarization which makes two-axis mirror scanning difficult to implement when very small changes in polarization need to be detected.     

Position-sensitive diffractive imaging in STEM by an automated chaining diffraction algorithm

The diffractive imaging process used for retrieval of an aberration-free exit-wave function of a complex-valued object is optimized with a newly developed automated chaining diffraction (ACD) algorithm. Our algorithm enables automatic recovery of the amplitude and phase of the complex-valued objects with diffraction-limited resolution, starting from selected-area electron diffraction (SAED) patterns recorded from partially overlapping regions in STEM/CTEM. Based on a ‘differential map’ (DM) approach, the ACD algorithm meets very general requirements and, similar to ‘hybrid input–output’ (HIO) algorithm, can be applied to non-periodic, real or complex structures. In contrast to many other algorithms, it is not limited by the object's finite size or tight object support. Wide-field-of-view reconstructions for the complex-object-wave amplitude and phase made with ACD algorithm from SAED patterns down to sub-Angström resolution show the potential of diffractive imaging for quantitative analysis of functional materials at different length scales in terms of absorption and scattering mechanisms. The method can be applied also for imaging magnetic properties of samples by the electron or neutron microscopy and/or imaging of non-periodic objects with X-ray microscopy.