Multidirectional focus measure for accurate three-dimensional shape recovery of microscopic objects

Shape from focus (SFF) is a technique to recover the shape of an object from multiple images taken at various focus settings. Most of conventional SFF techniques compute focus value of a pixel by applying one of focus measure operators on neighboring pixels on the same image frame. However, in the optics with limited depth of field, neighboring pixels of an image have different degree of focus for curved objects, thus the computed focus value does not reflect the accurate focus level of the pixel. Ideally, an accurate focus value of a pixel needs to be measured from the neighboring pixels lying on tangential plane of the pixel in image space. In this article, a tangential plane on each pixel location (i, j) in image sensor is searched by selecting one of five candidate planes based on the assumption that the maximum variance of focus values along the optical axis is achieved from the neighborhood lying on tangential plane of the pixel (i, j). Then, a focus measure operator is applied on neighboring pixels lying on the searched plane. The experimental results on both the synthetic and real microscopic objects show the proposed method produces more accurate three-dimensional shape in comparison to conventional SFF method that applies focus measures on original image planes.     

Two‐dimensional structured illumination microscopy

In widefield fluorescence microscopy, images from all but very flat samples suffer from fluorescence emission from layers above or below the focal plane of the objective lens. Structured illumination microscopy provides an elegant approach to eliminate this unwanted image contribution. To this end a line grid is projected onto the sample and phase images are taken at different positions of the line grid. Using suitable algorithms ‘quasi‐confocal images’ can be derived from a given number of such phase‐images. Here, we present an alternative structured illumination microscopy approach, which employs two‐dimensional patterns instead of a one‐dimensional one. While in one‐dimensional structured illumination microscopy the patterns are shifted orthogonally to the pattern orientation, in our two‐dimensional approach it is shifted at a single, pattern‐dependent angle, yet it already achieves an isotropic power spectral density with this unidirectional shift, which otherwise would require a combination of pattern‐shift and ‐rotation. Moreover, our two‐dimensional approach also yields a better signal‐to‐noise ratio in the evaluated image.     

Super‐resolution optical microscopy based on scannable cantilever‐combined microsphere

We report an ingenious method of super‐resolution optical microscopy utilizing scannable cantilever‐combined microsphere. By scanning the microsphere over the sample surface in a cantilever‐combined microsphere‐sample contact state, super‐resolution images can be acquired at arbitrary sample regions through near‐field information collection by the microsphere. In addition, such a state can effectively reduce the possibility of breaking the cantilever and damaging the microsphere or sample surface. This work has developed a new method and technique of sub‐diffraction‐limit optical microscopy, and can be practically applied in various fields of micro/nanoscopy. Microsc. Res. Tech. 78:1128–1132, 2015. © 2015 Wiley Periodicals, Inc.     

Label‐free identification of the microstructure of rat spinal cords based on nonlinear optical microscopy

The spinal cord is a vital link between the brain and the body and mainly comprises neurons, glial cells and nerve fibres. In this work, nonlinear optical (NLO) microscopy based on intrinsic tissue properties was employed to label‐freely analyze the cells and matrix in spinal cords at a molecular level. The high‐resolution and high‐contrast NLO images of unstained spinal cords demonstrate that NLO microscopy has the ability to show the microstructure of white and grey matter including ventral horn, intermediate area, dorsal horns, ventral column, lateral column and dorsal column. Neurons with various sizes were identified in grey matter by dark spots of nonfluorescent nuclei encircled by cytoplasm‐emitting two‐photon excited fluorescence signals. Nerve fibres and neuroglias were observed in white matter. Besides, the spinal arteries were clearly presented by NLO microscopy. Using spectral and morphological information, this technique was proved to be an effective tool for label‐freely imaging spinal cord tissues, based on endogenous signals in biological tissue. With future development, we foresee promising applications of the NLO technique for in vivo, real‐time assessment of spinal cord diseases or injures.     

MRT letter: A total variation based method for 3D shape recovery of microscopic objects through image focus

Generally, shape from focus methods use a single focus measure to compute focus quality and to obtain an initial depth map of an object. However, different focus measures perform differently in diverse conditions. Therefore, it is hard to get accurate 3D shape based on a single focus measure. In this article, we propose a total variation based method for recovering 3D shape of an object by combining multiple depth hypothesis obtained through different focus measures. Improved performance of the proposed method is evaluated by conducting several experiments using images of synthetic and real microscopic objects. Comparative analysis demonstrates the effectiveness of the proposed approach. Microsc. Res. Tech. 76:877–881, 2013. © 2013 Wiley Periodicals, Inc.     

MRT Letter: Two‐photon excitation‐based 2pi Light‐sheet system for nano‐lithography

We propose two‐photon excitation‐based light‐sheet technique for nano‐lithography. The system consists of 2 ‐configured cylindrical lens system with a common geometrical focus. Upon superposition, the phase‐matched counter‐propagating light‐sheets result in the generation of identical and equi spaced nano‐bump pattern. Study shows a feature size of as small as few tens of nanometers with a inter‐bump distance of few hundred nanometers. This technique overcomes some of the limitations of existing nano‐lithography techniques, thereby, may pave the way for mass‐production of nano‐structures. Potential applications can also be found in optical microscopy, plasmonics, and nano‐electronics. Microsc. Res. Tech. 78:1–7, 2015. © 2014 Wiley Periodicals, Inc.     

Out‐of‐focus background subtraction for fast structured illumination super‐resolution microscopy of optically thick samples

We propose a structured illumination microscopy method to combine super resolution and optical sectioning in three‐dimensional (3D) samples that allows the use of two‐dimensional (2D) data processing. Indeed, obtaining super‐resolution images of thick samples is a difficult task if low spatial frequencies are present in the in‐focus section of the sample, as these frequencies have to be distinguished from the out‐of‐focus background. A rigorous treatment would require a 3D reconstruction of the whole sample using a 3D point spread function and a 3D stack of structured illumination data. The number of raw images required, 15 per optical section in this case, limits the rate at which high‐resolution images can be obtained. We show that by a succession of two different treatments of structured illumination data we can estimate the contrast of the illumination pattern and remove the out‐of‐focus content from the raw images. After this cleaning step, we can obtain super‐resolution images of optical sections in thick samples using a two‐beam harmonic illumination pattern and a limited number of raw images. This two‐step processing makes it possible to obtain super resolved optical sections in thick samples as fast as if the sample was two‐dimensional.     

A stand‐alone compact EUV microscope based on gas‐puff target source

We report on a very compact desk‐top transmission extreme ultraviolet (EUV) microscope based on a laser‐plasma source with a double stream gas‐puff target, capable of acquiring magnified images of objects with a spatial (half‐pitch) resolution of sub‐50 nm. A multilayer ellipsoidal condenser is used to focus and spectrally narrow the radiation from the plasma, producing a quasi‐monochromatic EUV radiation (λ = 13.8 nm) illuminating the object, whereas a Fresnel zone plate objective forms the image. Design details, development, characterization and optimization of the EUV source and the microscope are described and discussed. Test object and other samples were imaged to demonstrate superior resolution compared to visible light microscopy.     

Harmonic demodulation and minimum enhancement factors in field‐enhanced near‐field optical microscopy

Field‐enhanced scanning optical microscopy relies on the design and fabrication of plasmonic probes which had to provide optical and chemical contrast at the nanoscale. In order to do so, the scattering containing the near‐field information recorded in a field‐enhanced scanning optical microscopy experiment, has to surpass the background light, always present due to multiple interferences between the macroscopic probe and sample. In this work, we show that when the probe–sample distance is modulated with very low amplitude, the higher the harmonic demodulation is, the better the ratio between the near‐field signal and the interferometric background results. The choice of working at a given n harmonic is dictated by the experiment when the signal at the n + 1 harmonic goes below the experimental noise. We demonstrate that the optical contrast comes from the nth derivative of the near‐field scattering, amplified by the interferometric background. By modelling the far and near field we calculate the probe–sample approach curves, which fit very well the experimental ones. After taking a great amount of experimental data for different probes and samples, we conclude with a table of the minimum enhancement factors needed to have optical contrast with field‐enhanced scanning optical microscopy.     

Contact X‐ray microscopy of living cells by using LiF crystal as imaging detector

In this paper, the use of lithium fluoride (LiF) as imaging radiation detector to analyse living cells by single‐shot soft X‐ray contact microscopy is presented. High resolved X‐ray images on LiF of cyanobacterium Leptolyngbya VRUC135, two unicellular microalgae of the genus Chlamydomonas and mouse macrophage cells (line RAW 264.7) have been obtained utilizing X‐ray radiation in the water window energy range from a laser plasma source. The used method is based on loading of the samples, the cell suspension, in a special holder where they are in close contact with a LiF crystal solid‐state X‐ray imaging detector. After exposure and sample removal, the images stored in LiF by the soft X‐ray contact microscopy technique are read by an optical microscope in fluorescence mode. The clear image of the mucilaginous sheath the structure of the filamentous Leptolyngbya and the visible nucleolus in the macrophage cells image, are noteworthiness results. The peculiarities of the used X‐ray radiation and of the LiF imaging detector allow obtaining images in absorption contrast revealing the internal structures of the investigated samples at high spatial resolution. Moreover, the wide dynamic range of the LiF imaging detector contributes to obtain high‐quality images. In particular, we demonstrate that this peculiar characteristic of LiF detector allows enhancing the contrast and reveal details even when they were obscured by a nonuniform stray light.     

Automated electron tomography with scanning transmission electron microscopy

We report the successful implementation of a fully automated tomographic data collection system in scanning transmission electron microscopy (STEM) mode. Autotracking is carried out by combining mechanical and electronic corrections for specimen movement. Autofocusing is based on contrast difference of a focus series of a small sample area. The focus gradient that exists in normal images due to specimen tilt is effectively removed by using dynamic focusing. An advantage of STEM tomography with dynamic focusing over TEM tomography is its ability to reconstruct large objects with a potentially higher resolution.     

Sequential processing of quantitative phase images for the study of cell behaviour in real‐time digital holographic microscopy

Transmitted light holographic microscopy is particularly used for quantitative phase imaging of transparent microscopic objects such as living cells. The study of the cell is based on extraction of the dynamic data on cell behaviour from the time‐lapse sequence of the phase images. However, the phase images are affected by the phase aberrations that make the analysis particularly difficult. This is because the phase deformation is prone to change during long‐term experiments. Here, we present a novel algorithm for sequential processing of living cells phase images in a time‐lapse sequence. The algorithm compensates for the deformation of a phase image using weighted least‐squares surface fitting. Moreover, it identifies and segments the individual cells in the phase image. All these procedures are performed automatically and applied immediately after obtaining every single phase image. This property of the algorithm is important for real‐time cell quantitative phase imaging and instantaneous control of the course of the experiment by playback of the recorded sequence up to actual time. Such operator's intervention is a forerunner of process automation derived from image analysis. The efficiency of the propounded algorithm is demonstrated on images of rat fibrosarcoma cells using an off‐axis holographic microscope.     

Accurate shape from focus based on focus adjustment in optical microscopy

Optical microscopy allows a magnified view of the sample while decreasing the depth of focus. Although the acquired images from limited depth of field have both blurred and focused regions, they can provide depth information. The technique to estimate the depth and 3D shape of an object from the images of the same sample obtained at different focus settings is called shape from focus (SFF). In SFF, the measure of focus–sharpness–is the crucial part for final 3D shape estimation. The conventional methods compute sharpness by applying focus measure operator on each 2D image frame of the image sequence. However, such methods do not reflect the accurate focus levels in an image because the focus levels for curved objects require information from neighboring pixels in the adjacent frames too. To address this issue, we propose a new method based on focus adjustment which takes the values of the neighboring pixels from the adjacent image frames that have approximately the same initial depth as of the center pixel and then it re-adjusts the center value accordingly. Experiments were conducted on synthetic and microscopic objects, and the results show that the proposed technique generates better shape and takes less computation time in comparison with previous SFF methods based on focused image surface (FIS) and dynamic programming. Microsc. Res. Tech., 2009. © 2008 Wiley-Liss, Inc.     

Registration of phase‐contrast images in propagation‐based X‐ray phase tomography

X‐ray phase tomography aims at reconstructing the 3D electron density distribution of an object. It offers enhanced sensitivity compared to attenuation‐based X‐ray absorption tomography. In propagation‐based methods, phase contrast is achieved by letting the beam propagate after interaction with the object. The phase shift is then retrieved at each projection angle, and subsequently used in tomographic reconstruction to obtain the refractive index decrement distribution, which is proportional to the electron density. Accurate phase retrieval is achieved by combining images at different propagation distances. For reconstructions of good quality, the phase‐contrast images recorded at different distances need to be accurately aligned. In this work, we characterise the artefacts related to misalignment of the phase‐contrast images, and investigate the use of different registration algorithms for aligning in‐line phase‐contrast images. The characterisation of artefacts is done by a simulation study and comparison with experimental data. Loss in resolution due to vibrations is found to be comparable to attenuation‐based computed tomography. Further, it is shown that registration of phase‐contrast images is nontrivial due to the difference in contrast between the different images, and the often periodical artefacts present in the phase‐contrast images if multilayer X‐ray optics are used. To address this, we compared two registration algorithms for aligning phase‐contrast images acquired by magnified X‐ray nanotomography: one based on cross‐correlation and one based on mutual information. We found that the mutual information‐based registration algorithm was more robust than a correlation‐based method.     

Rapid microscope auto‐focus method for uneven surfaces based on image fusion

Microscopic imaging of uneven surfaces is difficult because of the limited depth of field. In this study, we developed a rapid auto‐focus method for uneven surfaces based on image fusion. The Prewitt operator was used to detect the vertical edges of the images. Then, the focus position was theoretically calculated using a Gaussian function, and image fusion was applied to obtain the final in‐focus image. An experiment was designed to verify the developed method. The results revealed that this method is effective for printed circuit boards.     

Automated noninvasive epithelial cell counting in phase contrast microscopy images with automated parameter selection

Cell counting is commonly used to determine proliferation rates in cell cultures and for adherent cells it is often a ‘destructive’ process requiring disruption of the cell monolayer resulting in the inability to follow cell growth longitudinally. This process is time consuming and utilises significant resource. In this study a relatively inexpensive, rapid and widely applicable phase contrast microscopy‐based technique has been developed that emulates the contrast changes taking place when bright field microscope images of epithelial cell cultures are defocused. Processing of the resulting images produces an image that can be segmented using a global threshold; the number of cells is then deduced from the number of segmented regions and these cell counts can be used to generate growth curves. The parameters of this method were tuned using the discrete mereotopological relations between ground truth and processed images. Cell count accuracy was improved using linear discriminant analysis to identify spurious noise regions for removal. The proposed cell counting technique was validated by comparing the results with a manual count of cells in images, and subsequently applied to generate growth curves for oral keratinocyte cultures supplemented with a range of concentrations of foetal calf serum. The approach developed has broad applicability and utility for researchers with standard laboratory imaging equipment.     

Shape from focus based on 3D structure tensor using optical microscopy

Shape from focus (SFF) is a widely used passive optical method for 3D shape reconstruction. In SFF, a focus measure, which is used to estimate the relative focus level, plays a critical role in depth estimation. In this article, we present a new focus measure for accurate 3D shape estimation in optical microscopy based on the analysis of 3D structure tensor. First, the 3D tensors are computed from the input image sequence for each pixel. Then, each tensor is decomposed into point, curve, and surface tensors by decomposing tensors into eigenvalues and eigenvectors. Finally, the surfaceness is used to measure the quality of sharpness. The proposed focus measure provides accurate focus values and better resistance against noise. The proposed measure is evaluated by conducting experiments using image sequences of simulated and microscopic real objects. The comparative analysis demonstrates the effectiveness of the proposed focus measure in recovering 3D shape.     

An automatic vision‐based malaria diagnosis system

Malaria is a worldwide health problem with 225 million infections each year. A fast and easy‐to‐use method, with high performance is required to differentiate malaria from non‐malarial fevers. Manual examination of blood smears is currently the gold standard, but it is time‐consuming, labour‐intensive, requires skilled microscopists and the sensitivity of the method depends heavily on the skills of the microscopist. We propose an easy‐to‐use, quantitative cartridge‐scanner system for vision‐based malaria diagnosis, focusing on low malaria parasite densities. We have used special finger‐prick cartridges filled with acridine orange to obtain a thin blood film and a dedicated scanner to image the cartridge. Using supervised learning, we have built a Plasmodium falciparum detector. A two‐step approach was used to first segment potentially interesting areas, which are then analysed in more detail. The performance of the detector was validated using 5 420 manually annotated parasite images from malaria parasite culture in medium, as well as using 40 cartridges of 11 780 images containing healthy blood. From finger prick to result, the prototype cartridge‐scanner system gave a quantitative diagnosis in 16 min, of which only 1 min required manual interaction of basic operations. It does not require a wet lab or a skilled operator and provides parasite images for manual review and quality control. In healthy samples, the image analysis part of the system achieved an overall specificity of 99.999978% at the level of (infected) red blood cells, resulting in at most seven false positives per microlitre. Furthermore, the system showed a sensitivity of 75% at the cell level, enabling the detection of low parasite densities in a fast and easy‐to‐use manner. A field trial in Chittagong (Bangladesh) indicated that future work should primarily focus on improving the filling process of the cartridge and the focus control part of the scanner.     

MRT letter: Recovering weak‐textured surfaces using image focus

In nature, objects have partially weak texture and their shape reconstruction using focus based passive methods like shape from focus (SFF), is difficult. This article presents a new SFF algorithm which can compute precise depth of dense as well as weak textured objects. Segmentation is applied to discard wrong depth estimate and then later interpolating them from accurate depth values of their neighbors. The performance of the proposed method is tested, using different image sequences of synthetic and real objects, with varying textures. Microsc. Res. Tech., 2009. © 2009 Wiley‐Liss, Inc.