Shape from focus based on 3D structure tensor using optical microscopy

Shape from focus (SFF) is a widely used passive optical method for 3D shape reconstruction. In SFF, a focus measure, which is used to estimate the relative focus level, plays a critical role in depth estimation. In this article, we present a new focus measure for accurate 3D shape estimation in optical microscopy based on the analysis of 3D structure tensor. First, the 3D tensors are computed from the input image sequence for each pixel. Then, each tensor is decomposed into point, curve, and surface tensors by decomposing tensors into eigenvalues and eigenvectors. Finally, the surfaceness is used to measure the quality of sharpness. The proposed focus measure provides accurate focus values and better resistance against noise. The proposed measure is evaluated by conducting experiments using image sequences of simulated and microscopic real objects. The comparative analysis demonstrates the effectiveness of the proposed focus measure in recovering 3D shape.     

A novel method for shape from focus in microscopy using Bezier surface approximation

In this article, we introduce a novel shape from focus method to compute 3D shape of microscopic objects, based on modified‐pixel intensities and Bezier surface approximations. A new and simple but effective focus measure is proposed. In our focus measure, the original intensities of a sequence of small neighborhood are modified by subtracting the maximum of the values of first and last frames. An initial depth map is calculated by finding the maximum of the pixel's focused energy and its corresponding frame number. Missing information between two consecutive frames, false depth detection, and enhancement of noise related intensities may provide inaccurate depth map. To overcome these problems and to produce an accurate depth map, we proposed Bezier surface approximation. The proposed method is tested using synthetic and real image sequences. The comparative analysis demonstrates the effectiveness of the proposed method. Microsc. Res. Tech., 2010. © 2009 Wiley‐Liss, Inc.     

Searching surface orientation of microscopic objects for accurate 3D shape recovery

In this article, we propose a new shape from focus (SFF) method to estimate 3D shape of microscopic objects using surface orientation cue of each object patch. Most of the SFF algorithms compute the focus value of a pixel from the information of neighboring pixels lying on the same image frame based on an assumption that the small object patch corresponding to the small neighborhood of a pixel is a plane parallel to the focal plane. However, this assumption fails in the optics with limited depth of field where the neighboring pixels of an image have different degree of focus. To overcome this problem, we try to search the surface orientation of the small object patch corresponding to each pixel in the image sequence. Searching of the surface orientation is done indirectly by principal component analysis. Then, the focus value of each pixel is computed from the neighboring pixels lying on the surface perpendicular to the corresponding surface orientation. Experimental results on synthetic and real microscopic objects show that the proposed method produces more accurate 3D shape in comparison to the existing techniques.     

Application of the split-gradient method to 3D image deconvolution in fluorescence microscopy

The methods of image deconvolution are important for improving the quality of the detected images in the different modalities of fluorescence microscopy such as wide‐field, confocal, two‐photon excitation and 4Pi. Because deconvolution is an ill‐posed problem, it is, in general, reformulated in a statistical framework such as maximum likelihood or Bayes and reduced to the minimization of a suitable functional, more precisely, to a constrained minimization, because non‐negativity of the solution is an important requirement. Next, iterative methods are designed for approximating such a solution. In this paper, we consider the Bayesian approach based on the assumption that the noise is dominated by photon counting, so the likelihood is of the Poisson‐type, and that the prior is edge‐preserving, as derived from a simple Markov random field model. By considering the negative logarithm of the a posteriori probability distribution, the computation of the maximum a posteriori (MAP) estimate is reduced to the constrained minimization of a functional that is the sum of the Csiszár I‐divergence and a regularization term. For the solution of this problem, we propose an iterative algorithm derived from a general approach known as split‐gradient method (SGM) and based on a suitable decomposition of the gradient of the functional into a negative and positive part. The result is a simple modification of the standard Richardson–Lucy algorithm, very easily implementable and assuring automatically the non‐negativity of the iterates. Next, we apply this method to the particular case of confocal microscopy for investigating the effect of several edge‐preserving priors proposed in the literature using both synthetic and real confocal images. The quality of the restoration is estimated both by computation of the Kullback–Leibler divergence of the restored image from the detected one and by visual inspection. It is observed that the noise artefacts are considerably reduced and desired characteristics (edges and minute features as islets) are retained in the restored images. The algorithm is stable, robust and tolerant at various noise (Poisson) levels. Finally, by remarking that the proposed method is essentially a scaled gradient method, a possible modification of the algorithm is briefly discussed in view of obtaining fast convergence and reduction in computational time.     

Hyperspectral imaging of nanoparticles in biological samples: Simultaneous visualization and elemental identification

While engineered nanomaterials (ENMs) are increasingly incorporated into industrial processes and consumer products, the potential biological effects and health outcomes of exposure remain unknown. Novel advanced direct visualization techniques that require less time, cost, and resource investment than electron microscopy (EM) are needed for identifying and locating ENMs in biological samples. Hyperspectral imaging (HSI) combines spectrophotometry and imaging, using advanced optics and algorithms to capture a spectrum from 400 to 1000 nm at each pixel in an enhanced dark‐field microscopic (EDFM) image. HSI‐EDFM can be used to confirm the identity of the materials of interest in a sample and generate an image “mapping” their presence and location in a sample. Hyperspectral mapping is particularly important for biological samples, where ENM morphology is visually indistinct from surrounding tissue structures. While use of HSI (without mapping) is increasing, no studies to date have compared results from hyperspectral mapping with conventional methods. Thus, the objective of this study was to utilize EDFM‐HSI to locate, identify, and map metal oxide ENMs in ex vivo histological porcine skin tissues, a toxicological model of cutaneous exposure, and compare findings with those of Raman spectroscopy (RS), energy‐dispersive X‐ray spectroscopy (EDS), and scanning electron microscopy (SEM). Results demonstrate that EDFM‐HSI mapping is capable of locating and identifying ENMs in tissue, as confirmed by conventional methods. This study serves as initial confirmation of EDFM‐HSI mapping as a novel and higher throughput technique for ENM identification in biological samples, and serves as the basis for further protocol development utilizing EDFM‐HSI for semiquantitation of ENMs. Microsc. Res. Tech. 79:349–358, 2016. © 2016 Wiley Periodicals, Inc.     

Segmentation and tracking of live cells in phase-contrast images using directional gradient vector flow for snakes

Cell shape is an important characteristic of the physiological state of a cell and is used as a primary read-out of cell behaviour in various assays. Automated accurate segmentation of cells in microscopy images is hence of large practical importance in cell biology. We report a simple algorithm for automated cell segmentation in high-magnification phase-contrast images, which takes advantage of the characteristic directionality of the local image intensity gradient at cellular boundaries due to the 'halo-effect'. We employ a two-step algorithm in which a gradient vector flow (GVF) field is first used to direct active contours to an approximate cell boundary. A directional GVF (DGVF) field is then calculated by considering only edges for which the image intensity gradient is directed outwards with respect to the approximate cell contour. Subsequently, the DGVF field is used to refine the cell contour, by directing active contours to edges with the desired gradient directionality. This method allows us to accurately segment cells in an image series, as well as follow the dynamics of cell shape over time in an automated fashion.     

Influence of the phase effect on gradient‐based and statistics‐based focus measures in bright field microscopy

Autofocusing is essential to high throughput microscopy and live cell imaging and requires reliable focus measures. Phase objects such as separated single Chinese hamster ovary cells are almost invisible at the optical focus position in bright field microscopy images. Because of the phase effect, defocused images of phase objects have more contrast. In this paper, we show that widely used focus measures exhibit an untypical behaviour for such images. In the case of homogeneous cells, that is, when most cells tend to lie in the same focal plane, both gradient‐based and statistics‐based focus measures tend to have a local minimum instead of a global maximum at the optical focus position. On the other hand, if images show inhomogeneous cells, gradient‐based focus measures tend to yield typical focus curves, whereas statistics‐based focus measures deliver curves similar to the case of homogeneous cells. These results were interpreted using the equation describing the phase effect and patch‐wise analysis of the focus curves. Bioprocess engineering experts are also influenced by the phase effect. Forty‐four focus positions selected by them led to the conclusion that they prefer to look at defocused images instead of those at the optical focus.     

支撑点扩展快速立体匹配方法的设计与应用

为精确构建计算机立体视觉中的视差图,提出了一种快速全局优化匹配算法。该算法采用吉布斯随机场模型描述空间点与其邻域之间的关系,由改进的Graph Cuts方法对空间点的邻域进行匹配来获取场景的致密视差图。首先,计算出一组具有明确匹配关系的稀疏匹配点,将这些匹配点命名为“支撑点”;然后,对每一个支撑点的邻域进行扩展,采用改进的Graph Cuts全局优化算法计算扩展后的邻域空间的匹配关系,并将满足一定匹配度的邻域点设置为新的支撑点。最后,重复上述步骤并逐级扩展,直至扩展出的匹配空间覆盖整个视图,进而获取待匹配图对的致密视差图。实验结果表明,该方法不仅对不同场景视差图的质量具有良好的一致性,而且匹配速度较快(匹配时间约为0.8~1.2 s),大大高于其他传统的全局匹配算法。为体现本文算法的实际应用价值,以Smart Eye Ⅱ立体视觉试验台为测试平台,对真实场景进行了视差图构建,取得了良好的试验效果。     

Harmonic demodulation and minimum enhancement factors in field‐enhanced near‐field optical microscopy

Field‐enhanced scanning optical microscopy relies on the design and fabrication of plasmonic probes which had to provide optical and chemical contrast at the nanoscale. In order to do so, the scattering containing the near‐field information recorded in a field‐enhanced scanning optical microscopy experiment, has to surpass the background light, always present due to multiple interferences between the macroscopic probe and sample. In this work, we show that when the probe–sample distance is modulated with very low amplitude, the higher the harmonic demodulation is, the better the ratio between the near‐field signal and the interferometric background results. The choice of working at a given n harmonic is dictated by the experiment when the signal at the n + 1 harmonic goes below the experimental noise. We demonstrate that the optical contrast comes from the nth derivative of the near‐field scattering, amplified by the interferometric background. By modelling the far and near field we calculate the probe–sample approach curves, which fit very well the experimental ones. After taking a great amount of experimental data for different probes and samples, we conclude with a table of the minimum enhancement factors needed to have optical contrast with field‐enhanced scanning optical microscopy.     

Three-dimensional tracking and temporal analysis of liposomal transport in live cells using bright-field imaging

Gold nanoparticles (AuNPs) confined in liposomes of diameters around 200 nm produce strong scattering signal owing to surface plasmon resonance, and therefore bright-field optical tracking of the AuNP-encapsulating liposomes can be conducted in living cells. Using an optical profiling technique called noninterferometric wide-field optical profilometry and a bright-field tracking algorithm, the polynomial-fit Gaussian weight method, we analyze three-dimensional (3D) motion of such liposomes in living fibroblasts. The positioning accuracy in three dimensions is nearly 20 nm. We tag the liposome membranes with fibroblast growth factor-1 and reveal the intracellular transportation processes toward or away from the nucleus. On the basis of a temporal analysis of the intracellular 3D trajectories of AuNP-encapsulating liposomes, we identify directed and diffusive motions in the transportation processes.     

Analysis of changes in optical fibers during arc-fusion splicing by use of quantitative phase imaging

A non-interferometric imaging technique in conjunction with Abel inversion is used to directly and quantitatively examine the changes in optical fibers due to the heating produced during arc-fusion splicing as a function of fusion arc parameters. Phase images in the vicinity of a fusion splice are obtained using Quantitative Phase Microscopy, allowing the refractive-index change to be reconstructed with high spatial resolution. This simple, nondestructive method confirms that, for a fixed arc current, while the fusion time increases, the refractive-index of both fiber cores within the fusion region decreases in magnitude, the core region broadens, and the axial gradient decreases.     

Four-dimensional factor analysis of confocal image sequences (4D-FAMIS) to detect and characterize low copy numbers of human papillomavirus DNA by FISH in HeLa and SiHa cells

Visualization and localization of specific DNA sequences were performed by fluorescence in situ hybridization, confocal laser scanning microscopy (CLSM), and four-dimensional factor analysis of biomedical image sequences (4D-FAMIS). HeLa and SiHa cells containing, respectively 20–50 and 1–2 copies per cell of human papillomavirus (HPV) DNA type 18 and 16 integrated in cellular DNA were used as models. HPV-DNA was identified using DNA probes containing the whole genome of HPV-DNA type 18 or 16, and DNA–DNA hybrids were revealed by alkaline phosphatase and Fast Red. Cell nuclei were counterstained with thiazole orange (TO) or TOTO-iodide. 4D image sequences were obtained using successive dynamic or spectral sequences of images on different optical sections from CLSM. The location of fluorescent signals within the preparations was determined by FAMIS. This original method summarizes image sequences into a reduced number of images called factor images, and curves called factors. Factors estimate different individual physical behaviours in the sequence such as extinction velocity, spectral patterns and depth emission profiles. Factor images correspond to spatial distributions of the different factors. We distinguished between Fast Red and nucleus stainings in HPV-DNA hybridization signals by taking into account differences in their extinction velocities (fluorescence decay rate) or spectral patterns, and in their focus (depth emission profiles). In HeLa cells, factor images showed that Fast-Red-stained targets could be distinguished from nucleus stainings, and were located on different focal planes of the nuclei. In SiHa cells, 4D-FAMIS determined as few as 1–2 copies per cell of HPV-DNA type 16 located in continuous focal planes. Therefore, 4D-FAMIS, together with CLSM, made the detection and characterization of low copy numbers of genes in whole cells possible.     

Rapid in‐focus corrections on quantitative amplitude and phase imaging using transport of intensity equation method

Transport of intensity equation (TIE) method can acquire sample phase distributions with high speed and accuracy, offering another perspective for cellular observations and measurements. However, caused by incorrect focal plane determination, blurs and halos are induced, decreasing resolution and accuracy in both retrieved amplitude and phase information. In order to obtain high‐accurate sample details, we propose TIE based in‐focus correction technique for quantitative amplitude and phase imaging, which can locate focal plane and then retrieve both in‐focus intensity and phase distributions combining with numerical wavefront extraction and propagation as well as physical image recorder translation. Certified by both numerical simulations and practical measurements, it is believed the proposed method not only captures high‐accurate in‐focus sample information, but also provides a potential way for fast autofocusing in microscopic system.     

Improved visualization and quantitative analysis of fluorescent membrane sterol in polarized hepatic cells

Dehydroergosterol is a natural yeast sterol which has recently been employed for direct observation of intracellular sterol transport by UV microscopy. Here, methods are described for improved visualization and quantification of dehydroergosterol in the membranes of polarized HepG2 cells. Using a new online assay, it is shown that dehydroergosterol derived from a cyclodextrin complex inserted into the plasma membrane with a half time of t_1/2 ∼ 34 s. Based on a detailed bleaching analysis of dehydroergosterol, slightly different bleaching rates for dehydroergosterol in the basolateral and canalicular membrane were found, indicating different fluorophore environments. Bleaching correction in concert with 3D imaging allows for detection of dehydroergosterol enrichment in microvilli of the canalicular membrane forming the biliary canaliculus. Evidence is provided that some dehydroergosterol accumulating in a subapical compartment or apical recycling compartment can rapidly (t_1/2 ∼ 2 min) exchange in vesicles towards the biliary canaliculus while the majority of dehydroergosterol does not redistribute from this compartment. The rapidly exchanging pool resembles only a small portion of the total subapical compartment or apical recycling compartment-associated dehydroergosterol (about 15–30%). Kinetic modelling supports the theory that the subapical compartment or apical recycling compartment to biliary canaliculus transport pathway for sterol is unidirectional. This pathway might be important for rapid biliary transport of free sterol produced by hydrolysis of cholesteryl esters derived from high density lipoprotein.     

Measurements and analysis of the electromagnetic fields of mobile communication antennas

The owner of a mobile telephone is exposed to radiation from both the mobile telephone and base stations. They are often installed on the roofs of residential buildings. The article deals with electromagnetic fields generated by mobile communication antennas in the residential area. Measurements of electrical strength, magnetic strength and the electromagnetic field energy flux density were performed and compared to the established hygiene norms. Tests were conducted in the near and far zones of the antenna, in residential premises located directly in front of antenna, within the main radiation lobe of the antenna.     

Increased depth of field and stereo pairs of fluorescence micrographs via inverse filtering and maximum-likelihood estimation

Image processing methods are presented for effectively increasing the depth of field and for generating stereo pairs of fluorescence micrographs from a conventional optical microscope. In developing these methods the slice theorem of computed tomography is used. In this way the image reconstruction problem is reduced to one of processing only two-dimensional arrays rather than three-dimensional arrays and the classical difficult problem of restoring missing Fourier components within the missing cone region is circumvented. Two different approaches to such processing are presented. One approach is based on inverse filtering. Another approach is based on previous development of iterative image restoration algorithms for quantum-limited incoherent imagery, founded on maximum-likelihood estimation. Limited experimentation with real micrographs shows that both approaches work well. Some preliminary comparisons are made between the different variations of the methods tested, which point out the advantages and present limitations. Both methods can be implemented on IBM-AT-compatible computers with relatively fast execution times. Advantages that these methods have over confocal microscopy are (i) the optical and computing equipment required is less expensive, and (ii) a conventional microscope when set up properly can have much better light sensitivity than a confocal microscope.