Multidirectional focus measure for accurate three-dimensional shape recovery of microscopic objects

Shape from focus (SFF) is a technique to recover the shape of an object from multiple images taken at various focus settings. Most of conventional SFF techniques compute focus value of a pixel by applying one of focus measure operators on neighboring pixels on the same image frame. However, in the optics with limited depth of field, neighboring pixels of an image have different degree of focus for curved objects, thus the computed focus value does not reflect the accurate focus level of the pixel. Ideally, an accurate focus value of a pixel needs to be measured from the neighboring pixels lying on tangential plane of the pixel in image space. In this article, a tangential plane on each pixel location (i, j) in image sensor is searched by selecting one of five candidate planes based on the assumption that the maximum variance of focus values along the optical axis is achieved from the neighborhood lying on tangential plane of the pixel (i, j). Then, a focus measure operator is applied on neighboring pixels lying on the searched plane. The experimental results on both the synthetic and real microscopic objects show the proposed method produces more accurate three-dimensional shape in comparison to conventional SFF method that applies focus measures on original image planes.     

Accurate shape from focus based on focus adjustment in optical microscopy

Optical microscopy allows a magnified view of the sample while decreasing the depth of focus. Although the acquired images from limited depth of field have both blurred and focused regions, they can provide depth information. The technique to estimate the depth and 3D shape of an object from the images of the same sample obtained at different focus settings is called shape from focus (SFF). In SFF, the measure of focus–sharpness–is the crucial part for final 3D shape estimation. The conventional methods compute sharpness by applying focus measure operator on each 2D image frame of the image sequence. However, such methods do not reflect the accurate focus levels in an image because the focus levels for curved objects require information from neighboring pixels in the adjacent frames too. To address this issue, we propose a new method based on focus adjustment which takes the values of the neighboring pixels from the adjacent image frames that have approximately the same initial depth as of the center pixel and then it re-adjusts the center value accordingly. Experiments were conducted on synthetic and microscopic objects, and the results show that the proposed technique generates better shape and takes less computation time in comparison with previous SFF methods based on focused image surface (FIS) and dynamic programming. Microsc. Res. Tech., 2009. © 2008 Wiley-Liss, Inc.     

Shape from focus based on 3D structure tensor using optical microscopy

Shape from focus (SFF) is a widely used passive optical method for 3D shape reconstruction. In SFF, a focus measure, which is used to estimate the relative focus level, plays a critical role in depth estimation. In this article, we present a new focus measure for accurate 3D shape estimation in optical microscopy based on the analysis of 3D structure tensor. First, the 3D tensors are computed from the input image sequence for each pixel. Then, each tensor is decomposed into point, curve, and surface tensors by decomposing tensors into eigenvalues and eigenvectors. Finally, the surfaceness is used to measure the quality of sharpness. The proposed focus measure provides accurate focus values and better resistance against noise. The proposed measure is evaluated by conducting experiments using image sequences of simulated and microscopic real objects. The comparative analysis demonstrates the effectiveness of the proposed focus measure in recovering 3D shape.     

PCA-based method for 3D shape recovery of microscopic objects from image focus using discrete cosine transform

This article introduces a new algorithm for shape from focus (SFF) based on discrete cosine transform (DCT) and principal component analysis (PCA). DCT is applied on a small 3D neighborhood for each pixel in the image volume. Instead of summing all focus values in a window, AC parts of DCT are collected and then PCA is applied to transform this data into eigenspace. The first feature, containing maximum variation is employed to compute the depth. DCT and PCA are computationally intensive; however, the reduced data elements and algorithm iterations have made the new approach competitive and efficient. The performance of the proposed approach is compared with other methods by conducting experiments using image sequences of a synthetic and two microscopic objects. The evaluation is gauged on the basis of unimodality, monotonicity, and resolution of the focus curve. Two other global statistical metrics, root mean square error (RMSE) and correlation have also been applied for synthetic image sequence. Besides, noise sensitivity and computational complexity are also compared with other algorithms. Experimental results demonstrate the effectiveness and the robustness of the new method.     

MRT Letter: Guided filtering of image focus volume for 3D shape recovery of microscopic objects

In this letter, a shape from focus (SFF) method is proposed that utilizes the guided image filtering to enhance the image focus volume efficiently. First, image focus volume is computed using a conventional focus measure. Then each layer of image focus volume is filtered using guided filtering. In this work, the all‐in‐focus image, which can be obtained from the initial focus volume, is used as guidance image. Finally, improved depth map is obtained from the filtered image focus volume by maximizing the focus measure along the optical axis. The proposed SFF method is efficient and provides better depth maps. The improved performance is highlighted by conducting several experiments using image sequences of simulated and real microscopic objects. The comparative analysis demonstrates the effectiveness of the proposed SFF method. Microsc. Res. Tech. 77:959–963, 2014. © 2014 Wiley Periodicals, Inc.     

A novel method for shape from focus in microscopy using Bezier surface approximation

In this article, we introduce a novel shape from focus method to compute 3D shape of microscopic objects, based on modified‐pixel intensities and Bezier surface approximations. A new and simple but effective focus measure is proposed. In our focus measure, the original intensities of a sequence of small neighborhood are modified by subtracting the maximum of the values of first and last frames. An initial depth map is calculated by finding the maximum of the pixel's focused energy and its corresponding frame number. Missing information between two consecutive frames, false depth detection, and enhancement of noise related intensities may provide inaccurate depth map. To overcome these problems and to produce an accurate depth map, we proposed Bezier surface approximation. The proposed method is tested using synthetic and real image sequences. The comparative analysis demonstrates the effectiveness of the proposed method. Microsc. Res. Tech., 2010. © 2009 Wiley‐Liss, Inc.     

MRT letter: Recovering weak‐textured surfaces using image focus

In nature, objects have partially weak texture and their shape reconstruction using focus based passive methods like shape from focus (SFF), is difficult. This article presents a new SFF algorithm which can compute precise depth of dense as well as weak textured objects. Segmentation is applied to discard wrong depth estimate and then later interpolating them from accurate depth values of their neighbors. The performance of the proposed method is tested, using different image sequences of synthetic and real objects, with varying textures. Microsc. Res. Tech., 2009. © 2009 Wiley‐Liss, Inc.     

MRT letter: A total variation based method for 3D shape recovery of microscopic objects through image focus

Generally, shape from focus methods use a single focus measure to compute focus quality and to obtain an initial depth map of an object. However, different focus measures perform differently in diverse conditions. Therefore, it is hard to get accurate 3D shape based on a single focus measure. In this article, we propose a total variation based method for recovering 3D shape of an object by combining multiple depth hypothesis obtained through different focus measures. Improved performance of the proposed method is evaluated by conducting several experiments using images of synthetic and real microscopic objects. Comparative analysis demonstrates the effectiveness of the proposed approach. Microsc. Res. Tech. 76:877–881, 2013. © 2013 Wiley Periodicals, Inc.     

Contour extraction of a laser stripe located on a microscope image from a stereo light microscope

A stereo light microscope has a special optical structure that consists of two optical paths. With two cameras fitted on its imaging planes, a microscopic binocular vision system can be constructed, and this system can be applied in three‐dimensional (3D) shape measurements of a microscopic object. A novel‐shape reconstruction system is developed in this article composed of laser fringe scanning and a stereo light microscope. A laser projector emits a thin light sheet and projects it onto the microscopic object's surface. The microscopic object is placed on a micro‐displacement mechanism and moved in a given direction. The entire surface of the object is scanned by the light sheet. This system captures a series of microscopic images of the laser stripe, which are used to restore the 3D shape of the object. In this article, we mainly focus on the study of the laser stripe detection method, which is derived based on the Canny rule. First, the Canny rule outputs pixels at the left and right edges of the laser stripe. Then, a subpixel edge extraction method is proposed based on polynomial fitting that outputs the center curve of the laser stripe. Finally, an edge filter is used to smooth the edge burrs, and the Hermite interpolation method is used to link the broken edges and construct continuous, smoothing edge contours. The results show that this method can effectively find the subpixel position of the laser stripe and output high‐quality edge contours. The method is suitable for extracting the contours of a laser stripe located in a microscope image.     

金相显微镜上实现三维微观形貌恢复

金相显微镜是一种经典的测量分析仪器,已经实现了金相显微镜图像的数字化采集,并恢复了三维微观形貌。首先,对金相显微镜的载物台加装电机,利用电机带动载物台进行精密移动,在载物台移动过程中,对被测物体等步距拍摄序列图像。然后,根据聚焦形貌恢复的原理,采用修正的拉普拉斯算子进行聚焦程度判断,利用高斯插值方法获取聚焦程度最大值以及此时被测表面的高度信息,恢复三维形貌。最后,利用金相显微镜对粗糙度样块进行测量,获得了相符的测量结果。该方法对于扩展金相显微镜的测量范围具有积极意义。     

Analysis of 3D shape and strain distributions of a deformable object using stereo vision

This paper proposes a method for analyzing three-dimensional shape and strain distributions of a deformable object using stereo vision. The three-dimensional coordinates of the surface points of an object have been calculated using stereoscopic method. The Fourier transform grid method is applied to high-precision 3D shape and strain distribution of deformable objects by analyzing the phase distribution of grating images. In the conventional automated grid method, because the position of a grid point is expressed by an integer number of pixels, it is difficult to obtain accurate measurement. Using two-dimensional Fourier transform grid method and a two-dimensional cross grating on the surface of the object, we can easily and accrately find the correspondence using phase information and seperate each directional grating line from the two-dimensional grating. Applications for analyzing shape and strain distributions of deformable objects are shown.     

Sampling effects influence heights measured with atomic force microscopy

The atomic force microscope (AFM) is an exquisitely delicate probe measuring the height of a specimen at discrete sampling points in a fixed two‐dimensional (2D) raster. The resulting topograph is a 2D digital image, with each pixel representing a distinct height measurement. The height of an object is determined as the average of the maximum heights measured above the supporting surface. We show that such object heights derived from a variety of organic samples depend critically on the sampling or pixel size of the 2D raster. It is concluded that to obtain accurate specimen heights, the pixel size must be small enough to resolve submolecular structures and thus ensure representative sampling of the height variation on the surface.     

Visibility of objects in computer simulations of noisy micrographs

Thresholds of visibility for objects in images with random pixel noise are predicted in terms of the signal-to-noise ratio. From trials with volunteers marking test images, we determined visibility thresholds of objects obscured by random pixel noise. The test images had objects with a variety of simple shapes and relatively little internal structure. Aside from the noise, the background of the test images was smooth and featureless. We extend the threshold signal-to-noise ratio measurements of Rose and others to a variety of object sizes and shapes. For objects with areas less than a disc subtending 2° at the eye, visibility depends on the averaged difference in intensity from background, the noise level and the number of pixels in the object. Visibility does not seem to depend on object shape.     

The tandem scanning reflected light microscope

Reflected light microscopy of biological material has been a very difficult task and many different but hardly successful attempts have been made to get usable images. The main reasons for this state are: Weak reflections from the biologically important structures in the object as against strong reflection at its surface; reflections at optical surfaces in the microscope which cause a deterioration of contrast; and the mixing together of reflections from many levels in the object so that the signal coming from the focussed-on level is lost in noise and d.c. components. We tried, and successfully, to reverse this situation and to make it possible for the signal to overwhelm spurious and scattered light using double, or, as we call it, tandem scanning. The object is illuminated only in small patches lying in one plane and moving across this plane, and only the light reflected from these illuminated patches is allowed to pass into the image plane and participate in image formation. The image consists of points which travel over the image plane and which are geometrical images of the illuminated patches in the focussed-on object plane. To ensure that only the light belonging to these geometrical images be allowed to enter the image, the image plane is covered with an opaque diaphragm having holes in locations exactly corresponding to the locations of the geometrical images of the illuminated object points; this diaphragm travels in concordance with the first scanning and so the complete image of the focussed-on object plane is formed successively. Both scans are performed by a single device, a Nipkow disc carrying in its annular periphery several ten thousands of holes arranged in Archimedean spirals. The disc is 100 mm in diameter and rotates about 100 rpm driven by an electric motor. On one side the disc is illuminated in a circle 18 mm in diameter, and the light transmitted through several hundred holes and reflected in a mirror system passes a microscope objective which forms the images of the disc holes in the object plane. The light reflected here passes through the same objective and mirror system (one mirror being a “limitlessly thin” beam splitter) to pass conjugate aperture holes on the observation side of the Nipkow disc. As the disc lies at the intermediate image plane of the objective lens on both its illuminating and observation sides, only light emanating from reflection or fluorescence in the plane of focus can contribute to the image. High contrast images of very thin focussed-on layers are thus formed. The practical arrangements are such that very large specimens can be examined: The specimens for this microscope need not themselves be microscopic.     

Application of convolutional neural network to acquisition of clear images for objects with large vertical size in stereo light microscope vision system

For an object with large vertical size that exceeds the certain depth of a stereo light microscope (SLM), its image will be blurred. To obtain clear images, we proposed an image fusion method based on the convolutional neural network (CNN) for the microscopic image sequence. The CNN was designed to discriminate clear and blurred pixels in the source images according to the neighborhood information. To train the CNN, a training set that contained correctly labeled clear and blurred images was created from an open‐access database. The image sequence to be fused was aligned at first. The trained CNN was then used to measure the activity level of each pixel in the aligned source images. The fused image was obtained by taking the pixels with the highest activity levels in the source image sequence. The performance was evaluated using five microscopic image sequences. Compared with other two fusion methods, the proposed method obtained better performance in terms of both visual quality and objective assessment. It is suitable for fusion of the SLM image sequence.     

基于支持向量回归的小尺寸零件精密测量

为提高小尺寸零件测量精度和速度,提出了基于支持向量回归(SVR)的小尺寸零件精密测量方法。系统以齿形链板为研究对象,对其主要参数进行测量。系统采用透射照明方式,使用A102FCCD数字摄像头采集齿形链板的图像,经过IEEE1394数字接口卡传输到计算机。对含有噪声的原始数字图像实施中值滤波降噪、二值化,轮廓提取及图像旋转等处理,使图像转变成易于检测的单像素宽边缘信息。然后根据齿形链板长度与宽度比例确定待检测区域,以待检测区域内的边缘轮廓上的各像素点构成对应线段的训练集,进行支持向量回归,获得具有亚像素表示的各检测线段的回归函数,并据此对齿形链板的主要参数进行测量。最后,对测量误差进行了分析。测量结果满足零件的公差要求,测量精度可达2 μm。理论分析及实验结果表明,该方法测量速度快,测量精度高,同时对图像平面内旋转、尺度变化、噪声等具有较强的鲁棒性。     

局部正交子空间投影高光谱图像异常检测算法

正交子空间投影是一种监督分类算法,需要已知待分类目标的特征信息。为了扩展该算法的应用场合,提出局部正交子空间投影算法,并成功应用于高光谱图像异常检测。异常检测常用于自然背景下的人工目标提取,在小范围邻域内地物类型相对单一。基于此,以被检测点作为感兴趣目标、被检测点邻域内样本均值作为不感兴趣目标,构造局部投影算子。实验结果表明：能够检测出含量高于30％的亚像元目标;可通过适当增大滑动窗尺寸,检测像元面积较大的目标;不受Hughes效应影响;当波段数为80时,运算时间低于RX算法的十分之一。局部正交子空间投影算法检测精度高、运算速度快,适用于实时高光谱图像异常检测。     

3-D image formation in high-aperture fluorescence confocal microscopy: a numerical analysis

The imaging properties of a confocal fluorescence microscope are considered on the basis of a theoretical model. The model takes into account high-aperture objectives, the polarization state of the excitation light and a finite detector pinhole. Electromagnetic diffraction theory of the field near focus as developed by Richards and Wolf is used to compute the optical properties of the model. These are shown to be dependent on the polarization of the light. With the resulting three-dimensional point spread function we have studied the imaging of point, line and plane objects as a function of their orientation with respect to the confocal plane. In addition, the effect of the pinhole size on the image formation of these objects is discussed. The results indicate the necessity to take object orientation into account during image processing activities such as segmentation or analysis.     

Optimal depth estimation using modified Kalman filter in the presence of non‐Gaussian jitter noise

The consideration of the noise that affects 3D shape recovery is becoming very important for accurate shape reconstruction. In Shape from Focus, when 2D image sequences are obtained, mechanical vibrations, referred as jitter noise, occur randomly along the z‐axis, in each step. To model the noise for real world scenarios, this article uses Lévy distribution for noise profile modeling. Next, focus curves acquired by one of focus measure operators are modeled as Gaussian function to consider the effects of the jitter noise. Finally, since conventional Kalman filter provides good output under Gaussian noise only, a modified Kalman filter, as proposed method, is used to remove the jitter noise. Experiments are carried out using synthetic and real objects to show the effectiveness of the proposed method.