Accurate shape from focus based on focus adjustment in optical microscopy

Optical microscopy allows a magnified view of the sample while decreasing the depth of focus. Although the acquired images from limited depth of field have both blurred and focused regions, they can provide depth information. The technique to estimate the depth and 3D shape of an object from the images of the same sample obtained at different focus settings is called shape from focus (SFF). In SFF, the measure of focus–sharpness–is the crucial part for final 3D shape estimation. The conventional methods compute sharpness by applying focus measure operator on each 2D image frame of the image sequence. However, such methods do not reflect the accurate focus levels in an image because the focus levels for curved objects require information from neighboring pixels in the adjacent frames too. To address this issue, we propose a new method based on focus adjustment which takes the values of the neighboring pixels from the adjacent image frames that have approximately the same initial depth as of the center pixel and then it re-adjusts the center value accordingly. Experiments were conducted on synthetic and microscopic objects, and the results show that the proposed technique generates better shape and takes less computation time in comparison with previous SFF methods based on focused image surface (FIS) and dynamic programming. Microsc. Res. Tech., 2009. © 2008 Wiley-Liss, Inc.     

MRT Letter: Guided filtering of image focus volume for 3D shape recovery of microscopic objects

In this letter, a shape from focus (SFF) method is proposed that utilizes the guided image filtering to enhance the image focus volume efficiently. First, image focus volume is computed using a conventional focus measure. Then each layer of image focus volume is filtered using guided filtering. In this work, the all‐in‐focus image, which can be obtained from the initial focus volume, is used as guidance image. Finally, improved depth map is obtained from the filtered image focus volume by maximizing the focus measure along the optical axis. The proposed SFF method is efficient and provides better depth maps. The improved performance is highlighted by conducting several experiments using image sequences of simulated and real microscopic objects. The comparative analysis demonstrates the effectiveness of the proposed SFF method. Microsc. Res. Tech. 77:959–963, 2014. © 2014 Wiley Periodicals, Inc.     

Multidirectional focus measure for accurate three-dimensional shape recovery of microscopic objects

Shape from focus (SFF) is a technique to recover the shape of an object from multiple images taken at various focus settings. Most of conventional SFF techniques compute focus value of a pixel by applying one of focus measure operators on neighboring pixels on the same image frame. However, in the optics with limited depth of field, neighboring pixels of an image have different degree of focus for curved objects, thus the computed focus value does not reflect the accurate focus level of the pixel. Ideally, an accurate focus value of a pixel needs to be measured from the neighboring pixels lying on tangential plane of the pixel in image space. In this article, a tangential plane on each pixel location (i, j) in image sensor is searched by selecting one of five candidate planes based on the assumption that the maximum variance of focus values along the optical axis is achieved from the neighborhood lying on tangential plane of the pixel (i, j). Then, a focus measure operator is applied on neighboring pixels lying on the searched plane. The experimental results on both the synthetic and real microscopic objects show the proposed method produces more accurate three-dimensional shape in comparison to conventional SFF method that applies focus measures on original image planes.     

Searching surface orientation of microscopic objects for accurate 3D shape recovery

In this article, we propose a new shape from focus (SFF) method to estimate 3D shape of microscopic objects using surface orientation cue of each object patch. Most of the SFF algorithms compute the focus value of a pixel from the information of neighboring pixels lying on the same image frame based on an assumption that the small object patch corresponding to the small neighborhood of a pixel is a plane parallel to the focal plane. However, this assumption fails in the optics with limited depth of field where the neighboring pixels of an image have different degree of focus. To overcome this problem, we try to search the surface orientation of the small object patch corresponding to each pixel in the image sequence. Searching of the surface orientation is done indirectly by principal component analysis. Then, the focus value of each pixel is computed from the neighboring pixels lying on the surface perpendicular to the corresponding surface orientation. Experimental results on synthetic and real microscopic objects show that the proposed method produces more accurate 3D shape in comparison to the existing techniques.     

PCA-based method for 3D shape recovery of microscopic objects from image focus using discrete cosine transform

This article introduces a new algorithm for shape from focus (SFF) based on discrete cosine transform (DCT) and principal component analysis (PCA). DCT is applied on a small 3D neighborhood for each pixel in the image volume. Instead of summing all focus values in a window, AC parts of DCT are collected and then PCA is applied to transform this data into eigenspace. The first feature, containing maximum variation is employed to compute the depth. DCT and PCA are computationally intensive; however, the reduced data elements and algorithm iterations have made the new approach competitive and efficient. The performance of the proposed approach is compared with other methods by conducting experiments using image sequences of a synthetic and two microscopic objects. The evaluation is gauged on the basis of unimodality, monotonicity, and resolution of the focus curve. Two other global statistical metrics, root mean square error (RMSE) and correlation have also been applied for synthetic image sequence. Besides, noise sensitivity and computational complexity are also compared with other algorithms. Experimental results demonstrate the effectiveness and the robustness of the new method.     

MRT letter: A total variation based method for 3D shape recovery of microscopic objects through image focus

Generally, shape from focus methods use a single focus measure to compute focus quality and to obtain an initial depth map of an object. However, different focus measures perform differently in diverse conditions. Therefore, it is hard to get accurate 3D shape based on a single focus measure. In this article, we propose a total variation based method for recovering 3D shape of an object by combining multiple depth hypothesis obtained through different focus measures. Improved performance of the proposed method is evaluated by conducting several experiments using images of synthetic and real microscopic objects. Comparative analysis demonstrates the effectiveness of the proposed approach. Microsc. Res. Tech. 76:877–881, 2013. © 2013 Wiley Periodicals, Inc.     

A novel method for shape from focus in microscopy using Bezier surface approximation

In this article, we introduce a novel shape from focus method to compute 3D shape of microscopic objects, based on modified‐pixel intensities and Bezier surface approximations. A new and simple but effective focus measure is proposed. In our focus measure, the original intensities of a sequence of small neighborhood are modified by subtracting the maximum of the values of first and last frames. An initial depth map is calculated by finding the maximum of the pixel's focused energy and its corresponding frame number. Missing information between two consecutive frames, false depth detection, and enhancement of noise related intensities may provide inaccurate depth map. To overcome these problems and to produce an accurate depth map, we proposed Bezier surface approximation. The proposed method is tested using synthetic and real image sequences. The comparative analysis demonstrates the effectiveness of the proposed method. Microsc. Res. Tech., 2010. © 2009 Wiley‐Liss, Inc.     

MRT letter: Recovering weak‐textured surfaces using image focus

In nature, objects have partially weak texture and their shape reconstruction using focus based passive methods like shape from focus (SFF), is difficult. This article presents a new SFF algorithm which can compute precise depth of dense as well as weak textured objects. Segmentation is applied to discard wrong depth estimate and then later interpolating them from accurate depth values of their neighbors. The performance of the proposed method is tested, using different image sequences of synthetic and real objects, with varying textures. Microsc. Res. Tech., 2009. © 2009 Wiley‐Liss, Inc.     

Optimal depth estimation using modified Kalman filter in the presence of non‐Gaussian jitter noise

The consideration of the noise that affects 3D shape recovery is becoming very important for accurate shape reconstruction. In Shape from Focus, when 2D image sequences are obtained, mechanical vibrations, referred as jitter noise, occur randomly along the z‐axis, in each step. To model the noise for real world scenarios, this article uses Lévy distribution for noise profile modeling. Next, focus curves acquired by one of focus measure operators are modeled as Gaussian function to consider the effects of the jitter noise. Finally, since conventional Kalman filter provides good output under Gaussian noise only, a modified Kalman filter, as proposed method, is used to remove the jitter noise. Experiments are carried out using synthetic and real objects to show the effectiveness of the proposed method.     

MRT Letter: Optimal composite depth function for 3D shape recovery of microscopic objects

Generally, in shape from focus techniques, a single focus measure is used in estimating the three‐dimensional structure of microscopic objects. However, the performance of a single focus measure is limited to estimate accurately the depth map of diverse type of objects. To cope with this problem, we propose genetic programming based novel approach by developing an optimal composite depth (OCD) function for accurate depth estimation. This OCD function optimally combines the initial depth and focus information extracted from individual focus measures. An improved performance of this function is reported for synthetic and real world microscopic objects. Microsc. Res. Tech., 2010. © 2010 Wiley‐Liss, Inc.     

基于多图像融合的MEMS显微三维形貌重构

针对微型机电系统(MEMS)的三维测量,显微镜或光学轮廓干涉仪等传统方法存在显微测量精度低、设备成本高等问题,且当结构含有较多断裂面时,解包裹算法效果欠佳。本文提出一种基于多图像融合的MEMS显微三维测量方法。不同于多角度显微三维测量方法,本研究首先利用单目显微镜,通过单一轴向移动获取一系列测量目标深度信息的单一角度图像,并利用去雾算法对图像进行预处理,实现了去噪和有效信息提取的目的;然后通过聚焦测度算法获取待测对象的深度信息;最后利用数据处理软件进行三维拟合。基于上述原理,本文以焦平面阵列(FPA)作为待测目标进行了测量实验。本文提出的三维测量方法和图像处理算法可获得更准确的FPA形貌,可清晰显示反射面与支腿部分及反射面上的释放孔,测得FPA的支腿长度为110.6μm,每个反射面的像元尺寸为120.8μm×70.8μm,与设计值基本吻合,解决了断裂面难以测量的问题,同时降低了微结构测量的难度和成本。单目显微镜单向移动的多图像融合测量技术对MEMS的三维形貌测量具有重要意义,去雾算法在图像融合与三维测量的图像处理也有很好的应用价值。     

Depth from image focus methods for micro-manufacturing

The analysis of industrially important electronic components of micro dimensions such as thin film transistor liquid crystal display color filter is of fundamental importance in consumer electronics. Three dimensional (3D) visualization, size, area, and surface roughness are important parameters for the analysis of micro components. For this purpose, the devices equipped with active but expensive depth estimation techniques are commonly used. However, these can be replaced by inexpensive passive methods. In this paper, we propose an inexpensive and simple system for the analysis of micro components. It comprises a CCD camera mounted on a conventional microscope and a passive optical method for 3D shape recovery. To improve the accuracy of the system, we introduce an accurate depth estimation scheme by using cubic degree Bezier–Bernstein polynomial. The proposed system is tested by using image sequences of synthetic and real objects. The experimental results demonstrate the usefulness and effectiveness of the proposed system for the analysis of micro size electronic components.     

基于小波变换的显微图像清晰度评价函数及3-D自动调焦技术

提出了基于小波变换的图像清晰度评价函数。采用大NA(数值孔径)和小NA的显微图像序列,比较分析了本文提出的评价函数和经典的归一化方差函数、熵函数、能量拉普拉斯函数以及另外两种基于小波变换评价函数的清晰度评价性能。同时采用带有标准偏差为25的高斯噪声显微图像序列,比较了这五种评价函数的抗噪能力。实验结果表明:提出的评价函数具有最高的聚焦精度和聚焦分辨率,且具有与抗噪能力最强的归一化方差函数相当的抗噪能力。提出了基于区域选择的自动聚焦方法,实现了处于不同深度的微操作对象的3-D自动聚焦。该评价函数和区域选择聚焦技术可以用于高精度的自动微操作作业中。进一步说明自动调焦是实现自动化微操作的关键技术,而其核心是清晰度评价函数的选取或构建。     

Developing a 3D colour image reproduction system for additive manufacturing of facial prostheses

In this study, a new 3D colour image reproduction system is proposed for the automated and accurate additive manufacturing of soft tissue facial prostheses. A framework of 3D colour image reproduction was defined and a protocol for each sub-process was developed for this specific application. Colour management processes were developed and integrated into the proposed 3D image reproduction system; colour profiles for both the 3dMD photogrammetry system and the Z Corp Z510 3D printer were established utilising conventional colour reproduction techniques for 2D images. The soft tissue prototypes of both nose and ear prostheses were produced using the proposed system. The quality of prostheses was evaluated. The results show that the protocol used in the 3D manufacturing process was capable of producing accurate skin colour with fine textures and 3D shape, with significant savings in both time and cost.     

Construction and evaluation of a wavelet-based focus measure for microscopy imaging

Microscopy imaging can not achieve both high resolution and wide image space simultaneously. Autofocusing is of fundamental importance to automated micromanipulation. This article proposes a new wavelet-based focus measure, which is defined as a ratio of high frequency coefficients and low frequency coefficients. 8 series of 49 microscope images each acquired under five magnifications are used to comprehensively compare the performance of our focus measure with the classic and popular focus measures, including Normalized Variance, Entropy, Energy Laplace and wavelet-based high frequency focus measures. The robustness of these focus measures is evaluated using noisy image sequences corrupted by Gaussian white noise with standard deviations (STD) 5 and 15. An evaluation methodology is proposed, based on which these 5 focus measures are ranked. Experimental results show that the proposed focus measure can provide significantly the best overall performance and robustness. This focus measure can be widely applied to the automated biological and biomedical applications.     

金字塔光流三维运动估计与深度重建直接方法

针对基于图像序列光流的三维运动估计与深度重建问题,提出一种基于图像金字塔光流的三维运动估计与深度重建直接方法。首先根据光流计算亮度守恒假设和像素点光流与三维空间点运动的对应关系推导出基于图像亮度的三维运动守恒假设;然后借鉴变分原理,通过设计基于L1模型的鲁棒数据项以及图像与运动联合控制的平滑项构造基于变分光流的三维运动估计能量函数;为了应对图像序列中包含的大位移运动及运动遮挡问题,采用图像金字塔分层策略设计三维运动估计模型;最后根据图像三维运动估计结果重建图像中运动物体或场景的深度信息。实验表明,该方法能够较好地应对图像序列中光照变化、多目标大位移运动以及运动遮挡等情况,具有较高的三维运动估计与深度重建精度和较好的鲁棒性。     

一种变物距的自动对焦方法

针对像距无法改变的特定环境,提出了一种变物距的自动对焦方法。首先通过设置标志矩形,采用SUSAN算法中的角点获取算法,取得对焦窗口;其次建立对焦窗口转换函数,使不同物距下的对焦窗口包括相同的成像信息;最后使用能量梯度对焦评价函数,取得最佳对焦位置。实验表明,该系统适用于像距无法改变的环境,并且具有良好的自动对焦精度、速度以及鲁棒性。     

3D ECT reconstruction by an improved Landweber iteration algorithm

Electrical capacitance tomography (ECT) is a relatively mature non-invasive imaging technique that attempts to map dielectric permittivity of materials. Recently, 3D ECT has gained interest because of its potential to generate volumetric images. The study of a fast and accurate image reconstruction algorithm is a challenge task, especially for 3D reconstruction. In this paper, we propose an improved Landweber iteration algorithm. We incorporate an additional acceleration term into the cost function and apply an adaptive threshold operation to the image obtained in each iteration for reducing artefacts. The algorithm proposed is tested by the noise-free and noise-contaminated capacitance data. Sensitivity matrixes and capacitance data of a 3D ECT sensor are obtained by using the finite element (FE) method. Extensive simulations in 3D reconstruction are carried out. The results verify the effectiveness of these improvements. Both the reconstruction time and the artefacts in the reconstructed image are reduced obviously. The experimental results of 3D reconstruction of objects in the shape of letters U and L confirm the effectiveness of the proposed algorithm further.