Doppler encoded excitation pattern tomographic optical microscopy

Most far-field optical imaging systems rely on lenses and spatially resolved detection to probe distinct locations on the object. We describe and demonstrate a high-speed wide-field approach to imaging that instead measures the complex spatial Fourier transform of the object by detecting its spatially integrated response to dynamic acousto-optically synthesized structured illumination. Tomographic filtered backprojection is applied to reconstruct the object in two or three dimensions. This technique decouples depth of field and working distance from resolution, in contrast to conventional imaging, and can be used to image biological and synthetic structures in fluoresced or scattered light employing coherent or broadband illumination. We discuss the electronically programmable transfer function of the optical system and its implications for imaging dynamic processes. We also explore wide-field fluorescence imaging in scattering media by coherence gating. Finally, we present two-dimensional high-resolution tomographic image reconstructions in both scattered and fluoresced light demonstrating a thousandfold improvement in the depth of field compared to conventional lens-based microscopy.     

Miniaturized three-dimensional endoscopic imaging system based on active stereovision

A miniaturized three-dimensional endoscopic imaging system is presented. The system consists of two imaging in channels that can be used to obtain an image from an object of interest and to project as tructured light onto the imaged object to measure the surface topology. The structured light was generated with a collimated monochromatic light source and a holographic binary phase grating. The imaging and projection channels were calibrated by use of a modified pinhole camera. The surface profile was extracted by use of triangulation between the projected feature points and the two channel ofthe endoscope. The imaging system was evaluated in three-dimensional measurements of several objects with known geometries. The results show that surface profiles of the objects with different surfaces and dimensions can be obtained at high accuracy. The in vivo measurements at tissue sites of human skin and an oral cavity demonstrated the potential of the technique for clinical applications.     

Noise reduction from two frame speckle-shifting ghost images with morphology algorithms

Edge detection is the basis of image segmentation and object recognition, as edge generally contains important information of an object. In this paper, we propose a novel speckle-shifting ghost imaging (SSGI) method to extract the edge of an unknown object. In this method, the gradient operation is directly carried out to the illumination patterns rather than the captured object image. The structured patterns for illumination are only divided into two groups, which can extract the edge in all directions. The imaging result is clearer than the conventional SSGI, but the noise is still serious. To solve the problem, we further investigate a denoising method with morphology algorithms, such as frame difference and connected region labelling. Numerical simulations and experiments are carried out to verify the feasibility and effectiveness.     

Coloured computational imaging with single-pixel detectors based on a 2D discrete cosine transform

We propose and demonstrate a computational imaging technique that uses structured illumination based on a two-dimensional discrete cosine transform to perform imaging with a single-pixel detector. A scene is illuminated by a projector with two sets of orthogonal patterns, then by applying an inverse cosine transform to the spectra obtained from the single-pixel detector a full-colour image is retrieved. This technique can retrieve an image from sub-Nyquist measurements, and the background noise is easily cancelled to give excellent image quality. Moreover, the experimental set-up is very simple.     

Face recognition by using feature-specific imaging

We present a face-recognition system based on the optical measurement of linear features. We describe a polarization-based optical system that computes linear projections of an incident irradiance distribution. We quantify the fundamental limitations of optical feature measurement. We find that higher feature fidelity can be obtained by feature-specific imaging than by postprocessing a conventional image. We present feature-fidelity results for wavelet, principal component, and Fisher features. We study face recognition by using a k-nearest neighbors classifier and two different feed-forward neural networks. Each image block is reduced to either a one- or a two-dimensional feature space for input to these recognition algorithms. As high as 99% recognition has been achieved with one-dimensional wavelet feature projections and 100% has been achieved with two-dimensional projections. A 95-fold increase in noise tolerance by use of feature-specific imaging has been demonstrated for an example of the face-recognition problem. An optical experiment is performed to validate these results.     

Feature-specific imaging

We analyze the performance of feature-specific imaging systems. We study incoherent optical systems that directly measure linear projects of the optical irradiance distribution. Direct feature measurement exploit, the multiplex advantage, and for small numbers of projections can provide higher feature-fidelity than those systems that postprocess a conventional image. We examine feature-specific imaging using Wavelet, Karhunen-Loeve (KL), Hadamard, and independent-component features, quantifying feature fidelity in Gaussian-, shot-, and quantization-noise environments. An example of feature-specific imaging based on KL projections is analyzed and demonstrates that within a high-noise environment it is possible to improve image fidelity via direct feature measurement. A candidate optical system is presented and a preliminary implementational study is undertaken.     

Resolution in structured illumination microscopy: a probabilistic approach

Structured illumination can be employed to extend the lateral resolution of wide-field fluorescence microscopy. Since a structured illumination microscopy image is reconstructed from a series of several acquired images, we develop a modified formulation of the imaging response of the microscope and a probabilistic analysis to assess the resolution performance. We use this model to compare the fluorescence imaging performance of structured illumination techniques to confocal microscopy. Specifically, we examine the trade-off between achievable lateral resolution and signal-to-noise ratio when photon shot noise is dominant. We conclude that for a given photon budget, structured illumination invariably achieves better lateral resolution than confocal microscopy.     

Wide-field depth-sectioning fluorescence microscopy using projector-generated patterned illumination

We present a simple and cost-effective wide-field, depth-sectioning, fluorescence microscope utilizing a commercial multimedia projector to generate excitation patterns on the sample. Highly resolved optical sections of fluorescent pollen grains at 1.9 microm axial resolution are constructed using the structured illumination technique. This requires grid excitation patterns to be scanned across the sample, which is straightforwardly implemented by creating slideshows of gratings at different phases, projecting them onto the sample, and synchronizing camera acquisition with slide transition. In addition to rapid dynamic pattern generation, the projector provides high illumination power and spectral excitation selectivity. We exploit these properties by imaging mouse neural cells in cultures multistained with Alexa 488 and Cy3. The spectral and structural neural information is effectively resolved in three dimensions. The flexibility and commercial availability of this light source is envisioned to open multidimensional imaging to a broader user base.     

Superresolved imaging in digital holography by superposition of tilted wavefronts

A technique based on superresolution by digital holographic microscopic imaging is presented. We used a two dimensional (2-D) vertical-cavity self-emitting laser (VCSEL) array as spherical-wave illumination sources. The method is defined in terms of an incoherent superposition of tilted wavefronts. The tilted spherical wave originating from the 2-D VCSEL elements illuminates the target in transmission mode to obtain a hologram in a Mach-Zehnder interferometer configuration. Superresolved images of the input object above the common lens diffraction limit are generated by sequential recording of the individual holograms and numerical reconstruction of the image with the extended spatial frequency range. We have experimentally tested the approach for a microscope objective with an exact 2-D reconstruction image of the input object. The proposed approach has implementation advantages for applications in biological imaging or the microelectronic industry in which structured targets are being inspected.     

Model-based image reconstruction by means of a constrained least-squares solution

The problem of the optimal use of object model information in image reconstruction is addressed. A closed-form solution for the estimated object spectrum is derived with the Lagrange multiplier technique, which assumes a measured image, knowledge of the optical transfer function, statistical information about the measurement noise, and a model of the object. This reconstruction algorithm is iterative in nature because the optimal Lagrange multiplier is not generally known at the start of the problem. We derive the estimator, describe one technique for determining the optimal Lagrange multiplier, demonstrate a stopping criterion based on the mean-square error between a noise-free image and the photon-limited version of the image, and show representative results for both filled- and sparse-aperture imaging applications.     

Area modulation grating for sinusoidal structure illumination on phase-measuring profilometry

Sinusoidal structured illumination is used widely in three-dimensional (3-D) sensing and machine vision. Phase algorithms, for example, in phase-measuring profilometry, are inherently free of errors only with perfect sinusoidal fringe projection. But it is difficult to produce a perfect sinusoidal grating. We propose a new concept, area modulation, to improve the sinusoidality of structured illumination. A binary-coded picture is made up of many micrometer units. An aperture is open in every micrometer unit, and its area is determined by the value of the sinusoidal function. When such a grating is projected onto an object surface, the image of the grating becomes sinusoidal because of the convolution function of an optical system. We have designed and manufactured an area modulation grating for sinusoidal structure illumination using a large-scale integration technique. The area modulation grating has been used in the high-precision phase-measuring profilometry system, and the phase errors caused by the area modulation grating are reduced to 0.1%. The grating guaranteed the entire measuring accuracy to a 1% equivalent wavelength. The experimental results proved that area modulation grating would be of significant help in improving the phase-measurement accuracy of the 3-D sensing system.     

On the influence of noise statistics on polarimetric contrast optimization

In active scalar polarimetric imaging systems, the illumination and analysis polarization states are degrees of freedom that can be used to maximize the performance. These optimal states depend on the statistics of the noise that perturbs image acquisition. We investigate the problem of optimization of discrimination ability (contrast) of such imagers in the presence of three different types of noise statistics frequently encountered in optical images (Gaussian, Poisson, and Gamma). To compare these different situations within a common theoretical framework, we use the Bhattacharyya distance and the Fisher ratio as measures of contrast. We show that the optimal states depend on a trade-off between the target/background intensity difference and the average intensity in the acquired image, and that this trade-off depends on the noise statistics. On a few examples, we show that the gain in contrast obtained by implementing the states adapted to the noise statistics actually present in the image can be significant.     

Sparsity constrained regularization for multiframe image restoration

In this paper we present a new algorithm for restoring an object from multiple undersampled low-resolution (LR) images that are degraded by optical blur and additive white Gaussian noise. We formulate the multiframe superresolution problem as maximum a posteriori estimation. The prior knowledge that the object is sparse in some domain is incorporated in two ways: first we use the popular l(1) norm as the regularization operator. Second, we model wavelet coefficients of natural objects using generalized Gaussian densities. The model parameters are learned from a set of training objects, and the regularization operator is derived from these parameters. We compare the results from our algorithms with an expectation-maximization (EM) algorithm for l(1) norm minimization and also with the linear minimum-mean-squared error (LMMSE) estimator. Using only eight 4 x 4 pixel downsampled LR images the reconstruction errors of object estimates obtained from our algorithm are 5.5% smaller than by the EM method and 14.3% smaller than by the LMMSE method.     

Dark-field imaging with cylindrical-vector beams

Dark-field illumination provides an imaging mode that rejects specular light, thereby highlighting edge features. We analyze dark-field imaging by using cylindrical vector beam illumination with a confocal microscope equipped with a microstructure fiber mode filter. A numerical model based on rigorous coupled-wave analysis has been used to analyze the method. We acquired images of separated edges features to investigate the edge separation resolution of the method. A through-focus comparison of azimuthal and radial polarization shows a measurable dependence of edge separation on polarization.     

Optical architectures for compressive imaging

We compare three optical architectures for compressive imaging: sequential, parallel, and photon sharing. Each of these architectures is analyzed using two different types of projection: (a) principal component projections and (b) pseudo-random projections. Both linear and nonlinear reconstruction methods are studied. The performance of each architecture-projection combination is quantified in terms of reconstructed image quality as a function of measurement noise strength. Using a linear reconstruction operator we find that in all cases of (a) there is a measurement noise level above which compressive imaging is superior to conventional imaging. Normalized by the average object pixel brightness, these threshold noise standard deviations are 6.4, 4.9, and 2.1 for the sequential, parallel, and photon sharing architectures, respectively. We also find that conventional imaging outperforms compressive imaging using pseudo-random projections when linear reconstruction is employed. In all cases the photon sharing architecture is found to be more photon-efficient than the other two optical implementations and thus offers the highest performance among all compressive methods studied here. For example, with principal component projections and a linear reconstruction operator, the photon sharing architecture provides at least 17.6% less reconstruction error than either of the other two architectures for a noise strength of 1.6 times the average object pixel brightness. We also demonstrate that nonlinear reconstruction methods can offer additional performance improvements to all architectures for small values of noise.     

Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity

Temporal frame-to-frame noise in multipattern structured light projection can significantly corrupt depth measurement repeatability. We present a rigorous stochastic analysis of phase-measuring-profilometry temporal noise as a function of the pattern parameters and the reconstruction coefficients. The analysis is used to optimize the two-frequency phase measurement technique. In phase-measuring profilometry, a sequence of phase-shifted sine-wave patterns is projected onto a surface. In two-frequency phase measurement, two sets of pattern sequences are used. The first, low-frequency set establishes a nonambiguous depth estimate, and the second, high-frequency set is unwrapped, based on the low-frequency estimate, to obtain an accurate depth estimate. If the second frequency is too low, then depth error is caused directly by temporal noise in the phase measurement. If the second frequency is too high, temporal noise triggers ambiguous unwrapping, resulting in depth measurement error. We present a solution for finding the second frequency, where intensity noise variance is at its minimum.     

基于CNN的点云图像融合目标检测

针对自动驾驶场景中目标检测存在尺度变化、光照变化和缺少距离信息等问题,提出一种极具鲁棒性的多模态数据融合目标检测方法,其主要思想是利用激光雷达提供的深度信息作为附加的特征来训练卷积神经网络(CNN)。首先利用滑动窗对输入数据进行切分匹配网络输入,然后采用两个CNN特征提取器提取RGB图像和点云深度图的特征,将其级联得到融合后的特征图,送入目标检测网络进行候选框的位置回归与分类,最后进行非极大值抑制(NMS)处理输出检测结果,包含目标的位置、类别、置信度和距离信息。在KITTI数据集上的实验结果表明,本文方法通过多模态数据的优势互补提高了在不同光照场景下的检测鲁棒性,附加滑动窗处理改善了小目标的检测效果。对比其他多种检测方法,本文方法具有检测精度与检测速度上的综合优势。     

Two-dimensional subwavelength imaging from a hemispherical hyperlens

We report a hemispherical-shaped hyperlens with subwavelength resolution less than 100 nm. Simulations with the finite-element method show that with a 365 nm illumination, the hemispherical hyperlens isotropically magnifies the image along the radial direction. Under linearly polarized light, portions of an object can be resolved. A complete image of the object can be generated by superposing sufficient number of images obtained with incident light in different polarization directions. Such a hyperlens has great potential for realization of nanoscale imaging.     

运动背景中结合特征位移矢量场模糊分割与 OTSU法的运动检测

运动背景中的运动检测难度较大,背景运动补偿后差分以及分割光流场可实现动目标和背景的分离,差分前需进行鲁棒的背景估计,且差分后易出现空洞,而光流估计在噪声以及目标运动速度较大时并不准确,尤其在光照变化时,两种方法均易失效。本文提出一种特征点位移矢量场模糊分割与图像自适应阈值化相结合的运动检测方法,实现在无任何关于运动目标或者运动背景先验信息条件下的动目标检测。通过改进的 SIFT匹配方法生成鲁棒的特征位移矢量场,采用模糊 C均值聚类算法对 SIFT位移矢量场进行无监督分类,实现动目标与背景特征的自适应分离。 OTSU法和形态学操作实现图像的自适应分割,用以修正特征点凸包,最终分割出动目标区域。与鲁棒的背景运动补偿后差分以及光流估计的对比实验表明,在目标运动速度较大、光照变化以及噪声情况下,本文方法均能够检测出运动目标,且在光照变化下的优势明显。     

Super-resolved spatial light interference microscopy

We report a scheme to achieve resolution beyond the diffraction limit in spatial light interference microscopy (SLIM). By adding a grating to the optical path, the structured illumination technique can be used to improve the resolution by a factor of 2. We show that a direct application of the structured illumination technique, however, has proved to be unsuccessful. Through two crucial modifications, namely, one to the pupil plane of the objective and the other to the demodulation procedure, faithful phase information of the object is recovered and the resolution is improved by a factor of 2.