Thin Observation Module by Bound Optics (TOMBO): Concept and Experimental Verification

A compact image-capturing system called TOMBO (an acronym for thin observation module by bound optics) is presented in which the compound-eye imaging system is utilized to achieve a thin optical configuration. The captured multiple images are processed to retrieve the image of the target object. For image retrieval, two kinds of processing method are considered: image sampling and backprojection. Computer simulations and preliminary experiments were executed on an evaluation system to verify the principles of the system and to clarify the issues related to its implementation.     

Bimodal fingerprint capturing system based on compound-eye imaging module

A compact imaging system called TOMBO (Thin Observation Module by Bound Optics) is proposed in which a microlens array is used for thin hardware configuration. This paper describes a fingerprint-capturing module as an application of the TOMBO. Experimental results by the TOMBO prototype system are shown to clarify the applicability of the TOMBO to the fingerprint capturing. Different types of biometrics, i.e., fingerprint and face images, are captured by the same hardware, which shows the extendability of the system for multimodal identification.     

Super-Resolution Reconstruction of Images Based on Microarray Camera

In the field of images and imaging, super-resolution (SR) reconstruction of images is a technique that converts one or more low-resolution (LR) images into a highresolution (HR) image. The classical two types of SR methods are mainly based on applying a single image or multiple images captured by a single camera. Microarray camera has the characteristics of small size, multi views, and the possibility of applying to portable devices. It has become a research hotspot in image processing. In this paper, we propose a SR reconstruction of images based on a microarray camera for sharpening and registration processing of array images. The array images are interpolated to obtain a HR image initially followed by a convolution neural network (CNN) procedure for enhancement. The convolution layers of our convolution neural network are 3×3 or 1×1 layers, of which the 1×1 layers are used to improve the network performance particularly. A bottleneck structure is applied to reduce the parameter numbers of the nonlinear mapping and to improve the nonlinear capability of the whole network. Finally, we use a 3×3 deconvolution layer to significantly reduce the number of parameters compared to the deconvolution layer of FSRCNN-s. The experiments show that the proposed method can not only ameliorate effectively the texture quality of the target image based on the array images information, but also further enhance the quality of the initial high resolution image by the improved CNN.     

Computer-generated holograms of a real three-dimensional object based on stereoscopic video images

A novel method of using stereoscopic video images to synthesize the computer-generated hologram (CGH) patterns of a real 3D object is proposed. Stereoscopic video images of a real 3D object are captured by a 3D camera system. Disparity maps between the captured stereo image pairs are estimated and from these estimated maps the depth data for each pixel of the object can be extracted on a frame basis. By using these depth data and original color images, hologram patterns of a real object can be computationally generated. In experiments, stereoscopic video images of a real 3D object, a wooden rhinoceros doll, are captured by using the Wasol 3D adapter system and its depth data are extracted from them. Then, CGH patterns of 1280 pixels x 1024 pixels are generated with these depth-annotated images of the wooden rhinoceros doll, and the CGH patterns are experimentally displayed via a holographic display system.     

基于二次曲面拟合的亚象素图象匹配算法

在图象测量系统中，目标的精确定位是一个关键问题，也是应用其它图像处理技术的基础。传统的图象匹配算法只能在象素极定位，本文基于相关函数的二次曲面拟合提出了一种亚象素精度的匹配算法，它对于无噪声图象匹配的绝对误差小于０．０１象素。模拟实验表明，在有噪声的情况下该算法仍具有较小的偏差。     

Small-kernel superresolution methods for microscanning imaging systems

Two computationally efficient methods for superresolution reconstruction and restoration of microscanning imaging systems are presented. Microscanning creates multiple low-resolution images with slightly different sample-scene phase shifts. The digital processing methods developed here combine the low-resolution images to produce an image with higher pixel resolution (i.e., superresolution) and higher fidelity. The methods implement reconstruction to increase resolution and restoration to improve fidelity in one-pass convolution with a small kernel. One method uses a small-kernel Wiener filter and the other method uses a parametric cubic convolution filter. Both methods are based on an end-to-end, continuous-discrete-continuous microscanning imaging system model. Because the filters are constrained to small spatial kernels they can be efficiently applied by convolution and are amenable to adaptive processing and to parallel processing. Experimental results with simulated imaging and with real microscanned images indicate that the small-kernel methods efficiently and effectively increase resolution and fidelity.     

Image reconstruction for thin observation module by bound optics by using the iterative backprojection method

A method for reconstructing high-spatial-resolution images in an imaging system known as thin observation module by bound optics (TOMBO) is reported. We investigate a novel procedure combining a pixel-rearrangement method and iterative backprojection (IBP). Pixel rearrangement has been used until now in TOMBO, and IBP is a digital superresolution technique. We verify the effectiveness of the combined procedure with simulated and experimental results.     

Accurate pixel-to-pixel correspondence adjustment in a digital micromirror device camera by using the phase-shifting moiré method

A camera based on the digital micromirror device (DMD) technology has been previously developed. In this optical system, the correspondence of each mirror of the DMD to each pixel of the CCD cannot readily be done since the pixel sizes of the DMD and the CCD are very small. An accurate pixel-to-pixel correspondence adjustment in the DMD camera by means of the phase-shifting moiré method is proposed. To perform high accurate adjustment of the optical system, the phase distribution of a moiré fringe pattern is analyzed when the CCD pixels and the DMD mirrors have a mismatch and/or misalignment with each other. This technique does not need a complicated setting or complex image processing to generate the moiré fringe pattern, and it needs only one captured image. In the adjustment experiment, the proposed method provided very accurate adjustment whose error was less than 1/25 pixel. An experiment of phase analysis to demonstrate the usefulness was performed.     

Use of Local Region Maps on Convolutional LSTM for Single-Image HDR Reconstruction

Low dynamic range (LDR) images captured by consumer cameras have a limited luminance range. As the conventional method for generating high dynamic range (HDR) images involves merging multiple-exposure LDR images of the same scene (assuming a stationary scene), we introduce a learning-based model for single-image HDR reconstruction. An input LDR image is sequentially segmented into the local region maps based on the cumulative histogram of the input brightness distribution. Using the local region maps, SParam-Net estimates the parameters of an inverse tone mapping function to generate a pseudo-HDR image. We process the segmented region maps as the input sequences on long short-term memory. Finally, a fast super-resolution convolutional neural network is used for HDR image reconstruction. The proposed method was trained and tested on datasets including HDR-Real, LDR-HDR-pair, and HDR-Eye. The experimental results revealed that HDR images can be generated more reliably than using contemporary end-to-end approaches.     

Superresolution image reconstruction from a sequence of aliased imagery

We present a superresolution image reconstruction from a sequence of aliased imagery. The subpixel shifts (displacement) among the images are unknown due to the uncontrolled natural jitter of the imager. A correlation method is utilized to estimate subpixel shifts between each low-resolution aliased image with respect to a reference image. An error-energy reduction algorithm is derived to reconstruct the high-resolution alias-free output image. The main feature of this proposed error-energy reduction algorithm is that we treat the spatial samples from low-resolution images that possess unknown and irregular (uncontrolled) subpixel shifts as a set of constraints to populate an oversampled (sampled above the desired output bandwidth) processing array. The estimated subpixel locations of these samples and their values constitute a spatial domain constraint. Furthermore, the bandwidth of the alias-free image (or the sensor imposed bandwidth) is the criterion used as a spatial frequency domain constraint on the oversampled processing array. The results of testing the proposed algorithm on the simulated low- resolution forward-looking infrared (FLIR) images, real-world FLIR images, and visible images are provided. A comparison of the proposed algorithm with a standard interpolation algorithm for processing the simulated low-resolution FLIR images is also provided.     

Upscaling image resolution of compact imaging systems using wavefront coding and a property of the point-spread function

We propose an image-resolution upscaling method for compact imaging systems. The image resolution is calculated using the resolving power of the optics and the pixel size of a digital image sensor. The resolution limit of the compact imaging system comes from its size and the number of allowed lenses. To upscale the image resolution but maintain the small size, we apply wavefront coding and image restoration. Conventional image restoration could not enhance the image resolution of the sensor. Here, we use the upscaled image of a wavefront-coded optical system and apply an image-restoration algorithm using a more precisely calculated point-spread function (PSF) as the deconvolution filter. An example of a wavefront-coded optical system with a 5-megapixel image sensor is given. The final image had a resolution equivalent to that of a 10-megapixel image using only four plastic lenses. Moreover, image degradation caused by hand motion could also be reduced using the proposed method.     

基于图像的启发式三维火焰重建算法

研究了火焰的重建,为改进现有稀疏视角输入下的层析成像算法的重建精度,提出了一种基于图像的启发式火焰重建方法。该重建方法通过迭代优化求解能量约束模型,实现对火焰三维温度场的重建。算法允许复杂的成像模型应用于重建过程中,使渲染过程更接近火焰真实成像过程,并对数据平滑性进行约束,以提高重建结果质量。算法在迭代过程中使用启发式规则引导温度场数据调整,使算法快速收敛,并通过使用图形处理器加速重建过程。使用模拟数据和真实捕获数据进行实验,结果表明,与现有方法相比,提出的方法有效提高了重建结果的精度及平滑性,验证了提出方法的有效性。     

用傅里叶相移特性估计位移

提出了一种从频率域出发,估计出运动物体在空间域位移的算法。利用傅里叶变换的自配准性质和位移特性,用极坐标形式下连续图像相位谱的差直接估计运动目标的位移。与传统的寻找迪拉克峰值的方法相比,本方法无需重新变换回空间域,从而节省了处理时间,具有更好的实时性。实验证明,这种先求出相位谱的差,再用其周期数来估计位移的算法简单明了,其分辨能力不低于一个像素。该理论可以应用于图像跟踪和图像后期处理中。     

Differential pixel-based low-power and high-speed implementation of DCT for on-board satellite image processing

Low-power and high-speed discrete cosine transform (DCT) implementation of the images captured by the satellites presents a hardware design problem. The cost of the DCT implementation is dominated by the complexity of the multiplication of input data (image) with the DCT matrix. The techniques for minimising the complexity of multiplication by employing a differential pixel method are presented. In the proposed method 8times8 blocks of input image matrix are considered, the difference between the adjacent pixels is calculated and those differential pixels are used in DCT transformation. Synthesis results on 0.18 mum CMOS technology show that the proposed method gives an average of 13.2% reduction in power consumption and 10.9% improvement in speed over the conventional method. The proposed method can also be combined with the common subexpression elimination method for further reduction.     

Single-shot full resolution region-of-interest (ROI) reconstruction in image plane digital holographic microscopy

We describe a numerical processing technique that allows single-shot region-of-interest (ROI) reconstruction in image plane digital holographic microscopy with full pixel resolution. The ROI reconstruction is modelled as an optimization problem where the cost function to be minimized consists of an L2-norm squared data fitting term and a modified Huber penalty term that are minimized alternately in an adaptive fashion. The technique can provide full pixel resolution complex-valued images of the selected ROI which is not possible to achieve with the commonly used Fourier transform method. The technique can facilitate holographic reconstruction of individual cells of interest from a large field-of-view digital holographic microscopy data. The complementary phase information in addition to the usual absorption information already available in the form of bright field microscopy can make the methodology attractive to the biomedical user community.     

Efficient fringe image enhancement based on dual-tree complex wavelet transform

In optical phase shift profilometry (PSP), parallel fringe patterns are projected onto an object and the deformed fringes are captured using a digital camera. It is of particular interest in real time three-dimensional (3D) modeling applications because it enables 3D reconstruction using just a few image captures. When using this approach in a real life environment, however, the noise in the captured images can greatly affect the quality of the reconstructed 3D model. In this paper, a new image enhancement algorithm based on the oriented two-dimenional dual-tree complex wavelet transform (DT-CWT) is proposed for denoising the captured fringe images. The proposed algorithm makes use of the special analytic property of DT-CWT to obtain a sparse representation of the fringe image. Based on the sparse representation, a new iterative regularization procedure is applied for enhancing the noisy fringe image. The new approach introduces an additional preprocessing step to improve the initial guess of the iterative algorithm. Compared with the traditional image enhancement techniques, the proposed algorithm achieves a further improvement of 7.2 dB on average in the signal-to-noise ratio (SNR). When applying the proposed algorithm to optical PSP, the new approach enables the reconstruction of 3D models with improved accuracy from 6 to 20 dB in the SNR over the traditional approaches if the fringe images are noisy.     

基于Hǒlder指数和ICA的海域SAR图点目标检测

本文将分形Hǒlder指数和信号分离相结合,利用独立成分分析技术（ICA,Independent Component Analysis）,实现了海杂波SAR图的散斑抑制和点目标检测。首先,计算点态Hǒlder指数图,并提出二值模糊方法对其处理;接着使用ICA技术得到该图的基图像和独立成分;提出空间分离法,对独立成分进行分离,同时对基图进行对应分类,获得非噪声和噪声两个空间。最后在非噪声空间上重构图像。实验部分,将该算法与传统算法进行对比,证实了该算法的有效性和优越性。     

Fast-updating and nonrepeating Lissajous image reconstruction method for capturing increased dynamic information

We present a fast-updating Lissajous image reconstruction methodology that uses an increased image frame rate beyond the pattern repeat rate generally used in conventional Lissajous image reconstruction methods. The fast display rate provides increased dynamic information and reduced motion blur, as compared to conventional Lissajous reconstruction, at the cost of single-frame pixel density. Importantly, this method does not discard any information from the conventional Lissajous image reconstruction, and frames from the complete Lissajous pattern can be displayed simultaneously. We present the theoretical background for this image reconstruction methodology along with images and video taken using the algorithm in a custom-built miniaturized multiphoton microscopy system.     

Image Processing Vision Systems: Standard Image Sensors Versus Retinas

To decrease the computational complexity of computer vision algorithms, one of the solutions is to achieve some low-level image processing on the sensor focal plan. It becomes a smart sensor device called a retina. This concept makes the vision systems more compact. It increases performances thanks to the reduction of data flow exchanges with external circuits. This paper presents a comparison relating two different vision system architectures. The first one implements a logarithmic complimentary metal-oxide-semiconductor (CMOS)/active pixel sensor interfaced to a microprocessor, where all computations are carried out. The second involves a CMOS sensor including analog processors allowing on-chip image processing. An external microprocessor is used to control the on-chip data flow and integrated operators. We have designed two vision systems as proof-of-concept. The comparison is related to image processing computation time, processing reliability, programmability, precision, bandwidth, and subsequent stages of computations.     

3-D ultrasound volume reconstruction using the direct frame interpolation method

A new method for 3-D ultrasound volume reconstruction using tracked freehand 3-D ultrasound is proposed. The method is based on solving the forward volume reconstruction problem using direct interpolation of high-resolution ultrasound B-mode image frames. A series of ultrasound B-mode image frames (an image series) is acquired using the freehand scanning technique and position sensing via optical tracking equipment. The proposed algorithm creates additional intermediate image frames by directly interpolating between two or more adjacent image frames of the original image series. The target volume is filled using the original frames in combination with the additionally constructed frames. Compared with conventional volume reconstruction methods, no additional filling of empty voxels or holes within the volume is required, because the whole extent of the volume is defined by the arrangement of the original and the additionally constructed B-mode image frames. The proposed direct frame interpolation (DFI) method was tested on two different data sets acquired while scanning the head and neck region of different patients. The first data set consisted of eight B-mode 2-D frame sets acquired under optimal laboratory conditions. The second data set consisted of 73 image series acquired during a clinical study. Sample volumes were reconstructed for all 81 image series using the proposed DFI method with four different interpolation orders, as well as with the pixel nearest-neighbor method using three different interpolation neighborhoods. In addition, volumes based on a reduced number of image frames were reconstructed for comparison of the different methods' accuracy and robustness in reconstructing image data that lies between the original image frames. The DFI method is based on a forward approach making use of a priori information about the position and shape of the B-mode image frames (e.g., masking information) to optimize the reconstruction procedure and to reduce computation times and memory requirements. The method is straightforward, independent of additional input or parameters, and uses the high-resolution B-mode image frames instead of usually lower-resolution voxel information for interpolation. The DFI method can be considered as a valuable alternative to conventional 3-D ultrasound reconstruction methods based on pixel or voxel nearest-neighbor approaches, offering better quality and competitive reconstruction time.