Enhanced depth of field of integral imaging display using bifocal microlens array fabricated by two-step lithography

Due to the limitation of traditional microlens arrays (MLAs) in integral imaging display, the depth of field (DOF) is restricted in space and the center depth plane is difficult to extend in a large range. Here, we propose a microfabrication method based on bifocal MLAs to improve DOF. The bifocal MLAs for extended DOF were fabricated by using two-step photolithography and thermal reflow. This method allows diverse microlenses of high to low numerical aperture to achieve high spatial resolution as well as accurate depth estimation. Microlenses of different focal lengths were simultaneously deposited on a substrate by repeated photolithography with multiple photomasks with alignment mark to define micro-posts of different thicknesses. Hexagonally packaged bifocal MLAs clearly show the DOF extended from 0.004 to 4.908 mm for 57.6 μm in lens diameter, and their corresponding object distance ranges from 0.125 to 0.165 mm. Based on the proposed scheme, this method provides potential applications in integral imaging 3D display or light field display.     

A high optical efficiency 3D/2D convertible integral imaging display

We propose a high optical efficiency three‐dimensional (3D)/two‐dimensional (2D) convertible integral imaging display by using a pinhole array on a reflective polarizer. The 3D mode is realized by adopting a pinhole array on a reflective polarizer to generate a point light source array. Three‐dimensional/2D convertible feature is realized by electrically controlling a polarization switcher. The reflective polarizer can reflect the light that has the orthogonal polarization direction with the reflective polarizer and transmit the light that has the same polarization direction with the reflective polarizer. The reflected light is recycled, so the optical efficiencies for both 3D and 2D modes are enhanced. In the practical experiments, the optical efficiencies of the proposed integral imaging display increase by 8.04 times and 1.65 times in 3D and 2D modes comparing with the conventional integral imaging display that has no light recycle, respectively.     

Registration and occlusion handling based on the FAST ICP-ORB method for augmented reality systems

Spatial position consistency and occlusion consistency are two important problems in augmented reality systems. In this paper, we proposed a novel method that can address the registration problem and occlusion problem simultaneously by using an RGB-D camera. First, to solve the image alignment errors caused by the imaging mode of the RGB-D camera, we developed a depth map inpainting method that combines the FMM and RGB-D information. Second, we established an automatic method to judge the close-range mode based on the depth histogram to solve the registration failure problem caused by hardware limitations. In the close-range mode, the registration method combining the fast ICP and ORB was adopted to calculate the camera pose. Third, we developed an occlusion handling method based on the geometric analysis of the scene. Several experiments were performed to validate the performance of the proposed method. The experimental results indicate that our method can obtain stable and accurate registration and occlusion handling results in both the close-range and non-close-range modes. Moreover, the mutual occlusion problem can be handled effectively, and the proposed method can satisfy the real-time requirements of augmented reality systems.

     

360°‐viewable cylindrical integral imaging system using a 3‐D/2‐D switchable and flexible backlight

Abstract— A 360°‐viewable cylindrical three‐dimensional (3‐D) display system based on integral imaging has been implemented. The proposed system is composed of a cylindrically arranged electroluminescent (EL) pinhole film, an EL film backlight, a barrier structure, and a transmission‐type flexible display panel. The cylindrically arranged point‐light‐source array, which is generated by the EL pinhole film reconstructs 360°‐viewable virtual 3‐D images at the center of the cylinder. In addition, the proposed system provides 3‐D/2‐D convertibility using the switching of EL pinhole film from a point light source to a surface light source. In this paper, the principle of operation, analysis of the viewing parameters, and the experimental results are presented.     

Fabrication of pinhole/micro‐lens array for improving resolution of integral imaging 3‐D display

In this paper, we present a new fabrication of micro‐lens array (MLA) with pinhole array—pinhole/micro‐lens array (P/MLA) for integral imaging 3‐D display (II), which combine lithography and ink‐jet printing. A black circular groove array (BCGA) is used as pinhole array, and laser 3‐D microscope and a homemade setup have been used for the characterization of P/MLA. The results show that high‐precision P/MLA can be obtained using BCGA as templates. By controlling the driving voltage at different steps, the distance between nozzle and substrate, as well as the number of liquid droplets, P/MLA with smooth morphology, different sizes, good repeatability of geometry parameters, great uniformity of focusing, and good converging performance can be achieved. For demonstration, P/MLA with curvature, focal length, numerical aperture, and F‐number of 815.8 μm, 1.60 mm, 0.1311, and 3.8 are applied for the reconstruction in II, exhibiting good reconstruction performance with high resolution, and BCGA reduces the influence of stray light on II and improves the quality of the reconstructed image.     

A theoretical 2D image model for locating 3D targets

To construct a water quality monitoring system, challenging issues need to be addressed regarding the acquisition of target information (e.g. 3D location and occlusion) as well as the behavioural analysis of aquatic organisms. This paper presents a novel 3D information acquisition and location method, by means of an information acquisition platform consisting of a monitoring terminal, frame grabbers, a single camera and a single mirror. Using this platform, we propose a theoretical 2D image model for locating 3D targets and then validate it using data obtained from both real and artificial fish. The proposed model is based on the principles of light refraction, plane mirror imaging, underwater objects and camera imaging as well as the technologies of digital to analog conversion and object segmentation. In contrast with existing methods, our method can accurately reflect 3D information of aquatic organisms, thus providing critical technical support for the development of water quality monitoring systems in the future.     

3D遮挡模型引导的光场图像深度获取

目的 光场相机可以通过单次曝光同时从多个视角采样单个场景,在深度估计领域具有独特优势。消除遮挡的影响是光场深度估计的难点之一。现有方法基于2D场景模型检测各视角遮挡状态,但是遮挡取决于所采样场景的3D立体模型,仅利用2D模型无法精确检测,不精确的遮挡检测结果将降低后续深度估计精度。针对这一问题,提出了3D遮挡模型引导的光场图像深度获取方法。方法 向2D模型中的不同物体之间添加前后景关系和深度差信息,得到场景的立体模型,之后在立体模型中根据光线的传输路径推断所有视角的遮挡情况并记录在遮挡图（occlusion map）中。在遮挡图引导下,在遮挡和非遮挡区域分别使用不同成本量进行深度估计。在遮挡区域,通过遮挡图屏蔽被遮挡视角,基于剩余视角的成像一致性计算深度;在非遮挡区域,根据该区域深度连续特性设计了新型离焦网格匹配成本量,相比传统成本量,该成本量能够感知更广范围的色彩纹理,以此估计更平滑的深度图。为了进一步提升深度估计的精度,根据遮挡检测和深度估计的依赖关系设计了基于最大期望（exception maximization,EM）算法的联合优化框架,在该框架下,遮挡图和深度图通过互相引导的方式相继提升彼此精度。结果 实验结果表明,本文方法在大部分实验场景中,对于单遮挡、多遮挡和低对比度遮挡在遮挡检测和深度估计方面均能达到最优结果。均方误差（mean square error,MSE）对比次优结果平均降低约19.75%。结论 针对遮挡场景的深度估计,通过理论分析和实验验证,表明3D遮挡模型相比传统2D遮挡模型在遮挡检测方面具有一定优越性,本文方法更适用于复杂遮挡场景的深度估计。     

Fast calculation of computer generated hologram based on single Fourier transform for holographic three-dimensional display

We present an efficient method for the fast calculation of computer generated hologram (CGH). The 3D object is split into sub-layers according to its depth information. A 2D all-in-focus image is generated by sequential tiling all the layers in one plane. A Fourier hologram that contains all the information of 3D object is calculated from the fast Fourier transform (FFT) of the reassembled 2D image. By multiplying a pre-calculated multifocal off-axis digital phase mask (DPM) to the Fourier hologram, the content of each layer is axially relocated to different depth in the Fourier transform optical system to reconstruct the 3D object. The computation speed of the proposed method is greatly improved with only single FFT calculation process. Both of simulation and experimental results proves the validation of the proposed method.     

Light field resampling method for elemental image array generation of integral imaging

In this paper, we proposed a light field resampling method for generating elemental image array (EIA) of integral imaging. The proposed method can break through the constraints between parameters of traditional integral imaging record device and display device and allow to generate a new set of EIA suited to be displayed in an integral imaging display device with arbitrary parameters from any given recorded EIA. Three‐dimensional display results show the correctness and superiority of the proposed method.     

Numerical investigation of zoomable holographic projection without a zoom lens

Holographic projection is for laser displays and has the merits of being aberration free, producing high‐contrast images, having the ability of color reconstruction with one spatial light modulator, and so on. In this paper, we propose a zoomable holographic projection without using a zoom lens and verify the proposed method by using numerical simulation. Although such a system can be readily realized to use the features of holography, which is capable of recording a large image exceeding the hologram size, the required calculation is very time consuming. For acceleration, we used shifted Fresnel diffraction for setting different sampling rates on a hologram and projected image. The proposed method can project any zoom‐in and zoom‐out image between zeroth‐order and first‐order lights, and the processing time and required memory for the zoom are constant.     

Close-range hyperspectral imaging for geological field studies: workflow and methods

Close-range hyperspectral imaging is a new method for geological research, in which imaging spectrometry is applied from the ground, allowing the mineralogy and lithology in near-vertical cliff sections to be studied in detail. Contemporary outcrop studies often make use of photorealistic three-dimensional (3D) models, derived from terrestrial laser scanning (lidar), that facilitate geological interpretation of geometric features. Hyperspectral imaging provides complementary geochemical information that can be combined with lidar models, enhancing quantitative and qualitative analyses. This article describes a complete workflow for applying close-range hyperspectral imaging, from planning the optimal scan conditions and data acquisition, through pre-processing the hyperspectral imagery and spectral mapping, integration with lidar photorealistic 3D models, and analysis of the geological results. Pre-processing of the hyperspectral images involves the reduction of scanner artefacts and image discontinuities, as well as relative reflectance calibration using empirical line correction, based on two calibrated reflection targets. Signal-to-noise ratios better than 70:1 are achieved for materials with 50% reflectance. The lidar-based models are textured with products such as hyperspectral classification maps. Examples from carbonate and siliciclastic geological environments are presented, with results showing that spectrally similar material, such as different dolomite types or sandstone and siltstone, can be distinguished and spectrally mapped. This workflow offers a novel and flexible technique for applications, in which a close-range instrument setup is required and the spatial distribution of minerals or chemical variations is valuable.     

Disparity-based just-noticeable-difference model for perceptual stereoscopic video coding using depth of focus blur effect

Human 3D perception provides an important clue to the removal of redundancy in stereoscopic 3D (S3D) videos. Because objects outside the binocular fusion limit cannot be fused on retina, the human visual system (HVS) makes them blur according to the depth-of-focus (DOF) effect to increase the binocular fusion limit and suppress diplopia, i.e. double vision. Based on human depth perception, we propose a disparity-based just-noticeable-difference model (DJND) to save bit-rate and improve visual comfort in S3D videos. We combine the DOF blur effect with conventional JND models in the pixel domain into DJND. Firstly, we use disparity information to get the average disparity value of each block. Then, we integrate the DOF blur effect into luminance JND (LJND) by a selective low pass Gaussian filter to minimize the visual stimulus in S3D videos. Finally, we incorporate disparity information into the filtered JND models to obtain DJND. Experimental results demonstrate that the proposed method successfully improves both image quality and visual comfort in viewing S3D videos without increasing the bit-rate.     

Combining optical holograms with interactive computer graphics

Holograms can reconstruct complete optical wavefronts, capturing images that have a three-dimensional appearance and can be observed from different perspectives. Museum exhibits often use optical hologram technology because it permits presentation of 3D objects with almost no loss in visual quality. Optical holograms are static, however, and lack interactivity. Combining 3D computer graphical elements with stereoscopic presentation techniques provides an alternative that allows interactivity.     

数字近景摄影测量技术在矿山地表沉陷监测中的应用研究

为了解决传统的矿山地表沉陷监测方法存在的劳动强度大、不能实时观测、难以获得瞬时3维移动变形信息等缺陷,提出了一种利用基于非量测数相机的数字近景摄影测量技术进行塌陷区沉陷监测的方法。该方法首先利用标定过的高分辨率数字相机获取塌陷区的数字立体影像像对;然后采用相对定向、影像匹配等摄影测量解析处理技术提取塌陷区地表的数字高程模型;最后通过与该区域开采前数字高程模型对比分析来完成塌陷区沉陷范围、深度、体积等沉陷参数计算。应用研究结果表明,数字近景摄影测量的测定精度基本可以满足这次应用条件下沉陷监测的精度要求。该方法为矿山地表塌陷区的沉陷监测提供了一条可行的途径。     

Three-Dimensional Scene Encryption Algorithm Based on Phase Iteration Algorithm of the Angular-Spectral Domain

In order to increase the capacity of encrypted information and reduce the loss of information transmission, a three-dimensional (3D) scene encryption algorithm based on the phase iteration of the angular spectrum domain is proposed in this paper. The algorithm, which adopts the layer-oriented method, generates the computer generated hologram by encoding the three-dimensional scene. Then the computer generated hologram is encoded into three pure phase functions by adopting the phase iterative algorithm based on angular spectrum domain, and the encryption process is completed. The three-dimensional scene encryption can improve the capacity of the information, and the three-phase iterative algorithm can guarantee the security of the encryption information. The numerical simulation results show that the algorithm proposed in this paper realized the encryption and decryption of three-dimensional scenes. At the same time, it can ensure the safety of the encrypted information and increase the capacity of the encrypted information.      

Computer Generated Holograms for Optical Neural Networks

While numerous artificial neural network (ANN) models have been electronically implemented and simulated by conventional computers, optical technology provides a far superior mechanism for the implementation of large-scale ANNs. The properties of light make it an ideal carrier of data signals. With optics, very large and high speed neural network architectures are possible. Because light is a predictable phenomenon, it can be described mathematically and its behavior can be simulated by conventional computers. A hologram is in essence a capture of the light field at a particular moment in time and space. Later, the hologram can be used to reconstruct the three dimensional light field carrying optical data. This makes a hologram an ideal medium for capturing, storing, and transmitting data in optical computers, such as optical neural networks (ONNs). Holograms can be created using conventional methods, but they can also be computer generated. In this paper, we will present an overview of optical neural networks, with emphasis on the holographic neural networks. We will take a look at the mathematical basis of holography in terms of the Fresnel Zone Plate and how it can be utilized in making computer generated holograms (CGHs). Finally, we will present various methods of CGH implementation in a two layer holographic ONN.     

改进格雷码结构光双目视觉工件3D测量

机器人喷涂、焊接、研磨等现代制造过程,通常需要获取工件3D几何形状信息以实现路径自动规划。针对此需求,提出了一种改进灰度编码的结构光双目视觉成像与三维测量系统。在离线标定获得相机参数的基础上,采用黑白反向重投影方法解决传统格雷码结构光成像中边界模糊问题;同时,提出快速区域块搜索算法,用于立体匹配和视差图生成;最后,采用Lanczos插值算法填补遮挡造成的视差图空白,从而获得完整的深度信息,实现工件高精度3D重建。实际工件成像实验结果表明：该方法能够实现工件快速、高精度三维重建,尤其对无纹理和弱纹理目标能获得较好的结果,能够满足工业环境测量需求。     

Plenoptic video geometry

More and more processing of visual information is nowadays done by computers, but the images captured by conventional cameras are still based on the pinhole principle inspired by our own eyes. This principle though is not necessarily the optimal image-formation principle for automated processing of visual information. Each camera samples the space of light rays according to some pattern. If we understand the structure of the space formed by the light rays passing through a volume of space, we can determine the camera, or in other words the sampling pattern of light rays, that is optimal with regard to a given task. In this work we analyze the differential structure of the space of time-varying light rays described by the plenoptic function and use this analysis to relate the rigid motion of an imaging device to the derivatives of the plenoptic function. The results can be used to define a hierarchy of camera models with respect to the structure from motion problem and formulate a linear, scene-independent estimation problem for the rigid motion of the sensor purely in terms of the captured images.     

Camera calibration and light source orientation from solar shadows

In this paper, we describe a method for recovering camera parameters from perspective views of daylight shadows in a scene, given only minimal geometric information determined from the images. This minimal information consists of two 3D stationary points and their cast shadows on the ground plane. We show that this information captured in two views is sufficient to determine the focal length, the aspect ratio, and the principal point of a pinhole camera with fixed intrinsic parameters. In addition, we are also able to compute the orientation of the light source. Our method is based on exploiting novel inter-image constraints on the image of the absolute conic and the physical properties of solar shadows. Compared to the traditional methods that require images of some precisely machined calibration patterns, our method uses cast shadows by the sun, which are common in natural environments, and requires no measurements of any distance or angle in the 3D world. To demonstrate the accuracy of the proposed algorithm and its utility, we present the results on both synthetic and real images, and apply the method to an image-based rendering problem.