Viewing angle enhanced integral imaging display based on double‐micro‐lens array

A viewing angle enhanced integral imaging display, which consists of a double microlens array, and a display panel is proposed. The double microlens array includes a convex microlens array and a concave microlens array. The display panel is used to display original elemental image array. The convex microlens array, located near the display panel, is used to provide a virtual elemental image array for the concave microlens array. The concave microlens array, located far away from the display panel, is used to display integral images with the virtual elemental image array. Compared with the original elemental image, the pitch for each virtual elemental image is magnified by the corresponding convex microlens. As a result, the viewing angle is expanded. Simulations based on ray‐tracing are performed and the results agree well with the theory.     

Light field resampling method for elemental image array generation of integral imaging

In this paper, we proposed a light field resampling method for generating elemental image array (EIA) of integral imaging. The proposed method can break through the constraints between parameters of traditional integral imaging record device and display device and allow to generate a new set of EIA suited to be displayed in an integral imaging display device with arbitrary parameters from any given recorded EIA. Three‐dimensional display results show the correctness and superiority of the proposed method.     

快速3维坐标变换的绘制算法

目的 基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法 主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果 实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355dB,SSIM平均提高了0.00115。结论 改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。     

Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging

In this paper, we propose an orthographic view image generation method based on integral imaging called multiple elemental image mapping. The proposed method is based on matching regions in multiple elemental images, which are generated with different micro‐lens from the same area of a 3D object. By mapping all the matching regions to the corresponding orthographic view plane, we can enhance the resolution of the generated orthographic view image. Simulation experiments are carried out, and the results show the resolution of orthographic view image is improved obviously.     

Enhanced depth of field of integral imaging display using bifocal microlens array fabricated by two-step lithography

Due to the limitation of traditional microlens arrays (MLAs) in integral imaging display, the depth of field (DOF) is restricted in space and the center depth plane is difficult to extend in a large range. Here, we propose a microfabrication method based on bifocal MLAs to improve DOF. The bifocal MLAs for extended DOF were fabricated by using two-step photolithography and thermal reflow. This method allows diverse microlenses of high to low numerical aperture to achieve high spatial resolution as well as accurate depth estimation. Microlenses of different focal lengths were simultaneously deposited on a substrate by repeated photolithography with multiple photomasks with alignment mark to define micro-posts of different thicknesses. Hexagonally packaged bifocal MLAs clearly show the DOF extended from 0.004 to 4.908 mm for 57.6 μm in lens diameter, and their corresponding object distance ranges from 0.125 to 0.165 mm. Based on the proposed scheme, this method provides potential applications in integral imaging 3D display or light field display.     

Depth image‐based rendering for multiview generation

Abstract— Techniques for 3‐D display have evolved from stereoscopic 3‐D systems to multiview 3‐D systems, which provide images corresponding to different viewpoints. Currently, new technology is required for application in multiview display systems that use input‐source formats such as 2‐D images to generate virtual‐view images of multiple viewpoints. Due to the changes in viewpoints, occlusion regions of the original image become disoccluded, resulting in problems related to the restoration of output image information that is not contained in the input image. In this paper, a method for generating multiview images through a two‐step process is proposed: (1) depth‐map refinement and (2) disoccluded‐area estimation and restoration. The first step, depth‐map processing, removes depth‐map noise, compensates for mismatches between RGB and depth, and preserves the boundaries and object shapes. The second step, disoccluded‐area estimation and restoration, predicts the disoccluded area by using disparity and restores information about the area by using information about neighboring frames that are most similar to the occlusion area. Finally, multiview rendering generates virtual‐view images by using a directional rendering algorithm with boundary blending.     

Double‐viewing‐zone integral imaging 3D display without crosstalk based on a tilted barrier array

We propose an integral imaging (II) three‐dimensional (3D) display using a tilted barrier array and a stagger microlens array. The tilted barrier array consists of two orthogonally polarized sheets. In the stagger microlens array, the center of the microlens has p/2 shift with the elemental image along the horizontal direction, where p is the pitch of the microlens. The proposed II 3D display produces two different viewing zones and each of them is almost equal to that of the conventional II 3D display, and it has no crosstalk. We verify the feasibility of the proposed II 3D display in the simulation results.     

Resolution enhancement of integral‐imaging three‐dimensional display using directional elemental image projection

Abstract— A new approach to resolution enhancement of an integral‐imaging (II) three‐dimensional display using multi‐directional elemental images is proposed. The proposed method uses a special lens made up of nine pieces of a single Fresnel lens which are collected from different parts of the same lens. This composite lens is placed in front of the lens array such that it generates nine sets of directional elemental images to the lens array. These elemental images are overlapped on the lens array and produce nine point light sources per each elemental lens at different positions in the focal plane of the lens array. Nine sets of elemental images are projected by a high‐speed digital micromirror device and are tilted by a two‐dimensional scanning mirror system, maintaining the time‐multiplexing sequence for nine pieces of the composite lens. In this method, the concentration of the point light sources in the focal plane of the lens array is nine‐times higher, i.e., the distance between two adjacent point light sources is three times smaller than that for a conventional II display; hence, the resolution of three‐dimensional image is enhanced.     

Fabrication of pinhole/micro‐lens array for improving resolution of integral imaging 3‐D display

In this paper, we present a new fabrication of micro‐lens array (MLA) with pinhole array—pinhole/micro‐lens array (P/MLA) for integral imaging 3‐D display (II), which combine lithography and ink‐jet printing. A black circular groove array (BCGA) is used as pinhole array, and laser 3‐D microscope and a homemade setup have been used for the characterization of P/MLA. The results show that high‐precision P/MLA can be obtained using BCGA as templates. By controlling the driving voltage at different steps, the distance between nozzle and substrate, as well as the number of liquid droplets, P/MLA with smooth morphology, different sizes, good repeatability of geometry parameters, great uniformity of focusing, and good converging performance can be achieved. For demonstration, P/MLA with curvature, focal length, numerical aperture, and F‐number of 815.8 μm, 1.60 mm, 0.1311, and 3.8 are applied for the reconstruction in II, exhibiting good reconstruction performance with high resolution, and BCGA reduces the influence of stray light on II and improves the quality of the reconstructed image.     

Integral floating‐image display using two lenses with reduced distortion and enhanced depth

Abstract— An integral floating display (IFD) with a long depth range without floating lens distortion is proposed. Two lenses were used to reduce barrel distortion of the floating lens and three‐dimensional (3‐D) image deformation from object‐dependent longitudinal and lateral magnifications in the floating‐display system, combined with an integral imaging display. The distance between the floating lenses is the sum of their focal lengths. In the proposed configuration, lateral and longitudinal magnifications are constant regardless of the distance of the integrated 3‐D images, so the distortions from the distant‐dependent magnifications of the floating lens do not occur with the proposed method. In addition, the proposed floating system expands the depth range of the integral imaging display. As a result, the display can show a correct 3‐D floating image with a large depth range. Experimental results demonstrate that the proposed method successfully displays a 3‐D image without floating lens distortions across a large depth range.     

Viewpoint image generation for head tracking 3D display using multi‐camera and approximate depth information

A simple and high image quality method for viewpoint image synthesis from multi‐camera images for a stereoscopic 3D display using head tracking is proposed. In this method, slices of images for depth layers are made using approximate depth information, the slices are linearly blended corresponding to the distance between the viewpoint and cameras at each layer, and the layers are overlaid from the perspective of viewpoint. Because the linear blending automatically compensates for depth error because of the visual effects of depth‐fused 3D (DFD), the resulting image is natural to observer's perception. Smooth motion parallax of wide depth range objects induced by viewpoint movement for left‐and‐right and front‐and‐back directions is achieved using multi‐camera images and approximate depth information. Because the calculation algorithm is very simple, it is suitable for real time 3D display applications.     

Depth‐enhanced integral‐imaging 3D display using different optical path lengths by polarization devices or mirror barrier array

Abstract— Depth‐enhanced integral three‐dimensional (3D) imaging using different optical path lengths by using a polarization selective mirror pair or mirror barrier array is proposed. In the proposed approach, the enhancement of image depth is achieved by repositioning two types of elemental image planes, thus effectively two central depth planes are obtained. One of the two implementation methods makes use of the two‐arm structure that has different optical path lengths and polarization‐selective mirrors. The other utilizes the mirror barrier array. The primary advantage of the method with polarization devices is that we can observe 3D images that maintain some level of viewing resolution with a large depth difference without any mechanical moving part. The mirror barrier array has the advantage of the compact thickness. We demonstrated and verified our proposals experimentally.     

Viewing‐angle‐enhanced integral imaging system using high‐refractive‐index medium and curved micro‐lens array

In the integral imaging (II) system using a curved micro‐lens array (MLA), the viewing angle is limited by the gap mismatch. Here, we propose a system to decrease the gap mismatch for enhancing the viewing angle. In the proposed system, a layer of high‐refractive‐index medium is assembled between the display panel and the curved MLA. The principle of the proposed II system is studied in detail. Simulations based on ray tracing are performed, and the results show that the proposed II system can effectively enhance the viewing angle.     

An integral‐imaging‐based head‐mounted light field display using a tunable lens and aperture array

A head‐mounted light field display based on integral imaging is considered as one of the promising methods that can render correct or nearly correct focus cues and address the well‐known vergence‐accommodation conflict problem in head‐mounted displays. Despite its great potential, it still suffers some of the same limitations of conventional integral imaging‐based displays such as low spatial resolution and crosstalk. In this paper, we present a prototype design using tunable lens and aperture array to render 3D scenes over a large depth range while maintaining high image quality and minimizing crosstalk. Experimental results verify and show that the proposed design could significantly improve the viewing experience.     

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.     

Image modification for immersive projection display based on pseudo‐projection models

This paper describes a practical method that enables actual images to be converted so that they can be projected onto an immersive projection display (IPD) screen. IPD screens are particularly unique in that their angle of view is extremely wide; therefore, the images projected onto them need to be taken on a special format. In practice, however, it is generally very difficult to shoot images that completely satisfy the specifications of the targeting IPD environment due to cost, technical problems or other reasons. To overcome these problems, we developed a method to modify the images by abandoning geometrical consistency. We were able to utilize this method by assuming that the given image was shot according to a special projection model. Because this model differed from the actual projection model with which the image was taken, we termed it the pseudo‐projection model. Since our method uses simple geometry, and can easily be expressed by a parametric function, the degree of modification or the time sequence for modification can readily be adjusted according to the features of each type of content. Copyright © 2003 John Wiley & Sons, Ltd.     

Passivity‐based asymptotic stabilization for switched nonlinear systems using the sampled integral stabilization technique

The asymptotic stabilization problem is studied for a cascade connection of passive switched nonlinear systems and a passive switched nonlinear system in this paper. When each subsystem is asymptotically zero state detectable and passive on active time intervals, asymptotic stabilization is achieved via co‐design of switching laws and controllers without damping injection. First, an output‐feedback controller is designed to asymptotically stabilize a cascade connection of two passive switched systems if outputs are measurable. Second, when the output of the first switched system is noisy or unmeasurable, a sampled integral stabilization (SIS) technique is employed to investigate asymptotical stabilization of a cascade connection by measuring only the storage function of the second switched system. Finally, as a special case of a cascade connection, the SIS technique is used to stabilize a passive switched system without damping injection. Under this circumstance, the controller is designed by sampling the integral of the passive output. The two‐link robot manipulator is provided to illustrate the effectiveness of the SIS technique.     

Depth‐map generation for multi‐view autostereoscopic 3‐D displays based on the SIFT algorithm constrained by boundary

Abstract— A depth‐map estimation method, which converts two‐dimensional images into three‐dimensional (3‐D) images for multi‐view autostereoscopic 3‐D displays, is presented. The proposed method utilizes the Scale Invariant Feature Transform (SIFT) matching algorithm to create the sparse depth map. The image boundaries are labeled by using the Sobel operator. A dense depth map is obtained by using the Zero‐Mean Normalized Cross‐Correlation (ZNCC) propagation matching method, which is constrained by the labeled boundaries. Finally, by using depth rendering, the parallax images are generated and synthesized into a stereoscopic image for multi‐view autostereoscopic 3‐D displays. Experimental results show that this scheme achieves good performances on both parallax image generation and multi‐view autostereoscopic 3‐D displays.     

Ambient‐light‐adaptive image quality enhancement for full‐color e‐paper displays using a saturation‐based tone‐mapping method

Full‐color e‐paper displays that have a small color gamut encounter image tone distortions and contour artifacts in high‐saturation regions. To solve these issues, in this paper, we comprehensively measure the colorimetric responses of input image signals and demonstrate that the perceptual colorfulness in terms of chroma abnormally declines when the input saturation is beyond a certain value. Accordingly, saturation‐based tone‐mapping curves are developed to map the abnormal high saturation to the range that the display is able to normally render. By testing several test charts and natural images, the recoveries of the original image tones and the removals of the contour artifacts are experimentally verified. More important, by discussing how to conduct a database of tone‐mapping curves for different ambient lights and utilize the database for an unknown ambient light, the proposed method is proven to be completely ambient light adaptive.